System and method for controlling a vehicle user interface based on gesture angle

Patent metadata

Field	Value
Publication number	US-9346471-B2
Application number	US-201414514160-A
Country	US
Kind code	B2
Filing date	Oct 14, 2014
Priority date	Mar 15, 2013
Publication date	May 24, 2016
Grant date	May 24, 2016

Abstract

An in-vehicle computing system allows a user to control components of the vehicle by performing gestures. The user provides a selecting input to indicate that he wishes to control one of the components. After the component is identified, the user performs a gesture to control the component. The gesture and the component that was previously selected are analyzed to generate a command for the component. Since the command is based on both the gesture and the identified component, the user can perform the same gesture in the same position within the vehicle to control different components.

First claim

What is claimed is:

1. A vehicle-based computer-implemented method for controlling a component of the vehicle, the method comprising:
receiving, via a sensor within the vehicle, a first selecting input from a user inside the vehicle, wherein the first selecting input represents a directional gesture performed by the user;
processing the first selecting input to identify and select a first component of the vehicle to be controlled by the user, wherein processing the first selecting input includes determining a line that is generated matching a direction of the directional gesture and finding at least one component whose stored three-dimensional coordinates are closest to the line;
providing, to the user, a first feedback signal confirming selection of the first component to be controlled;
subsequent to providing the first feedback signal, receiving a first data signal for controlling the first component, the first data signal representing a the directional gesture that is directed at the first component inside the vehicle; and
processing the first data signal to determine a first command for controlling the first component.

2. The method of claim 1 , wherein:
the sensor within the vehicle comprises a camera system including one or more cameras positioned to capture the directional gestures performed in a capture region inside the vehicle;
and
processing the first data signal comprises performing gesture recognition on the directional gesture wherein performing gesture recognition on the directional gesture includes determining a positon of a hand of the user, a plane representing an orientation of the hand of the user, and an angle at which each joint of the hand of the user is bent.

3. The method of claim 1 , wherein:
the sensor within the vehicle comprises a microphone configured to detect voice signals within the vehicle;
the first selecting input comprises a voice signal that includes a name of the first component; and
processing the first selecting input comprises performing voice recognition on the voice signal to identify the first component of the vehicle.

4. The method of claim 1 , wherein:
the first feedback signal is an audio feedback signal provided through one or more speakers inside the vehicle; and
the audio feedback signal contains a name of the first component and includes audio instructions for controlling the first component.

5. The method of claim 1 , wherein:
the first feedback signal is a visual feedback signal provided through one or more displays inside the vehicle; and
the visual feedback signal contains a name or an image of the first component and includes textual, graphical, or animation instructions for controlling the first component.

6. The method of claim 1 , further comprising:
subsequent to providing the first feedback signal, providing the user a prompt to verify whether the first identified component was correctly selected based on the first selecting input.

7. The method of claim 6 , further comprising:
responsive to receiving, from the user, an indication that the first component was incorrectly selected, providing the user an additional prompt to enable the user to rectify the selection.

8. A vehicle-based computing system for controlling a component of the vehicle, the system comprising:
one or more processors; and
a non-transitory computer-readable storage medium for storing computer program instructions, the program instructions when executed by the processors cause the processors to perform steps including:
receiving, via a sensor within the vehicle, a first selecting input from a user inside the vehicle, wherein the first selecting input represents a directional gesture performed by the user;
processing the first selecting input to identify and select a first component of the vehicle to be controlled by the user, wherein processing the first selecting input includes determining a line that is generated matching a direction of the directional gesture and finding at least one component whose stored three-dimensional coordinates are closest to the line;
providing, to the user, a first feedback signal confirming selection of the first component to be controlled;
subsequent to providing the first feedback signal, receiving a first data signal for controlling the first component, the first data signal representing a the directional gesture that is directed at the first component inside the vehicle; and
processing the first data signal to determine a first command for controlling the first component.

9. The vehicle-based computing system of claim 8 , wherein:
the sensor within the vehicle comprises a camera system including one or more cameras positioned to capture the directional gestures performed in a capture region inside the vehicle; and
processing the first data signal comprises performing gesture recognition on the directional gesture wherein performing gesture recognition on the directional gesture includes determining a positon of a hand of the user, a plane representing an orientation of the hand of the user, and an angle at which each joint of the hand of the user is bent.

10. The vehicle-based computing system of claim 8 , wherein:
the sensor within the vehicle comprises a microphone configured to detect voice signals within the vehicle;
the first selecting input comprises a voice signal that includes a name of the first component; and
instructions for processing the first selecting input comprise instructions for performing voice recognition on the voice signal to identify the first component of the vehicle.

11. The vehicle-based computing system of claim 8 , wherein:
the first feedback signal is an audio feedback signal provided through one or more speakers inside the vehicle; and
the audio feedback signal contains a name of the first component and includes audio instructions for controlling the first component.

12. The vehicle-based computing system of claim 8 , wherein:
the first feedback signal is a visual feedback signal provided through one or more displays inside the vehicle; and
the visual feedback signal contains a name or an image of the first component and includes textual, graphical, or animation instructions for controlling the first component.

13. The vehicle-based computing system of claim 8 , further comprising instructions for:
subsequent to providing the first feedback signal, providing the user a prompt to verify whether the first component was correctly selected based on the first selecting input.

14. The vehicle-based computing system of claim 13 , further comprising instructions for:
responsive to receiving, from the user, an indication that the first component was incorrectly selected, providing the user an additional prompt to enable the user to rectify the selection.

Assignees

• Honda Motor Co Ltd
• Edge3 Technologies Inc
• Edge 3 Technologies Inc

Inventors

• El Dokor Tarek A
• Cluster Jordan
• Holmes James E
• Vaghefinazari Pedram
• Yamamoto Stuart M

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06K9/00355

  Physics · mapped topic

• B60W50/10 Primary

  Interpretation of driver requests or demands · CPC title