Motor vehicle control interface with gesture recognition

Patent metadata

Field	Value
Publication number	US-9738158-B2
Application number	US-201414901436-A
Country	US
Kind code	B2
Filing date	Jun 25, 2014
Priority date	Jun 29, 2013
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

A method for operating a motor vehicle control panel having a camera system and a gesture recognition device. includes filming a person gesticulating using at least one arm in the inside of the vehicle. Description data relating to a position and/or a sequence of movements of the at least one gesticulating arm is determined by the gesture recognition device and is associated with a control gesture. At least one situation parameter, which describes a gesticulation context of the person, is determined by a plausibility device which decides whether the person has performed a possible control gesture or only a gesture which is to be ignored. If it is determined that it is a possible control gesture, a control command is generated for the control gesture.

First claim

The invention claimed is:

1. A method for operating a user interface in a motor vehicle, wherein the user interface comprises a camera, the method comprising:
filming a user who is gesticulating with a gesticulating arm, within an interior of the motor vehicle by the user being filmed by the camera to thereby produce image data;
generating descriptive data of a position or movement of the gesticulating arm by a gesture recognition device, based on the image data produced by the camera;
recognizing an operating gesture by associating the descriptive data with one of a plurality of predefined operating gestures based on a first classification;
determining from a displacement history using the image data, the displacement history relating to a prior movement process of the gesticulating arm;
determining, using a plausibility verification device of the user interface, at least one situation parameter that includes a current position of the gesticulating arm and describes a gesticulation context of the user;
determining, based on how a current movement process of the gesticulating arm relates to the displacement history, the at least one situation parameter, and a second classification, whether the user is performing a possible operating gesture that the user intends to be recognized or is performing a gesture that the user does not intend to be recognized and that is to be ignored;
outputting a control command to a receiver for a motor vehicle system depending on the operating gesture recognized based on the first classification if it is determined that the user is performing the possible operating gesture that the user intends to be recognized; and
inhibiting the output of the control command if it is determined that the gesture is to be ignored,
wherein the gesture recognition device and the plausibility verification device include at least one processor.

2. A method for operating a user interface in a motor vehicle, wherein the user interface comprises a camera, the method comprising:
filming a user who is gesticulating with a gesticulating arm, within an interior of the motor vehicle by the user being filmed by the camera to thereby produce image data;
generating descriptive data of a position or movement of the gesticulating arm by a gesture recognition device, based on the image data produced by the camera;
recognizing an operating gesture by associating the descriptive data with one of a plurality of predefined operating gestures based on a first classification;
determining, using a plausibility verification device of the user interface, at least one situation parameter that describes a gesticulation context of the user;
determining, using the at least one situation parameter and a second classification, whether the user is performing a possible operating gesture or only a gesture that is not defined for operation and that is to be ignored;
outputting a control command to a receiver for a motor vehicle system depending on the operating gesture recognized based on the first classification if it is determined that the user is performing a possible operating gesture; and
inhibiting the output of the control command if it is determined that the gesture is to be ignored,
wherein the at least one situation parameter includes at least one of:
a speed of at least one movable part or joint of the gesticulating arm,
an acceleration of the at least one movable part or joint of the gesticulating arm,
an indication as to whether the at least one movable part or joint of the gesticulating arm has been moved with a jerky movement, by which a change of position within a predefined time is greater than a predetermined maximum change of position,
an angle of the at least one movable part or joint of the gesticulating arm relative to at least one other movable part or joint, and
a location of each movable part or joint of the gesticulating arm,

wherein:
if the at least one situation parameter includes the speed of the at least one movable part or joint of the gesticulating arm, it is determined that the user is performing the possible operating gesture only if each speed lies within a respective predefined range of speed values,
if the at least one situation parameter includes the acceleration of the at least one movable part or joint of the gesticulating arm it is determined that the user is performing the possible operating gesture only if each acceleration lies within a respective predefined range of acceleration values,
if the at least one situation parameter indicates the jerky movement, it is determined that the user is performing a gesture to be ignored,
if the at least one situation parameter includes the angle of the at least one movable part or joint of the gesticulating arm relative to at least one other movable part or joint, it is determined that the user is performing the possible operating gesture only if each angle lies within a respective predefined range of angle values, and
if the at least one situation parameter includes the location of each movable part or joint of the gesticulating arm, it is determined that the user is performing the possible operating gesture if each location lies within a respective predefined spatial region.

3. The method as claimed in claim 1 , wherein
the at least one situation parameter comprises an acceleration of at least one movable part or joint of the at least one arm, and
it is determined that the user is performing a possible operating gesture only if each acceleration lies within a respective predefined range of acceleration values.

4. The method as claimed in claim 1 , wherein
the at least one situation parameter indicates whether at least one movable part or joint of the gesticulating arm has been moved with a jerky movement, by which a change of position within a predefined time is greater than a predetermined maximum change of position, and
if the at least one situation parameter indicates the jerky movement, it is determined that the user is performing a gesture to be ignored.

5. The method as claimed in claim 1 , wherein
the at least one situation parameter comprises an angle of at least one movable part or joint of the gesticulating arm relative to at least one other movable part or joint, and
it is determined that the user is performing a possible operating gesture only if each angle lies within a respective predefined range of angle values.

6. The method as claimed in claim 1 , wherein
the at least one situation parameter comprises a location of each movable part or joint of the gesticulating arm, and
it is determined that the user is performing a possible operating gesture if each location lies within a respective predefined spatial region.

7. The method as claimed in claim 1 , wherein the control command is output without the user first activating a dedicated gesture activation button in the motor vehicle after the motor vehicle has been started.

8. The method as claimed in claim 1 , wherein
the displacement history relates to a prior movement process of the gesticulating arm, and
the displacement history is assembled so as to specify whether the prior movement process of the gesticulating arm was classified based on the first classification and caused an uninhibited control command to be output.

9. The method as claimed in claim 1 , wherein the at least one situation parameter is determined from the image data.

10. The method as claimed in claim 1 , wherein to suppress a defined control command, an associated operating gesture that will trigger the defined control command is excluded from the first classification.

11. The method as claimed in claim 1 , wherein the camera is at least one of: a stere

Assignees

• Audi Ag

Inventors

• Schlittenbauer Michael
• Roehder Martin
• Kuehne Marcus
• Bohrer Lorenz

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• B60K37/06 Primary

  Operations & Transport · mapped topic

• B60K2350/1052

  Operations & Transport · mapped topic

• B60K35/10 Primary

  Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor · CPC title

• B60K2360/146

  Instrument input by gesture · CPC title