Method and device for detecting a user input on the basis of a gesture

Patent metadata

Field	Value
Publication number	US-11430267-B2
Application number	US-201816625310-A
Country	US
Kind code	B2
Filing date	Jun 18, 2018
Priority date	Jun 20, 2017
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

A method which includes detecting a user input based on a gesture, acquiring image data and performing segmentation based on the acquired image data, and determining an object with an outline. A reference point is determined based on the object, a distance between each of a number of points located on the outline of the object and the reference point is determined and a measured distance profile is generated based on these distances. A gesture is determined based on the measured distance profile and an output signal is generated and output based on the gesture determined. A device for detecting a user input based on a gesture including an acquisition unit, a segmentation unit, a profile computing unit, an allocation unit, and an output unit (8).

First claim

The invention claimed is:

1. A method for detecting a user input based on a gesture, the method comprising:
capturing image data;
using the captured image data to perform a segmentation, wherein an object with an outline is determined;
using the object to determine a reference point;
determining a distance from the reference point for a multiplicity of points on the outline of the object and using the distances to generate a measured distance profile, the measured distance profile comprising the distance of the outline of the object to the reference point;
using the measured distance profile to determine a gesture; and
using the gesture determined to generate and output an output signal.

2. The method of claim 1 , wherein the captured image data comprise an at least partial depiction of a hand and the gesture corresponds to a number of extended fingers of the hand.

3. The method of claim 1 , wherein the captured image data comprise picture elements and the picture elements have associated distance information.

4. The method of claim 1 , wherein an angle of inclination of the object is determined and an equalizing transformation is performed for the object.

5. The method of claim 1 , wherein the reference point is the geometric centroid of the object.

6. The method of claim 1 , wherein a surface area of the object is determined and the measured distance profile is normalized based on the determined surface area.

7. The method of claim 1 , wherein geometric profile features of the measured distance profile are determined and the gesture is determined based on the geometric profile features.

8. The method of claim 1 , wherein the gesture is determined based on a profile comparison in which the measured distance profile is compared with a multiplicity of reference distance profiles, wherein the reference distance profiles each have an associated gesture.

9. The method of claim 8 , wherein geometric reference features are determined for the reference distance profiles and the profile comparison is performed based on the reference features and the profiles features of the measured distance profile.

10. The method of claim 8 , wherein at least two extreme values of the measured distance profile are determined and the profile comparison is performed based on the determined extreme values of the measured distance profile.

11. The method of claim 8 , wherein a first derivative of the distance profile is determined and the profile comparison is performed based on the determined derivative.

12. The method of claims of claim 8 , wherein the profile comparison is performed based on a machine learning method.

13. The method of claim 1 , wherein the captured image data comprise a series of images and a series of measured distance profiles is produced, the gesture being determined based on the series of measured distance profiles.

14. The method of claim 13 , wherein the gesture comprises a movement of a hand.

15. An apparatus for detecting a user input based on a gesture, the apparatus comprising:
a capture unit, by which image data are capturable;
a segmentation unit, by which a segmentation is carried out based on the captured image data, it being possible to determine an object with an outline;
a profile calculation unit, by which a reference point is determinable based on the object, a distance from the reference point being respectively determinable for a multiplicity of points on the outline of the object and a measured distance profile being producible based on the distances the measured distance profile comprising the distance of the outline of the object to the reference point;
an association unit, by which a gesture is determinable based on the measured distance profile; and
an output unit, by which an output signal is generated based on the determined gesture and is output.

16. The apparatus of claim 15 , wherein the captured image data comprise an at least partial depiction of a hand and the gesture corresponds to a number of extended fingers of the hand.

17. The apparatus of claim 15 , wherein the captured image data comprise picture elements and the picture elements have associated distance information.

18. The apparatus of claim 15 , wherein an angle of inclination of the object is determined and an equalizing transformation is performed for the object.

19. The apparatus of claim 15 , wherein the reference point is the geometric centroid of the object.

20. The apparatus of claim 15 , wherein a surface area of the object is determined and the measured distance profile is normalized based on the determined surface area.

21. The apparatus of claim 15 , wherein geometric profile features of the measured distance profile are determined and the gesture is determined based on the geometric profile features.

22. The apparatus of claim 15 , wherein the gesture is determined based on a profile comparison in which the measured distance profile is compared with a multiplicity of reference distance profiles, wherein the reference distance profiles each have an associated gesture.

23. The apparatus of claim 22 , wherein geometric reference features are determined for the reference distance profiles and the profile comparison is performed based on the reference features and the profiles features of the measured distance profile.

24. The apparatus of claim 22 , wherein at least two extreme values of the measured distance profile are determined and the profile comparison is performed based on the determined extreme values of the measured distance profile.

25. The apparatus of claim 22 , wherein a first derivative of the distance profile is determined and the profile comparison is performed based on the determined derivative.

26. The apparatus of claim 22 , wherein the profile comparison is performed based on a machine learning method.

27. The apparatus of claim 15 , wherein the captured image data comprise a series of images and a series of measured distance profiles is produced, the gesture being determined based on the series of measured distance profiles.

28. The apparatus of claim 15 , wherein the gesture comprises a movement of a hand.

Assignees

• Volkswagen Ag

Inventors

• Ette Bernd
• Wintsche Volker
• Gaida Christian

Classifications

• G06T7/11

  Region-based segmentation · CPC title

• G06V40/28 Primary

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T2207/30196

  Human being; Person · CPC title