Dynamic hand gesture recognition with selective enabling based on detected hand velocity

Patent metadata

Field	Value
Publication number	US-2015269425-A1
Application number	US-201414357894-A
Country	US
Kind code	A1
Filing date	Apr 18, 2014
Priority date	Oct 17, 2013
Publication date	Sep 24, 2015
Grant date	—

Abstract

An image processing system comprises an image processor configured to determine velocity of a hand in a plurality of images, and to selectively enable dynamic gesture recognition for at least one image responsive to the determined velocity. By way of example, the image processor illustratively includes a dynamic gesture preprocessing detector and a dynamic gesture recognizer, with the dynamic gesture preprocessing detector being configured to determine the velocity of the hand for a current frame and to compare the determined velocity to a specified velocity threshold. If the determined velocity is greater than or equal to the velocity threshold, the dynamic gesture recognizer operates on the current frame, and otherwise the dynamic gesture recognizer is bypassed for the current frame. The dynamic gesture recognizer when enabled is configured to generate similarity measures for respective ones of a plurality of gestures of a gesture vocabulary for the current frame.

First claim

What is claimed is:

1 . A method comprising:
determining velocity of a hand in a plurality of images; and selectively enabling dynamic gesture recognition for at least one image responsive to the determined velocity; wherein the determining and selectively enabling are implemented in an image processor comprising a processor coupled to a memory.

2 . The method of claim 1 wherein determining velocity comprises determining an average velocity of the hand utilizing a current frame and at least one previous frame.

3 . The method of claim 1 wherein selectively enabling dynamic gesture recognition for at least one image comprises selectively enabling dynamic gesture recognition for the current frame.

4 . The method of claim 1 wherein selectively enabling dynamic gesture recognition for at least one image comprises:
comparing the determined velocity for a current frame to a specified velocity threshold;
if the determined velocity is greater than or equal to the velocity threshold, executing the dynamic gesture recognition for the current frame; and
if the determined velocity is less than the velocity threshold, bypassing the dynamic gesture recognition for the current frame.

5 . The method of claim 1 wherein the dynamic gesture recognition comprises generating similarity measures for respective ones of a plurality of gestures of a gesture vocabulary for a current frame.

6 . The method of claim 5 wherein the similarity measures comprise respective negative log likelihoods.

7 . The method of claim 5 wherein the dynamic gesture recognition further comprises:
selecting a particular one of the similarity measures based on one or more specified criteria; and
controlling generation of a gesture recognition event based on comparison of the selected similarity measure to a specified distance threshold.

8 . The method of claim 5 further comprising:
grouping the gestures of the gesture vocabulary into classes comprising respective non-intersecting and non-empty subsets of the gestures based on hand shape; and
training classifiers for the respective classes.

9 . The method of claim 8 wherein the dynamic gesture recognition further comprises:
generating static hand pose similarity measures for the current frame using the respective classifiers;
evaluating dynamic hand parameters for the current frame; and
utilizing the static hand pose similarity measures in conjunction with the dynamic hand parameters to generate the similarity measures for the respective gestures for the current frame.

10 . The method of claim 9 wherein utilizing the static hand pose similarity measures in conjunction with the dynamic hand parameters comprises:
applying the static hand pose similarity measures and the dynamic hand parameters to gesture detectors configured to detect respective gestures of the gesture vocabulary;
for those gesture detectors that generate an affirmative detection output, enabling a corresponding gesture estimator to generate a preliminary similarity measure for the corresponding gesture; and
processing multiple preliminary similarity measures for respective ones of the gesture estimators utilizing historical information comprising static hand pose similarity measures and preliminary similarity measures from one or more previous frames in order to generate the similarity measures for the respective gestures for the current frame.

11 . A non-transitory computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in the image processor causes the image processor to perform the method of claim 1 .

12 . An apparatus comprising:
an image processor comprising image processing circuitry and an associated memory; wherein the image processor is configured to determine velocity of a hand in a plurality of images, and to selectively enable dynamic gesture recognition for at least one image responsive to the determined velocity.

13 . The apparatus of claim 12 wherein the image processor comprises:
a dynamic gesture preprocessing detector; and
a dynamic gesture recognizer;
wherein the dynamic gesture preprocessing detector is configured to determine the velocity of the hand for a current frame and to compare the determined velocity to a specified velocity threshold;
wherein if the determined velocity is greater than or equal to the velocity threshold, the dynamic gesture recognizer operates on the current frame; and
if the determined velocity is less than the velocity threshold, the dynamic gesture recognizer is bypassed for the current frame.

14 . The apparatus of claim 13 wherein the dynamic gesture recognizer is configured to generate similarity measures for respective ones of a plurality of gestures of a gesture vocabulary for a current frame.

15 . The apparatus of claim 14 wherein the image processor further comprises:
a selection element configured to select a particular one of the similarity measures based on one or more specified criteria; and
a dynamic gesture postprocessing detector configured to control generation of a gesture recognition event based on comparison of an output of the selection element to a specified distance threshold.

16 . The apparatus of claim 14 wherein the dynamic gesture recognizer comprises a static hand pose estimator implementing a plurality of classifiers trained for respective classes comprising respective non-intersecting and non-empty subsets of the gestures, with the gestures being grouped into the classes based on hand shape.

17 . The apparatus of claim 16 wherein the dynamic gesture recognizer further comprises:
a dynamic hand parameter evaluator; and
a plurality of gesture detectors configured to detect respective gestures of the gesture vocabulary;
the gesture detectors being configured to receive static hand pose similarity measures for the current frame from the respective classifiers of the static hand pose estimator;
the gesture detectors being further configured to receive dynamic hand parameters for the current frame from the dynamic hand parameter evaluator;
wherein the gesture detectors generate respective outputs indicating whether or not the corresponding gesture is detected in the current frame.

18 . The apparatus of claim 17 wherein the dynamic gesture recognizer further comprises:
a plurality of gesture estimators for respective ones of the gestures;
each of said gesture estimators having an input coupled to an output of a corresponding one of the gesture detectors;
wherein the gesture estimators are selectively enabled responsive to affirmative outputs from respective ones of the gesture detectors.

19 . The apparatus of claim 18 wherein the dynamic gesture recognizer further comprises an additional estimator having inputs coupled to outputs of respective ones of the gesture estimators, the additional estimator being configured to processing multiple preliminary similarity measures for respective enabled ones of the gesture estimators utilizing historical information comprising static hand pose similarity measures and preliminary similarity measures from one or more previous frames in order to generate the similarity measures for the respective gestures for the current frame.

20 . An integrated circuit comprising the apparatus of claim 12 .

21 . An image processing system comprising the apparatus of claim 12 .

Assignees

• Lsi Corp

Inventors

• Mazurenko Ivan L
• Brickner Barrett
• Petyushko Alexander A
• Parkhomenko Denis V
• Kholodenko Alexander B

Classifications

• G06V40/28 Primary

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title