Gesture recognition using gesture elements

Patent metadata

Field	Value
Publication number	US-2016124512-A1
Application number	US-201414548775-A
Country	US
Kind code	A1
Filing date	Nov 20, 2014
Priority date	Oct 29, 2014
Publication date	May 5, 2016
Grant date	—

Abstract

Aspects of the present disclosure provide a gesture recognition method and an apparatus for capturing gesture. The apparatus categorizes the raw data of a gesture into gesture elements, and utilizes the contextual dependency between the gesture elements to perform gesture recognition with a high degree of accuracy and small data size. A gesture may be formed by a sequence of one or more gesture elements.

First claim

What is claimed is:

1 . A method of recognizing gesture operable at an apparatus, comprising:
generating raw data of a gesture from one or more gesture capturing sensors; categorizing the raw data into one or more gesture elements; determining a contextual dependency between the one or more gesture elements; and recognizing the gesture based on the determined gesture elements.

2 . The method of claim 1 , further comprising:
recategorizing the gesture elements based on the contextual dependency between the one or more gesture elements.

3 . The method of claim 2 , wherein the recategorizing the gesture elements comprises:
in a first time interval, categorizing the raw data of a first portion of the gesture to be a first gesture element; and in a second time interval after the first time interval, categorizing the raw data of the first portion of the gesture as a second gesture element based on the contextual dependency of the gesture elements.

4 . The method of claim 1 , wherein the raw data obtained from the gesture capturing sensors, has not been subjected to processing or manipulation related to gesture recognition.

5 . The method of claim 1 , wherein the one or more gesture capturing sensors comprise at least one of a gyroscope, an accelerometer, a camera, a satellite tracker, a motion sensing device, or a position sensing device.

6 . The method of claim 1 , wherein the determining the contextual dependency comprises determining probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence.

7 . The method of claim 6 , wherein the probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence is determined by utilizing a Gaussian Mixture Model.

8 . The method of claim 6 , wherein the probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence is determined by utilizing a deep neural network.

9 . The method of claim 1 , wherein the gesture comprises a non-verbal input received by the apparatus.

10 . The method of claim 1 , wherein the recognizing the gesture comprises determining a gesture in a vocabulary corresponding to the gesture elements.

11 . The method of claim 1 , wherein the categorizing the raw data comprises processing the raw data using a Hidden Markov Model based method to determine the gesture elements.

12 . An apparatus for recognizing gesture, comprising:
one or more gesture capturing sensors; a raw data capture block configured to generate raw data of a gesture from the gesture capturing sensors; a gesture elements categorizing block configured to categorize the raw data into one or more gesture elements; a contextual dependency determining block configured to determine a contextual dependency between the one or more gesture elements; and a gesture recognition block configured to recognize the gesture based on the determined gesture elements.

13 . The apparatus of claim 12 , wherein the gesture elements categorizing block is configured to recategorize the gesture elements based on the contextual dependency between the one or more gesture elements.

14 . The apparatus of claim 13 , wherein the gesture elements categorizing block is configured to:
in a first time interval, categorize the raw data of a first portion of the gesture to be a first gesture element; and in a second time interval after the first time interval, categorize the raw data of the first portion of the gesture as a second gesture element based on the contextual dependency of the gesture elements.

15 . The apparatus of claim 12 , wherein the raw data obtained from the gesture capturing sensors, has not been subjected to processing or manipulation related to gesture recognition.

16 . The apparatus of claim 12 , wherein the one or more gesture capturing sensors comprise at least one of a gyroscope, an accelerometer, a camera, a satellite tracker, a motion sensing device, or a position sensing device.

17 . The apparatus of claim 12 , wherein the contextual dependency determining block is configured to determine probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence.

18 . The apparatus of claim 17 , wherein the probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence is determined by utilizing a Gaussian Mixture Model.

19 . The apparatus of claim 17 , wherein the probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence is determined by utilizing a deep neural network.

20 . The apparatus of claim 12 , wherein the gesture comprises a non-verbal input received by the apparatus.

21 . The apparatus of claim 12 , wherein the gesture recognition block is configured to recognize a gesture in a vocabulary corresponding to the gesture elements.

22 . The apparatus of claim 12 , wherein the gesture elements categorizing block is configured to process the raw data using a Hidden Markov Model based method to determine the gesture elements.

23 . An apparatus for recognizing gesture, comprising:
means for generating raw data of a gesture from one or more gesture capturing sensors; means for categorizing the raw data into one or more gesture elements; means for determining a contextual dependency between the one or more gesture elements; and means for recognizing the gesture based on the determined gesture elements.

24 . The apparatus of claim 23 , further comprising:
means for recategorizing the gesture elements based on the contextual dependency between the one or more gesture elements.

25 . The apparatus of claim 24 , wherein the means for recategorizing the gesture elements is configured to:
in a first time interval, categorize the raw data of a first portion of the gesture to be a first gesture element; and in a second time interval after the first time interval, categorize the raw data of the first portion of the gesture as a second gesture element based on the contextual dependency of the gesture elements.

26 . The apparatus of claim 23 , wherein the means for determining the contextual dependency is configured to determine probabilities of the one or more gesture elements appearing next to each other in a temporal order or sequence.

27 . A computer-readable medium comprising code for causing an apparatus to recognize gesture, the code when executed causes the apparatus to:
generate raw data of a gesture from one or more gesture capturing sensors; categorize the raw data into one or more gesture elements; determine a contextual dependency between the one or more gesture elements; and recognize the gesture based on the determined gesture elements.

28 . The computer-readable medium of claim 27 , wherein the code when executed further causes the apparatus to:
recategorize the gesture elements based on the contextual dependency between the one or more gesture elements.

29 . The computer-readable medium of claim 28 , wherein the code when executed further causes the apparatus to recategorize the gesture elements by:
in a first time interval, categorizing the raw data of a first portion of the gesture to be a first gesture element; and in a second time interval after the f

Assignees

• Qualcomm Inc

Inventors

• Maruya Yoshihisa
• Paddon Michael William
• Duggan Matthew Christian
• Tarui Kento

Classifications

• G06V10/768

  using context analysis, e.g. recognition aided by known co-occurring patterns · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title