Efficient gesture processing

Patent metadata

Field	Value
Publication number	US-9535506-B2
Application number	US-201414205210-A
Country	US
Kind code	B2
Filing date	Mar 11, 2014
Priority date	Jul 13, 2010
Publication date	Jan 3, 2017
Grant date	Jan 3, 2017

Abstract

Embodiments of the invention describe a system to efficiently execute gesture recognition algorithms. Embodiments of the invention describe a power efficient staged gesture recognition pipeline including multimodal interaction detection, context based optimized recognition, and context based optimized training and continuous learning. Embodiments of the invention further describe a system to accommodate many types of algorithms depending on the type of gesture that is needed in any particular situation. Examples of recognition algorithms include but are not limited to, HMM for complex dynamic gestures (e.g. write a number in the air), Decision Trees (DT) for static poses, peak detection for coarse shake/whack gestures or inertial methods (INS) for pitch/roll detection.

First claim

The invention claimed is:

1. At least one non-transitory computer readable storage medium having instructions stored thereon that, when executed on a machine, cause the machine to:
receive data from a motion sensor;
select a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on an amplitude of the data;
determine an energy magnitude of the data based on the amplitude of the data; and
determine a gesture from the data based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data.

2. The at least one computer readable storage medium 1 , wherein the machine is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of a total energy magnitude of the data with a total energy magnitude value associated with each of the plurality of gesture algorithms.

3. The at least one computer readable storage medium of claim 1 , wherein the machine is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of minimum/maximum energy magnitude values of the data with minimum/maximum energy magnitude values associated with each of the plurality of gesture algorithms.

4. The at least one computer readable storage medium of claim 1 , the machine to further:
determine a frequency spectrum of the data based, at least in part, on the amplitude of the data and a phase of the data.

5. The at least one computer readable storage medium of claim 4 , wherein the machine is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of the frequency spectrum of the data with one or more spectrum patterns associated with each of the plurality of gesture algorithms.

6. The at least one computer readable storage medium of claim 1 , wherein the motion sensor comprises at least one of an accelerometer or a gyroscope.

7. A mobile computing device comprising:
a motion sensor;
a memory;
at least one processor;
an algorithm selection module, stored in the memory and executed via the at least one processor, to select a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on an amplitude of a data from the motion sensor and determine an energy magnitude of the data from the motion sensor based on the amplitude of the data from the motion sensor; and
a gesture recognition module, stored in the memory and executed via the at least one processor, to determine a gesture from the data from the motion sensor based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data from the motion sensor.

8. The mobile computing device of claim 7 , wherein the at least one processor comprises a low power processing unit to execute the algorithm selection module, and a main processing unit to execute the gesture recognition module.

9. The mobile computing device of claim 7 , wherein the algorithm selection module is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of a total energy magnitude of the data from the motion sensor with a total energy magnitude value associated with each of the plurality of gesture algorithms.

10. The mobile computing device of claim 7 , wherein the algorithm selection module is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of minimum/maximum energy magnitude values of the data from the motion sensor with minimum/maximum energy magnitude values associated with each of the plurality of gesture algorithms.

11. The mobile computing device of claim 7 , wherein the algorithm selection module is to further:
determine a frequency spectrum of the data from the motion sensor based, at least in part, on the amplitude of the data from the motion sensor and a phase of the data from the motion sensor.

12. The mobile computing device of claim 11 , wherein the algorithm selection module is to select the subset of gesture recognition algorithm(s) is based, at least in part, on a comparison of the frequency spectrum of the data from the motion sensor with one or more spectrum patterns associated with each of the plurality of gesture algorithms.

13. The mobile computing device of claim 7 , wherein the motion sensor comprises an accelerometer.

14. The mobile computing device of claim 7 , wherein the motion sensor comprises a gyroscope.

15. The mobile computing device of claim 7 , further comprising:
a wearable housing including the motion sensor, the at least one processor, and the memory.

16. A machine-implemented method comprising:
receiving data from a motion sensor;
selecting a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on an amplitude of the data;
determining an energy magnitude of the data based on the amplitude of the data; and
determining a gesture from the data based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data.

17. The machine-implemented method of claim 16 , wherein selecting the subset of gesture recognition algorithm(s) is based, at least in part, on at least one of:
comparing a total energy magnitude of the data with a total energy magnitude value associated with each of the plurality of gesture algorithms; or
comparing minimum/maximum energy magnitude values of the data with minimum/maximum energy magnitude values associated with each of the plurality of gesture algorithms.

18. The machine-implemented method of claim 16 , further comprising:
determining a frequency spectrum of the data based, at least in part, on the amplitude of the data and a phase of the data; wherein selecting the subset of gesture recognition algorithm(s) is based, at least in part, on comparing the frequency spectrum of the data with one or more spectrum patterns associated with each of the plurality of gesture algorithms.

Assignees

• Raffa Giuseppe
• Nachman Lama
• Lee Jinwon
• Intel Corp

Inventors

• Raffa Giuseppe
• Nachman Lama
• Lee Jinwon

Classifications

• G01P15/18

  in two or more dimensions · CPC title

• G01C19/00

  Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects · CPC title

• H04M2250/12

  including a sensor for measuring a physical value, e.g. temperature or motion · CPC title

• G06F1/1694

  the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title