Systems, devices, and methods for gesture identification

Patent metadata

Field	Value
Publication number	US-9880632-B2
Application number	US-201514737081-A
Country	US
Kind code	B2
Filing date	Jun 11, 2015
Priority date	Jun 19, 2014
Publication date	Jan 30, 2018
Grant date	Jan 30, 2018

Abstract

Systems, devices, and methods adapt established concepts from natural language processing for use in gesture identification algorithms. A gesture identification system includes sensors, a processor, and a non-transitory processor-readable memory that stores data and/or instructions for performing gesture identification. A gesture identification system may include a wearable gesture identification device. The gesture identification process involves segmenting signals from the sensors into data windows, assigning a respective “window class” to each data window, and identifying a user-performed gesture based on the corresponding sequence of window classes. Each window class exclusively characterizes at least one data window property and is analogous to a “letter” of an alphabet. Under this model, each gesture is analogous to a “word” made up of a particular combination of window classes.

First claim

The invention claimed is:

1. A method of operating a gesture identification system to identify a user-performed gesture, the gesture identification system including at least one sensor responsive to user-performed gestures and a processor communicatively coupled to the at least one sensor, the method comprising:
providing at least one signal from the at least one sensor to the processor;
segmenting the at least one signal into data windows;
for each i th data window in at least a subset of the data windows:
determining a window class for the i th data window by the processor, the window class selected by the processor from a library of window classes, wherein each window class in the library of window classes exclusively characterizes at least one data window property;
determining, by the processor, a respective probability that each gesture in a gesture library is the user-performed gesture based on a) the window class for the i th data window and, when i>1, b) the window class for at least one j th data window, where j<i; and
identifying a highest-probability gesture for the i th data window by the processor, the highest-probability gesture corresponding to the gesture in the gesture library that has a highest probability of being the user-performed gesture for the i th data window; and

identifying the user-performed gesture by the processor based on the respective highest-probability gestures for at least two data windows in the at least a subset of data windows.

2. The method of claim 1 wherein providing at least one signal from the at least one sensor to the processor includes providing at least one substantially continuous data stream from the at least one sensor to the processor, and wherein segmenting the at least one signal into data windows includes segmenting the at least one substantially continuous data stream into data windows in real-time by the processor.

3. The method of claim 1 wherein each window class in the library of window classes exclusively characterizes a respective range of values for the same at least one data window property.

4. The method of claim 3 wherein each window class in the library of window classes exclusively characterizes a respective range of values for at least one Root Mean Square (“RMS”) value for the i th data window.

5. The method of claim 1 wherein the at least one sensor includes a plurality of sensors, and wherein:
providing at least one signal from the at least one sensor to the processor includes providing a respective signal from each respective sensor in the plurality of sensors to the processor; and
segmenting the at least one signal into data windows includes segmenting the respective signal from each respective sensor in the plurality of sensors into the data windows, wherein each data window includes a respective portion of the signal from each respective sensor in the plurality of sensors.

6. The method of claim 5 wherein the plurality of sensors includes N sensors, and wherein each window class in the library of window classes exclusively characterizes a respective region in an N-dimensional hyperspace and each dimension of the N-dimensional hyperspace represents a property of the signal from a respective one of the N sensors.

7. The method of claim 6 wherein each region of the N-dimensional hyperspace represents a respective combination of ranges for Root Mean Square (“RMS”) values of the signals from the N sensors.

8. The method of claim 6 wherein, for each window class in the library of window classes, the corresponding region in the N-dimensional hyperspace is exclusively characterized by at least one respective angle formed in the N-dimensional hyperspace.

9. The method of claim 1 wherein the at least one sensor includes at least one muscle activity sensor selected from the group consisting of: an electromyography (EMG) sensor and a mechanomyography (MMG) sensor, and wherein:
providing at least one signal from the at least one sensor to the processor includes providing at least one signal from the at least one muscle activity sensor to the processor.

10. The method of claim 1 wherein the at least one sensor includes at least one inertial sensor selected from the group consisting of: an accelerometer, a gyroscope, and an inertial measurement unit (IMU), and wherein:
providing at least one signal from the at least one sensor to the processor includes providing at least one signal from the at least one inertial sensor to the processor.

11. The method of claim 1 wherein the gesture identification system further comprises a non-transitory processor-readable storage medium that stores processor-executable gesture identification instructions, the non-transitory processor-readable storage medium communicatively coupled to the processor, and the method further comprising:
executing by the processor the gesture identification instructions to cause the processor to:
determine the window class for each i th data window;
determine the respective probability that each gesture in the gesture library is the user-performed gesture for each i th data window;
identify the highest-probability gesture for each i th data window; and
identify the user-performed gesture.

12. The method of claim 1 wherein determining, by the processor, a respective probability that each gesture in a gesture library is the user-performed gesture includes:
determining, by the processor, at least one respective n-gram transition model for each gesture in the gesture library; and
determining, by the processor and for each gesture in the gesture library, a probability that the gesture is the user-performed gesture based at least in part on the at least one n-gram transition model for the gesture.

13. The method of claim 12 wherein determining, by the processor, at least one respective n-gram transition model for each gesture in the gesture library includes determining, by the processor and for each gesture in the gesture library, at least one respective n-gram transition model selected from the group consisting of: a unigram transition model based on the window class of the i th data window and a bigram transition model based on the respective window classes of the i th data window and the j th =(i−1) th data window.

14. The method of claim 1 wherein, for each i th data window, determining, by the processor, a respective probability that each gesture in a gesture library is the user-performed gesture includes determining, by the processor, a respective probability that each gesture in the gesture library is the user-performed gesture based on both a) the window class for the i th data window and b) the window class for a single j th data window, where j is selected from the group consisting of: j=(i−1) and j<(i−1).

15. A gesture identification system comprising:
at least one sensor responsive to physical gestures performed by a user of the gesture identification system, wherein in response to a physical gesture performed by the user the at least one sensor provides at least one signal;
a processor communicatively coupled to the at least one sensor; and
a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable gesture identification instructions that, when executed by the processor, cause the gesture identification device to:
segment the at least one signal into data windows;
for each i th data window in at least a subset of the data windows:
determine a window class for the i th data window, the window class selected from a l

Assignees

• Thalmic Labs Inc

Inventors

• Ataee Pedram
• Aleem Idris S

Classifications

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/015

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title

• G06F2218/12

  Classification; Matching · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title