Determination of hand dimensions for hand and gesture recognition with a computing interface

Patent metadata

Field	Value
Publication number	US-10649536-B2
Application number	US-201514951370-A
Country	US
Kind code	B2
Filing date	Nov 24, 2015
Priority date	Nov 24, 2015
Publication date	May 12, 2020
Grant date	May 12, 2020

Abstract

Hand dimensions are determined for hand and gesture recognition with a computing interface. An input sequence of frames is received from a camera. Frames of the sequence are identified in which a hand is recognized. Points are identified in the identified frames corresponding to features of the recognized hand. A value is determined for each of a set of different feature lengths of the recognized hand using the identified points for each identified frame. Each different feature length value is collected for the identified frames independently of each other feature length value. Each different feature length value is analyzed to determine an estimate of each different feature length, and the estimated feature lengths are applied to a hand tracking system, the hand tracking system for applying commands to a computer system.

First claim

What is claimed is:

1. A method for recognizing hand gestures, the method comprising:
receiving an input sequence of frames from a camera;
identifying frames of the sequence in which a first hand is recognized;
(A) identifying points in the identified frames corresponding to features of the recognized first hand;
(B determining a value for each of a first set of different feature lengths of the recognized first hand using the identified points for each identified frame, wherein a different value is determined for a same feature length in different ones of the identified frames and different ones of the identified frames being used for different feature lengths when at least one of the identified frames having noise, one or more distortions or one or more occlusions obscuring one or more feature points in the recognized first hand thereby making the at least one identified frame unreliable for use in determining feature lengths of the one or more feature points that are obscured;
(C) collecting each different feature length value for the identified frames independently of each other feature length value, providing multiple different values for the same feature length from different frames;
(D) statistically analyzing each different feature length value to determine an estimate of each different feature length independently of determining each other feature length, including independently analyzing each different feature length value to determine whether the estimate of each different feature length of the first hand is to be generated; and
applying the estimated feature lengths to a hand tracking system after confidence as to the accuracy of each of the estimated feature lengths is determined to reach a satisfactory level, the hand tracking system for applying commands to a computer system;
determining when a second hand enters a first frame of the input sequence, wherein the second hand enters subsequent to the first hand exiting the input sequence;
measuring feature lengths for the second hand using one to three frames in which the second hand is present;
comparing feature length measurements for the second hand to the feature length estimates of the first hand to determine whether the second hand matches the first hand;
if the second hand matches the first hand, then using the corresponding feature length estimates from the first hand for the second hand, without repeating operations (A), (B), (C), or (D) for the second hand; and
if the second hand does not match the first hand, then repeating operations (A), (B), (C), and/or (D) for the second hand, to estimate a second set of different feature lengths of the second hand.

2. The method of claim 1 , wherein the different feature lengths of the first and second sets comprise finger lengths, palm width, and palm length points corresponding to fingertips, finger bases, and a wrist of the recognized hand.

3. The method of claim 1 , wherein the camera is a depth camera and wherein determining a value for the different feature lengths comprises determining a value using a position of the identified points in 3-D space.

4. The method of claim 1 , wherein statistically analyzing comprises independently determining a confidence value for each different feature length using each respective collection of different feature length values and estimating a feature length when the corresponding confidence value meets a threshold.

5. The method of claim 4 , wherein the confidence value comprises a standard deviation.

6. The method of claim 5 , wherein taking an average comprises first removing outliers from the corresponding collected feature length values.

7. The method of claim 4 , wherein estimating comprises taking an average of the collected feature length values for the corresponding feature length.

8. The method of claim 1 , wherein identifying frames comprises identifying frames in which the recognized hand has extended fingers.

9. The method of claim 1 , wherein identifying frames comprises determining a plane of the palm of the recognized hand and identifying frames in which the plane of the palm is within a predetermined angle with respect to the camera.

10. A non-transitory computer-readable medium having instructions thereon that when operated on by a computer causes the computer to perform operations for recognizing hand gestures, the operations comprising:
receiving an input sequence of frames from a camera;
identifying frames of the sequence in which a first hand is recognized;
identifying points in the identified frames corresponding to features of the recognized first hand;
determining a value for each of a first set of different feature lengths of the recognized first hand using the identified points for each identified frame, wherein a different value is determined for a same feature length in different ones of the identified frames and different ones of the identified frames being used for different feature lengths when at least one of the identified frames having noise, one or more distortions or one or more occlusions obscuring one or more feature points in the recognized first hand thereby making the at least one identified frame unreliable for use in determining feature lengths of the one or more feature points that are obscured;
collecting each different feature length value for the identified frames independently of each other feature length value, providing multiple different values for the same feature length from different frames;
statistically analyzing each different feature length value to determine an estimate of each different feature length independently of determining each other feature length, including independently analyzing each different feature length value to determine whether the estimate of each different feature length should be generated, wherein a hand model comprises the estimated feature lengths of the first hand;
applying the estimated feature lengths to a hand tracking system after confidence as to the accuracy of each of the estimated feature lengths is determined to reach a satisfactory level, the hand tracking system for applying commands to a computer system;
determining when a second hand enters the input sequence;
measuring feature lengths for the second hand using a first frame in which the second hand enters;
comparing feature length measurements for the second hand to the feature length estimates of the first hand to determine whether the second hand matches the first hand;
if the second hand matches the first hand, then using the corresponding prior feature length estimates from the first hand for the second hand, without refining the hand-model of the first hand; and
if the second hand does not match the first hand, then refining the hand-model such that the refined hand model corresponds to estimated feature lengths of the second hand.

11. The medium of claim 10 , wherein statistically analyzing comprises independently determining a standard deviation for each different feature length using each respective collection of different feature length values and estimating a feature length when a corresponding confidence value meets a threshold.

12. The medium of claim 10 , wherein identifying frames comprises identifying frames in which the recognized hand has extended fingers.

13. The medium of claim 10 , wherein identifying frames comprises determining a plane of the palm of the recognized hand and identifying frames in which the plane of the palm is within a predetermined angle with respect to the camera.

14. A non-transitory computer-readable medium having instructions thereon that when operated on by a computer causes the

Assignees

• Intel Corp

Inventors

• Lerner Alon
• Fleishman Shahar

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title