Electrical device for hand gestures detection

Patent metadata

Field	Value
Publication number	US-2017192513-A1
Application number	US-201514985728-A
Country	US
Kind code	A1
Filing date	Dec 31, 2015
Priority date	Dec 31, 2015
Publication date	Jul 6, 2017
Grant date	—

Abstract

Hand gesture detection electrical device for detecting hand gestures, comprising an IC electronically integrating:
(a) First interface connecting to imaging device(s). (b) Second interface connecting to controlled unit. (c) Data storage storing sequential logic models representing a hand gestures. The sequential logic models map a sequence of pre-defined hand poses and/or motions. (d) Memory storing code. (e) Processor(s) coupled to the first and second interfaces, data storage and memory for executing the code to:
(1) Receive timed images depicting a user's moving hand. (2) Generate a runtime sequence mapping runtime hand datasets each defined by discrete hand values indicating current state of the moving hand. (3) Estimate which hand gesture(s) best match the runtime sequence by optimizing the runtime sequence compared to the sequential logic models using SSVM functions. (4) Initiate action(s) to the controlled unit. The action(s) are associated with selected hand gesture(s) based on the estimation.

First claim

What is claimed is:

1 . A hand gesture detection electrical device for detecting hand gestures, comprising:
an integrated circuit (IC) electronically integrating the following: a first interface connecting to at least one imaging device; a second interface connecting to a controlled unit; a data storage storing a plurality of sequential logic models each representing one of a plurality of hand gestures, said sequential logic model maps a pre-defined sequence of at least one of a plurality of pre-defined hand poses and pre-defined hand motions; a memory storing a code; at least one processor coupled to said first interface, said second interface, said data storage and said memory for executing said stored code, said code comprising: code instructions to receive at least one of a plurality of timed images depicting a moving hand of a user; code instructions to generate a runtime sequence mapping at least one of a plurality of runtime hand datasets representing said moving hand; code instructions to estimate which one of said plurality of hand gestures best matches said runtime sequence through an optimization of said runtime sequence compared to each of said plurality of sequential logic models using at least one structured support vector machine (SSVM) function; and code instructions to initiate at least one action to said controlled unit, said at least one action is associated with a selected at least one of said plurality of hand gestures, said selection is based on said estimation.

2 . The hand gestures detection electrical device of claim 1 , wherein said hand gestures detection electrical device is a member selected from a group consisting of: IC, application specific integrated circuit (ASIC), system on chip (SOC) and an intellectual property (IP) module, said IP module is integrated in another IC.

3 . The hand gestures detection electrical device of claim 1 , wherein said at least one imaging device is a member selected from a group consisting of: a camera, an infrared (IR) camera, a stereo camera and a depth camera.

4 . The hand gestures detection electrical device of claim 1 , further comprising said first interface utilizes at least one interconnecting medium, said at least one interconnecting medium is a member selected of a group consisting of: IC internal interconnects, printed circuit board (PCB) traces, wired connectivity and wireless connectivity.

5 . The hand gestures detection electrical device of claim 1 , further comprising said second interface utilizes at least one interconnecting medium, said at least one interconnecting medium is a member selected of a group consisting of: IC internal interconnects, printed circuit board (PCB) traces, wired connectivity, wireless connectivity.

6 . The hand gestures detection electrical device of claim 1 , further comprising said at least one imaging device is integrated in said hand gesture detection electrical device.

7 . The hand gestures detection electrical device of claim 1 , further comprising at least one of: said data storage and said memory is utilized by an external device to said hand gesture detection electrical device.

8 . The hand gestures detection electrical device of claim 1 , further comprising said at least one SSVM function is executed by a dedicated electrical circuit integrated in said hand gesture detection electrical device.

9 . The hand gestures detection electrical device of claim 1 , wherein said code further comprising code instructions to transmit an indication of said selected at least one hand gesture to a host apparatus connected to said hand gestures detection electrical device through at least one of: said first interface and said second interface.

10 . The hand gestures detection electrical device of claim 1 , wherein said code further comprising code instructions to manipulate said at least one timed image to remove at least one non-relevant image portion.

11 . The hand gestures detection electrical device of claim 1 , wherein said code further comprising code instructions to scale said moving hand depicted in said at least one timed image.

12 . The hand gestures detection electrical device of claim 1 , wherein:
each of said plurality of pre-defined hand poses and hand motions is represented by one of a plurality of pre-defined hand features records each defined by at least some of a plurality of discrete hand values each indicating a state of a respective one of a plurality of hand features of a reference hand; and each one of said plurality of runtime hand datasets is defined by at least some of a plurality of discrete hand values scores each indicating a current state of a respective one of said plurality of hand features of said moving hand, said plurality of discrete hand values scores is inferred by said code instruction by analyzing said at least one timed image.

13 . The hand gestures detection electrical device of claim 12 , wherein each one of said plurality of discrete hand values is represented by a Boolean formula which is defined in the form of a Conjunctive Normal Form (CNF).

14 . The hand gestures detection electrical device of claim 1 , wherein said code instructions apply a plurality of hand feature classifying functions on said at least one timed image, each of said plurality of hand feature classifying functions outputs a current discrete hand value score of a respective one of a plurality of hand features of said moving hand;
wherein said plurality of hand feature classifying functions are trained statistical classifiers.

15 . The hand gestures detection electrical device of claim 1 , wherein said code further comprising code instructions to identify an in-plane rotation of said moving hand, said in-plane rotation is identified by applying a plurality of in-plane rotation classifying functions on said at least one timed image, said plurality of in-plane rotation classifying functions is selected according to a selected global orientation category (GOC) of said moving hand.

16 . The hand gestures detection electrical device of claim 15 , wherein said GOC is selected by applying a plurality of GOC classifying functions on at least one timed image, said plurality of GOC classifying functions are trained statistical classifiers.

17 . The hand gestures detection electrical device of claim 15 , wherein said plurality of in-plane rotation classifying functions are trained statistical classifiers.

18 . The hand gestures detection electrical device of claim 15 , wherein said code further comprising code instructions to align said moving hand depicted in said at least one timed image after identifying said in-plane rotation.

19 . The hand gestures detection electrical device of claim 15 , wherein said code further comprising code instructions to identify a center of hand of said moving hand prior to selecting said GOC, said center of hand is derived from a center of mass of said moving hand, said center of mass is identified by analyzing an image data available from at least one image depicting said moving hand.

20 . The hand gestures detection electrical device of claim 19 , wherein said code instructions to identify said center of hand apply a plurality of center of hand classifying functions on said at least one timed image, said plurality of center of hand classifying functions are trained statistical classifiers.

21 . The hand gestures detection electrical device of claim 1 , further comprising said sequential logic model is represented as a fini

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Karmon Kfir
• Krupka Eyal
• Diamant Adi

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V40/113

  Recognition of static hand signs · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title