Method and apparatus for identifying input features for later recognition

Patent metadata

Field	Value
Publication number	US-9747306-B2
Application number	US-201313901402-A
Country	US
Kind code	B2
Filing date	May 23, 2013
Priority date	May 25, 2012
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

Disclosed are methods and apparatuses to recognize actors during normal system operation. The method includes defining actor input such as hand gestures, executing and detecting input, and identifying salient features of the actor therein. A model is defined from salient features, and a data set of salient features and/or model are retained, and may be used to identify actors for other inputs. A command such as “unlock” may be executed in response to actor input. Parameters may be applied to further define where, when, how, etc. actor input is executed, such as defining a region for a gesture. The apparatus includes a processor and sensor, the processor defining actor input, identifying salient features, defining a model therefrom, and retaining a data set. A display may also be used to show actor input, a defined region, relevant information, and/or an environment. A stylus or other non-human actor may be used.

First claim

We claim:

1. A method, comprising:
defining an actor input for an actor;
defining a command associated with said actor input
defining a region for said actor input;
detecting said actor input in said region;
executing said command in response to said actor input;
identifying at least one structural geometric salient feature of said actor from said actor input associated with said executed command;
defining a structural geometric actor model of said actor from said at least one structural geometric salient feature;
retaining a data set comprising at least one of said at least one structural geometric salient feature or said structural geometric actor model; and
using said data set to identify subsequent actor inputs.

2. The method of claim 1 further comprising:
defining limiting parameters for executing said actor input;
detecting said actor input within said parameters.

3. The method of claim 1 , wherein:
said actor comprises a human vocal system; and
said actor input comprises human speech.

4. The method of claim 1 , wherein:
said actor comprises a human hand; and
said actor input comprises at least one of a hand gesture or a hand posture.

5. The method of claim 1 , further comprising:
establishing an imaging field of view; and
defining a said region in said field of view;
wherein:
detecting said actor input comprises receiving at least one image of said field of view, with said actor input therein and in said region.

6. The method of claim 5 , wherein:
said region is smaller than said field of view, and is substantially fully within said field of view.

7. The method of claim 6 , further comprising:
establishing first and second imaging fields of view;
defining said region in said first imaging field of view;
defining said region in said second imaging field of view;
executing said actor input in said region;
receiving at least one first image of said first imaging field of view, said first image comprising at least a portion of said actor input therein;
receiving at least one second image of said second imaging field of view, said second image comprising at least a portion of said actor input therein; and
identifying at least one structural geometric salient feature of said actor from said actor input in said region in said first and second images.

8. The method of claim 5 , wherein:
said actor comprises an end-effector.

9. The method of claim 8 , wherein:
said end-effector comprises at least one of a hand, a finger, or a stylus.

10. The method of claim 5 , further comprising:
identifying a plurality of structural geometric salient features of said actor from said region in said image.

11. The method of claim 5 , further comprising:
establishing first and second imaging fields of view;
defining a first region in said first imaging field of view;
defining a second region in said second imaging field of view corresponding to said first region in said first imaging field of view;
detecting said actor input in said first and second regions;
receiving at least one first image of said first field of view, said first image comprising at least a portion of said actor input therein;
receiving at least one second image of said second imaging field of view, said second image comprising at least a portion of said actor input therein;
identifying at least one first structural geometric salient feature of said actor from said first region in said first image;
identifying at least one second structural geometric salient feature of said actor from said second region in said second image;
defining said structural geometric actor model of said actor from said at least one first structural geometric salient feature and said at least one structural geometric second salient feature; and
retaining said data set comprising at least one of said at least one first salient feature, said at least one second salient feature, or said actor model.

12. The method of claim 11 , wherein:
said first and second imaging fields of view form a stereo pair.

13. The method of claim 5 , wherein:
said actor input comprises a substantially static posture of an end-effector.

14. The method of claim 5 , wherein:
said actor input comprises a dynamic gesture of an end-effector.

15. The method of claim 5 , further comprising:
outputting at least a portion of said field of view to a display.

16. The method of claim 15 , further comprising:
outputting to said display a graphic occupying at least a portion of said region.

17. The method of claim 16 , wherein:
said graphic comprises a virtual object.

18. The method of claim 17 , wherein:
said image comprises an unlock target.

19. The method of claim 18 , wherein said region is substantially shaped as a silhouette of said actor executing said actor input.

20. The method of claim 19 , wherein said region is substantially shaped as an outline of a hand executing said input.

21. The method of claim 18 , wherein:
said image comprises unlock instructions.

22. The method of claim 5 further comprising:
defining said region as substantially two dimensional.

23. The method of claim 5 further comprising:
defining said region as three dimensional.

24. The method of claim 5 further comprising:
defining a minimum rate of apparent motion;
identifying said at least one structural geometric salient feature in said region, ignoring portions of said region exhibiting less than said minimum rate of apparent motion.

25. The method of claim 5 , further comprising:
identifying said at least one structural geometric salient feature in said region, beginning at a bottom of said region.

26. The method of claim 5 , further comprising:
identifying said at least one structural geometric salient feature in said region, beginning in a horizontal stripe.

27. The method of claim 5 , further comprising:
determining a position of said at least one structural geometric salient feature in said at least one image;
obtaining a second image of said field of view; and
searching for said at least one structural geometric salient feature in a second position in said second image, said second position corresponding to said position of said at least one structural geometric salient feature in said at least one image.

28. The method of claim 1 , wherein:
said command comprises an unlock command.

29. The method of claim 1 , comprising:
executing said command only if said data set substantially matches a reference data set.

30. An apparatus, comprising:
at least one sensor; and
a processor in communication with said sensor;
wherein:
said processor is adapted to define an actor input and a command associate with said actor input;
said sensor is adapted to detect said actor input;
said processor is adapted to execute said command in response to said input;
said processor is adapted to identify at least one structural geometric salient feature of an actor from said actor input associated with said executed command;
said processor is adapted to define a structural geometric actor model of said actor from said at least one structural geometric salient feature;
said processor is adapted to retain a data set comprising at least one of said at least one structural geometric salient feature or said structural geometric actor model

Assignees

• Atheer Inc

Inventors

• Itani Sleiman
• Yang Allen Yang

Classifications

• G06T2207/20112

  Image segmentation details · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06F16/5854 Primary

  using shape and object relationship · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title