Hand-gesture recognition method

Patent metadata

Field	Value
Publication number	US-9696808-B2
Application number	US-33722508-A
Country	US
Kind code	B2
Filing date	Dec 17, 2008
Priority date	Jul 13, 2006
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

One embodiment of the invention includes a method of providing device inputs. The method includes illuminating hand gestures performed via a bare hand of a user in a foreground of a background surface with at least one infrared (IR) light source. The method also includes generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface and generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface. The method also includes determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images. The method also includes determining a provided input gesture based on the plurality of three-dimensional features of the bare hand and comparing the provided input gesture with a plurality of predefined gesture inputs in a gesture library. The method further includes providing at least one device input corresponding to interaction with displayed visual content based on the provided input gesture corresponding to one of the plurality of predefined gesture inputs.

First claim

What is claimed is:

1. A method or providing device inputs, the method comprising:
illuminating hand gestures performed via a bare hand of a user in a three-dimensional foreground of a background surface with at least one infrared (IR) light source;
generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;
generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand the background surface;
determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images;
determining a first provided input gesture based on the plurality of three-dimensional features of the bare hand, the provided input gesture comprising the bare hand being positioned in a flat, open hand gesture and being rotated at an associated wrist approximately 180 degrees;
determining a second provided input gesture comprising the flat, open hand gesture being maintained subsequent to rotation of the associated wrist while at least one additional gesture input is performed from one of the bare hand and another bare hand of the user;
comparing the provided first and second input gestures with a plurality of predefined gesture inputs in a gesture library;
providing at least one first device input corresponding to interaction with displayed visual content based on the provided first input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one first device input responsive to the provided first input gesture comprising one of a page-up input and a page-down input associated with a document; and
providing at least one second device input corresponding to interaction with the displayed visual content based on the provided second input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one second device input responsive to the provided second input gesture comprising one of:
scrolling a page of the document one of upwards and downwards in response to the at least one additional gesture input from the bare hand;
repeating at least one of the page-up and the page-down inputs associated with the document in response to the at least one additional gesture input; and
auto-turning a page of the document at one of an acceleration and a deceleration that is based on a duration at which the at least one additional gesture input is maintained.

2. The method of claim 1 , wherein determining the provided second input gesture further comprises determining a second input gesture comprising the bare hand being positioned in the flat, open hand gesture and being rotated at the wrist approximately 180 degrees while concurrently moving forward, and wherein providing the at least one device input comprises providing one of a plurality of page-up inputs and a plurality of page down inputs associated with the document responsive to the second input gesture.

3. A method of providing device inputs, the method comprising:
illuminating hand gestures performed via a bare hand of a user in a three-dimensional foreground of a background surface with at least one infrared (IR) light source;
generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;
generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;
determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images;
determining a provided input gesture based on the plurality of three-dimensional features of the bare hand, the provided input gesture comprising the bare hand switching from a first sub-gesture comprising an extended thumb, and extended index finger, and an extended middle finger to a second sub-gesture comprising a retracted thumb, an extended index finger, and an extended middle finger and is moved one of up, down, left, and right while maintaining the second sub-gesture;
comparing the provided input gesture with a plurality of predefined gesture inputs in a gesture library; and
providing at least one device input corresponding to interaction with displayed visual content based on the provided input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one device input responsive to the provided input gesture comprising tilting a view one of up and down in response to the movement of the maintained second sub-gesture up and down, respectively, and panning the view one of left and right in response to the movement of the maintained second sub-gesture left and right, respectively.

4. A method of providing device inputs, the method comprising:
illuminating hand gestures performed via a bare hand of a user in a three-dimensional foreground of a background surface with at least one infrared (IR) light source;
generating a first plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;
generating a second plurality of silhouette images associated with the bare hand based on an IR light contrast between the bare hand and the background surface;
determining a plurality of three-dimensional features of the bare hand relative to the background surface based on a parallax separation of the bare hand in the first plurality of silhouette images relative to the second plurality of silhouette images;
determining a provided input gesture based on the plurality of three-dimensional features of the bare hand, the provided input gesture comprising the bare hand of the user pointing at the portion of the displayed visual content with an extended index finger and an extended thumb that is subsequently retracted followed by a rotation of an associated wrist;
comparing the provided input gesture with a plurality of predefined gesture inputs in a gesture library; and
providing at least one device input corresponding to interaction with displayed visual content based on the provided input gesture corresponding to one of the plurality of predefined gesture inputs, the at least one device input responsive to the provided input gesture comprising tilting a view associated with the displayed visual content in response to the retraction of the thumb, the tilting being based on the rotation of the wrist.

Assignees

• Nishihara H Keith
• Hsu Shi-Ping
• Kaehler Adrian
• Jangaard Lars
• Northrop Grumman Systems Corp

Inventors

• Nishihara H Keith
• Hsu Shi-Ping
• Kaehler Adrian
• Jangaard Lars

Classifications

• G06F3/0484

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• G06F3/0485

  Scrolling or panning · CPC title

• G06F3/0425

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title

• G06K9/00355

  Physics · mapped topic

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title