Near-plane segmentation using pulsed light source

Patent metadata

Field	Value
Publication number	US-9304594-B2
Application number	US-201313862196-A
Country	US
Kind code	B2
Filing date	Apr 12, 2013
Priority date	Apr 12, 2013
Publication date	Apr 5, 2016
Grant date	Apr 5, 2016

Abstract

Methods for recognizing gestures within a near-field environment are described. In some embodiments, a mobile device, such as a head-mounted display device (HMD), may capture a first image of an environment while illuminating the environment using an IR light source with a first range (e.g., due to the exponential decay of light intensity) and capture a second image of the environment without illumination. The mobile device may generate a difference image based on the first image and the second image in order to eliminate background noise due to other sources of IR light within the environment (e.g., due to sunlight or artificial light sources). In some cases, object and gesture recognition techniques may be applied to the difference image in order to detect the performance of hand and/or finger gestures by an end user of the mobile device within a near-field environment of the mobile device.

First claim

What is claimed is:

1. An electronic device for recognizing gestures, comprising:
a light source;
a first sensor; and
one or more processors in communication with the light source and the first sensor, the one or more processors cause the light source to emit IR light into an environment at a first light intensity level during a first period of time, the one or more processors detect that an amount of ambient IR light within the environment is greater than a particular threshold, the one or more processors cause the light source to emit the IR light into the environment at a second light intensity level different from the first light intensity level during a second period of time in response to detecting that the amount of ambient IR light within the environment is greater than the particular threshold, the one or more processors cause the first sensor to capture a first image of the environment during the second period of time and to capture a third image of the environment during a third period of time different from the second period of time, the light source does not emit the IR light into the environment during the third period of time, the one or more processors generate a first difference image based on the first image and the third image, the one or more processors identify one or more hand features based on the first difference image, the one or more processors detect a gesture based on the one or more hand features, the one or more processors perform a computing operation in response to detecting the gesture.

2. The electronic device of claim 1 , further comprising:
a second sensor, the one or more processors cause the second sensor to capture a second image of the environment during the second period of time and to capture a fourth image of the environment during the third period of time, the one or more processors generate a second difference image based on the second image and the fourth image, the one or more processors identify the one or more hand features using the first difference image and the second difference image.

3. The electronic device of claim 2 , wherein:
the one or more processors determine depth information associated with a first hand feature of the one or more hand features based on a change in one or more pixel locations associated with the first hand feature between the first difference image and the second difference image.

4. The electronic device of claim 3 , wherein:
the first hand feature comprises a finger peak.

5. The electronic device of claim 1 , wherein:
the first sensor comprises a color filter and an IR filter.

6. The electronic device of claim 1 , wherein:
the second light intensity level is set such that backscattering of the emitted IR light due to objects within a first range of the mobile device may be detected by the first sensor.

7. The electronic device of claim 6 , wherein:
the first range comprises one meter.

8. The electronic device of claim 1 , wherein:
the electronic device comprises an HMD;
the second light intensity level is less than the first light intensity level;
the third period of time is subsequent to the second period of time; and
the second period of time is subsequent to the first period of time.

9. A method for recognizing gestures, comprising:
emitting IR light from a mobile device into an environment at a first light intensity level during a first period of time;
detecting that an amount of ambient IR light within the environment is greater than a particular threshold;
emitting the IR light from the mobile device into the environment at a second light intensity level different from the first light intensity level during a second period of time, the emitting the IR light from the mobile device into the environment at the second light intensity level is performed in response to detecting that the amount of ambient IR light within the environment is greater than the particular threshold;
capturing a first image of the environment during the second period of time using a first sensor;
capturing a third image of the environment during a third period of time different from the second period of time using the first sensor, the IR light is not emitted from the mobile device during the third period of time;
generating a first difference image based on the first image and the third image;
identifying one or more hand features based on the first difference image;
detecting a gesture based on the one or more hand features; and
performing a computing operation on the mobile device in response to detecting the gesture.

10. The method of claim 9 , further comprising:
capturing a second image of the environment during the second period of time using a second sensor;
capturing a fourth image of the environment during the third period of time using the second sensor; and
generating a second difference image based on the second image and the fourth image, the identifying one or more hand features includes identifying the one or more hand features using the first difference image and the second difference image.

11. The method of claim 9 , wherein:
the identifying one or more hand features includes determining depth information associated with the one or more hand features.

12. The method of claim 9 , wherein:
the first sensor comprises a monochrome filter and an IR filter.

13. The method of claim 9 , wherein:
the second light intensity level is set such that backscattering of the emitted IR light due to objects within a first range of the mobile device may be detected by the first sensor.

14. The method of claim 13 , wherein:
the first range comprises one meter.

15. The method of claim 9 , wherein:
the one or more hand features include one or more finger valleys.

16. The method of claim 9 , wherein:
the mobile device comprises an HMD;
the second light intensity level is less than the first light intensity level;
the third period of time is subsequent to the second period of time; and
the second period of time is subsequent to the first period of time.

17. One or more hardware storage devices containing processor readable code for programming one or more processors to perform a method for recognizing gestures using a mobile device comprising the steps of:
projecting IR light from the mobile device into an environment at a first light intensity level during a first period of time;
detecting that an amount of ambient IR light within the environment is greater than a particular threshold;
determining a second light intensity level less than the first light intensity level in response to detecting that the amount of ambient IR light within the environment is greater than the particular threshold;
projecting the IR light from the mobile device into the environment at the second light intensity level during a second period of time;
capturing a first set of images of the environment during the second period of time using a first sensor;
capturing a second set of images of the environment during the second period of time using a second sensor;
capturing a third set of images of the environment during a third period of time different from the second period of time using the first sensor, the IR light is not projected from the mobile device during the third period of time;
capturing a fourth set of images of the environment during the third period of time using the second sensor;
generating a first set of difference images based on the first set of images and the third set of images;
generating a second set of difference images based on the second set of images a

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Finocchio Mark
• Balan Alexandru
• Ackerman Nathan
• Margolis Jeffrey

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title