Methods for controlling electronic devices using gestures

Patent metadata

Field	Value
Publication number	US-9628843-B2
Application number	US-201113301425-A
Country	US
Kind code	B2
Filing date	Nov 21, 2011
Priority date	Nov 21, 2011
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

A system for controlling infrared (IR) enabled devices by projecting coded IR pulses from an active illumination depth camera is described. In some embodiments, a gesture recognition system includes an active illumination depth camera such as a depth camera that utilizes time-of-flight (TOF) or structured light techniques for obtaining depth information. The gesture recognition system may detect the performance of a particular gesture associated with a particular electronic device, determine a set of device instructions in response to detecting the particular gesture, and transmit the set of device instructions to the particular electronic device utilizing coded IR pulses. The coded IR pulses may imitate the IR pulses associated with a remote control protocol. In some cases, the coded IR pulses transmitted may also be used by the active illumination depth camera for determining depth information.

First claim

What is claimed is:

1. A method for controlling one or more electronic devices, comprising:
detecting a particular gesture;
determining a particular electronic device of the one or more electronic devices based on the particular gesture;
determining a plurality of coded light pulses for controlling the particular electronic device based on the particular gesture;
embedding a plurality of higher frequency light pulses within a first coded light pulse of the plurality of coded light pulses for controlling the particular electronic device;
projecting a sequence of light patterns originating from an active illumination depth camera into an environment, the sequence of light patterns includes the plurality of coded light pulses for controlling the particular electronic device; and
generating depth information associated with the environment, the generating depth information includes capturing a first image of the environment associated with a first high frequency light pulse of the plurality of higher frequency light pulses and generating the depth information using the first image, the generating depth information includes capturing one or more images associated with reflections of at least a subset of the sequence of light patterns within the environment.

2. The method of claim 1 , wherein:
the detecting the particular gesture includes performing skeletal tracking of a person within the environment performing the particular gesture.

3. The method of claim 1 , further comprising:
determining the particular electronic device of the one or more electronic devices based on a direction of motion associated with the particular gesture.

4. The method of claim 1 , wherein:
the active illumination depth camera utilizes time-of-flight techniques for obtaining the depth information; and
the plurality of coded light pulses correspond with a remote control protocol.

5. The method of claim 1 , wherein:
the generating depth information overlaps in time with the projecting a sequence of light patterns.

6. The method of claim 1 , further comprising:
detecting a new end user command based on the first image, the sequence of light patterns includes one or more new light pulses associated with the new end user command.

7. The method of claim 1 , further comprising:
determining environmental characteristics associated with the environment; and
modifying a light intensity associated with one or more light patterns of the sequence of light patterns based on the environmental characteristics.

8. The method of claim 1 , wherein:
the determining a plurality of coded light pulses includes acquiring one or more IR transmission protocols associated with the one or more electronic devices.

9. The method of claim 1 , wherein:
the determining a plurality of coded light pulses includes automatically identifying the one or more electronic devices.

10. The method of claim 9 , wherein:
the automatically identifying the one or more electronic devices includes transmitting one or more commands associated with the particular electronic device, the one or more commands cause the particular electronic device to generate sound, the automatically identifying the one or more electronic devices includes detecting and localizing sounds within the environment in response to the transmitting one or more commands.

11. A system for controlling one or more electronic devices, comprising:
an active illumination depth camera; and
one or more processors configured to detect a particular gesture and determine a particular electronic device of the one or more electronic devices based on the particular gesture, the one or more processors configured to determine a plurality of coded IR pulses for communicating with the particular electronic device based on the particular gesture and embed a plurality of higher frequency light pulses within a first coded light pulse of the plurality of coded IR pulses, the one or more processors configured to direct the active illumination depth camera to project a sequence of light patterns into an environment, the sequence of light patterns includes the plurality of coded IR pulses for communicating with the particular electronic device, the active illumination depth camera configured to capture a first image of the environment associated with a first high frequency light pulse of the plurality of higher frequency light pulses, the one or more processors configured to generate depth information associated with the environment based on the first image, the one or more processors configured to acquire one or more images associated with reflections of at least a subset of the sequence of light patterns within the environment and generate the depth information using the one or more images.

12. The electronic device of claim 11 , wherein:
the one or more processors configured to perform skeletal tracking of a person within the environment and detect the particular gesture based on the skeletal tracking.

13. One or more storage devices containing non-transitory processor readable code for programming one or more processors to perform a method for controlling one or more electronic devices comprising the steps of:
identifying a particular gesture associated with the one or more electronic devices;
determining a particular electronic device of the one or more electronic devices based on a direction of motion associated with the particular gesture;
determining a plurality of coded IR pulses for controlling the particular electronic device;
embedding a plurality of higher frequency light pulses within a first coded light pulse of the plurality of coded IR pulses for controlling the particular electronic device;
modulating one or more light patterns projected from an active illumination depth camera into an environment, the one or more light patterns includes the plurality of coded IR pulses for controlling the particular electronic device; and
generating depth information associated with the environment, the generating depth information includes capturing a first image of the environment associated with a first high frequency light pulse of the plurality of higher frequency light pulses and generating the depth information using the first image, the generating depth information includes capturing one or more images associated with reflections of at least a subset of the one or more light patterns within the environment.

14. The one or more storage devices of claim 13 , wherein:
the modulating one or more light patterns includes modulating an amplitude of at least a subset of the one or more light patterns projected from the active illumination depth camera.

15. The one or more storage devices of claim 13 , wherein:
the modulating one or more light patterns includes modulating a frequency of the one or more light patterns projected from the active illumination depth camera.

16. The one or more storage devices of claim 13 , further comprising:
identifying an end user command corresponding with the particular gesture and determining the plurality of coded IR pulses based on the end user command.

Assignees

• Katz Sagi
• Adler Avishai
• Yahav Giora
• Microsoft Technology Licensing Llc

Inventors

• Katz Sagi
• Adler Avishai
• Yahav Giora

Classifications

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• A63F2300/1093

  using visible light · CPC title

• G06K9/00201

  Physics · mapped topic

• H04N21/422 Primary

  Input-only peripherals {, i.e. input devices connected to specially adapted client devices}, e.g. global positioning system [GPS] {(input devices also receiving signals from specially adapted client devices H04N21/4104)} · CPC title

• H04N21/44218

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title