Identifying And Tracking Objects Via Lighting Patterns

Patent metadata

Field	Value
Publication number	US-2016307332-A1
Application number	US-201514686574-A
Country	US
Kind code	A1
Filing date	Apr 14, 2015
Priority date	Apr 14, 2015
Publication date	Oct 20, 2016
Grant date	—

Abstract

Techniques and systems for tracking multiple objects over a common time period and identifying (i.e., disambiguating) the objects from one another. As described herein, each object may include a respective light source (e.g., one or more LEDs) that may illuminate according to a defined lighting pattern. One or more cameras may capture images of a scene that includes the objects and the images may be analyzed to identify a location of each respective light source within the scene. By identifying these locations over multiple images, the movement of each object may be determined. In some instances, a system that tracks the movement of the objects may iterate through instructing each of the objects to illuminate its light source according to an identification lighting pattern, while other light source(s) of the other respective object(s) continue illuminating their light sources according to a default lighting pattern.

First claim

What is claimed is:

1 . A system, comprising:
one or more cameras configured to capture images of a first object and a second object, wherein:
the first object includes a first light source and a first controller configured to pulse the first light source; and
the second object includes a second light source and a second controller configured to pulse the second light source;
one or more processors; and one or more computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising:
instructing the first controller to pulse the first light source according to an identification pulsing pattern during a first time period such that the first light source pulses according to the identification pulsing pattern during the first time period and the second light source pulses according to a default pulsing pattern during the first time period;
analyzing images captured by the one or more cameras during the first time period to identify the first object and to track respective movement of the first and second objects;
instructing the second controller to pulse the second light source according to the identification pulsing pattern during a second time period such that the second light source pulses according to the identification pulsing pattern during the second time period and the first light source pulses according to the default pulsing pattern during the second time period; and
analyzing images captured by the one or more cameras during the second time period to identify the second object and to track the respective movement of the first and second objects.

2 . The system as recited in claim 1 , the acts further comprising configuring the one or more cameras to capture one or more respective images each time the first light source changes from an on state to an off state or from the off state to the on state.

3 . The system as recited in claim 1 , the acts further comprising periodically synchronizing the one or more cameras with the first and second controllers such that the one or more cameras capture images each time the first or second light source changes state.

4 . The system as recited in claim 1 , wherein:
the first controller comprises hardware, software, or a combination thereof; and the second controller comprises hardware, software, or a combination thereof

5 . The system as recited in claim 1 , wherein the system comprises a game console and the first and second objects comprise respective wireless game controllers for controlling operation of the game console.

6 . A method, comprising:
illuminating, during a first time period, a first light source of a first object according to a first lighting pattern and a second light source of a second object according to a second lighting pattern; analyzing images captured during the first time period to identify the first object and to track respective movement of the first object and the second object during the first time period; illuminating, during a second time period, the second light source of the second object according to the first lighting pattern and the first light source of the first object according to the second lighting pattern; and analyzing images captured during the second time period to identify the second object and to track respective movement of the first object and the second object during the second time period.

7 . The method as recited in claim 6 , wherein:
the illuminating during the first time period further comprises illuminating a third light source of a third object according to the second lighting pattern; the analyzing the images captured during the first time period is further to track movement of the third object during the first time period; the illuminating during the second time period further comprises illuminating the third light source of the third object according to the second lighting pattern; and the analyzing the images captured during the second time period is further to track movement of the third object during the second time period.

8 . The method as recited in claim 7 , further comprising:
illuminating, during a third time period, the third light source of the third object according to the first lighting pattern and the first light source of the first object and the second light source of the second object according to the second lighting pattern; and analyzing images captured during the third time period to identify the third object and to track respective movement of the first, second, and third objects during the third time period.

9 . The method as recited in claim 6 , wherein the first and second lighting patterns comprise respective pulsing patterns.

10 . The method as recited in claim 6 , wherein the first and second lighting patterns comprise respective wavelength patterns.

11 . The method as recited in claim 6 , further comprising capturing images at a periodicity substantially equal to or greater than a periodicity of a pulsing of the first and second light sources.

12 . The method as recited in claim 11 , further comprising sharing timestamp information between the first object and one or more cameras that capture the images to determine a time offset between a clock of the first object and one or more clocks of the one or more cameras, the time offset for synchronizing image capture by the one or more cameras with pulsing of the first light source by the first object.

13 . A method, comprising:
instructing a first object in an environment to illuminate its light source according to an identification lighting pattern; capturing, over a first time period, images of the environment while the first object illuminates its light source according to the identification lighting pattern and a second object in the environment illuminates its light source according to a default lighting pattern; instructing the second object to illuminate its light source according to the identification lighting pattern; and capturing, over a second time period, images of the environment while the first object illuminates its light source according to the default lighting pattern and the second object illuminates its light source according to the identification lighting pattern.

14 . The method as recited in claim 13 , further comprising using the images captured over the first time period to identify the first object and to track respective movement of the first and second objects.

15 . The method as recited in claim 14 , further comprising using the images captured over the second time period to identify the second object and to track respective movement of the first and second objects.

16 . The method as recited in claim 13 , further comprising synchronizing one or more cameras that capture images of the environment with the light sources such that the one or more cameras capture images when a light source transitions from an off state to an on state and from the on state to the off state.

17 . The method as recited in claim 13 , wherein the first and second objects comprise respective wireless game controllers for controlling operation of a game console.

18 . The method as recited in claim 13 , wherein the instructing occurs over a short-range wireless radio communication connection.

19 . The method as recited in claim 13 , further comprising:
determining a total number of objects to identify and track; determining an amount of time allotted for identification of the total n

Assignees

• Nintendo Co Ltd

Inventors

• Ranjan Sharad
• Sudharsanan Subramania

Classifications

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• A63F13/5258

  by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06V10/141

  Control of illumination · CPC title