Identifying and tracking objects via lighting patterns

Patent metadata

Field	Value
Publication number	US-10456685-B2
Application number	US-201514686574-A
Country	US
Kind code	B2
Filing date	Apr 14, 2015
Priority date	Apr 14, 2015
Publication date	Oct 29, 2019
Grant date	Oct 29, 2019

Abstract

Techniques and systems for tracking multiple objects over a common time period and identifying (i.e., disambiguating) the objects from one another. As described herein, each object may include a respective light source (e.g., one or more LEDs) that may illuminate according to a defined lighting pattern. One or more cameras may capture images of a scene that includes the objects and the images may be analyzed to identify a location of each respective light source within the scene. By identifying these locations over multiple images, the movement of each object may be determined. In some instances, a system that tracks the movement of the objects may iterate through instructing each of the objects to illuminate its light source according to an identification lighting pattern, while other light source(s) of the other respective object(s) continue illuminating their light sources according to a default lighting pattern.

First claim

What is claimed is:

1. A system, comprising:
one or more cameras configured to capture images of a first controller and a second controller, the first controller including a first light source and the second controller including a second light source;
one or more processors; and
one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising:
causing the first controller to pulse the first light source according to an identification pulsing pattern during a first time period;
causing the second controller to pulse the second light source according to a default pulsing pattern during the first time period, the identification pulsing pattern differing from the default pulsing pattern, wherein the identification pulsing pattern includes at least one of a first light pulse rate or a first light wavelength and the default pulsing pattern includes at least one of a second light pulse rate or a second light wavelength;
receiving, from the one or more cameras, first images captured during the first time period, wherein the first images correspond to the identification pulsing pattern and the default pulsing pattern;
detecting the identification pulsing pattern within the first images;
detecting the default pulsing pattern within the first images;
identifying the first controller based at least in part on detecting the identification pulsing pattern within the first images;
tracking a movement of the first controller and a movement of the second controller;
causing the second controller to pulse the second light source according to the identification pulsing pattern during a second time period;
causing the first controller to pulse the first light source according to the default pulsing pattern during the second time period;
receiving, from the one or more cameras, second images captured during the second time period, wherein the second images correspond to the identification pulsing pattern and the default pulsing pattern;
detecting the identification pulsing pattern within the second images;
detecting the default pulsing pattern within the second images;
identifying the second controller based at least in part on detecting the identification pulsing pattern within the second images; and
tracking the movement of the first controller and the movement of the second controller.

2. The system as recited in claim 1 , wherein the one or more computer-readable media further store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform an act comprising configuring the one or more cameras to capture one or more respective images each time the first light source changes from an on state to an off state or from the off state to the on state.

3. The system as recited in claim 1 , wherein the one or more computer-readable media further store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform an act comprising periodically synchronizing the one or more cameras with the first controller and the second controller such that the one or more cameras capture images each time the first light source or the second light source changes state.

4. The system as recited in claim 1 , wherein:
the first controller comprises hardware, software, or a combination thereof; and
the second controller comprises hardware, software, or a combination thereof.

5. The system as recited in claim 1 , wherein:
the system comprises a game console;
the first controller comprises a first wireless game controller for controlling an operation of the game console; and
the second controller comprises a second wireless game controller for controlling the operation of the game console.

6. A method, comprising:
causing a first object in an environment to illuminate a first light source associated with the first object according to a first lighting pattern;
causing a second object in the environment to illuminate a second light source associated with the second object according to a second lighting pattern, the second lighting pattern differing from the first lighting pattern, wherein the first lighting pattern includes at least one of a first light pulse rate or a first light wavelength and the second lighting pattern includes at least one of a second light pulse rate or a second light wavelength;
capturing, over a first time period, first images of the environment while the first light source is illuminated according to the first lighting pattern and the second light source is illuminated according to the second lighting pattern;
analyzing the first images to identify the first object within the environment based at least in part on the first lighting pattern;
analyzing the first images to identify the second object within the environment based at least in part on the second lighting pattern;
causing the second object to illuminate the second light source according to the first lighting pattern;
causing the first object to illuminate the first light source according to the second lighting pattern;
capturing, over a second time period, second images of the environment while the first light source is illuminated according to the second lighting pattern and the second light source is illuminated according to the first lighting pattern;
analyzing the second images to identify the first object within the environment based at least in part on the second lighting pattern; and
analyzing the second images to identify the second object within the environment based at least in part on the first lighting pattern.

7. The method as recited in claim 6 , further comprising based at least in part on the analyzing the first images:
tracking, during the first time period, a movement of the first object; and
tracking, during the first time period, a movement of the second object.

8. The method as recited in claim 7 , further comprising based at least in part on the analyzing the second images:
tracking, during the second time period, a movement of the first object; and
tracking, during the second time period, a movement of the second object.

9. The method as recited in claim 6 , further comprising synchronizing one or more cameras associated with the environment with the first light source and the second light source such that the one or more cameras capture images when at least one of the first light source or the second light source transitions from at least one of an off state to an on state or from the on state to the off state.

10. The method as recited in claim 6 , wherein:
the first object comprises a first wireless controller of a game console; and
the second object comprises a second wireless controller of the game console.

11. The method as recited in claim 6 , wherein the causing the first object to illuminate the first light source according to the first lighting pattern occurs over a short-range wireless radio communication connection;
the causing the second object to illuminate the second light source according to the second lighting pattern occurs over the short-range wireless radio communication connection;
the causing the second object to illuminate the second light source according to the first lighting pattern occurs over the short-range wireless radio communication connection; and
the causing the first object to illuminate the first light source according to the second lighting pattern occurs over the short-range wireless radio communication connection.

12. The method as recited in claim 6 , further comprising:

Assignees

• Nintendo Co Ltd

Inventors

• Ranjan Sharad
• Sudharsanan Subramania

Classifications

• G06V20/52 Primary

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• A63F13/5258 Primary

  by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06V10/141

  Control of illumination · CPC title