Multi-tracker object tracking

Patent metadata

Field	Value
Publication number	US-9269012-B2
Application number	US-201313973913-A
Country	US
Kind code	B2
Filing date	Aug 22, 2013
Priority date	Aug 22, 2013
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Systems and approaches are provided for tracking an object using multiple tracking processes. By combining multiple lightweight tracking processes, object tracking can be robust, use a limited amount of power, and enable a computing device to respond to input corresponding to the motion of the object in real time. The multiple tracking processes can be run in parallel to determine the position of the object by selecting the results of the best performing tracker under certain heuristics or combining the results of multiple tracking processes in various ways. Further, other sensor data of a computing device can be used to improve the results provided by one or more of the tracking processes.

First claim

What is claimed is:

1. A computing device, comprising:
one or more processors;
one or more cameras; and
memory including instructions that, upon being executed by the one or more processors, cause the computing device to:
capture, at a first time, one or more first images using the one or more cameras;
detect at least one portion of a representation of a face of a user in the one or more first images using a facial detection process;
capture, at a second time, one or more second images using the one or more cameras;
track the at least one portion of the representation of the face of the user between the one or more first images and the one or more second images by concurrently executing a plurality of types of object tracking processes, the plurality of types of object tracking processes including two or more of a feature-based tracking process, an elliptical tracking process, a template matching tracking process, an optical flow tracking process, or a silhouette or contour tracking process;
select an object tracking process from among the plurality of object tracking processes based at least in part upon the object tracking process best satisfying a condition; and
determine a position of the face of the user based at least in part upon where the selected object tracking process tracked the at least one portion of the representation of the face of the user in the one or more second images.

2. The computing device of claim 1 , wherein the condition corresponds to at least one of a highest level of confidence, a lowest error rate, an average corresponding to where the plurality of types of object tracking processes tracked the at least one portion of the representation of the face of the user in the one or more second images, and a consensus corresponding to where a majority of the plurality of types of object tracking processes tracked the at least one portion of the representation of the face of the user in the one or more second images.

3. The computing device of claim 1 , further comprising:
a display screen operable to display content in two dimensions,
wherein the instructions upon being executed further cause the computing device to:
determine a first position of the face of the user based at least in part upon where the at least one portion of the representation of the face of the user is detected in the one or more first images;
display a first view of the content on the display screen, the first view of the content including one or more graphical elements rendered according to a first three-dimensional perspective that is based at least in part upon the first position of the face of the user; and
display a second view of the content on the display screen, the second view of the content including the one or more graphical elements rendered according to a second three-dimensional perspective that is based at least in part upon the position of the face of the user.

4. A computer-implemented method for tracking an object, comprising:
under control of one or more computer systems configured with executable instructions,
obtaining data corresponding to where at least one portion of a representation of the object is located in one or more first images;
obtaining one or more second images, at least one of the second images including the at least one portion of the representation of the object;
tracking the at least one portion of the representation of the object between the one or more first images and the one or more second images by concurrently executing a plurality of types of object tracking processes, the plurality of types of object tracking processes including two or more of a feature-based tracking process, an elliptical tracking process, a template matching tracking process, an optical flow tracking process, or a silhouette or contour tracking process;
selecting an object tracking process from among the plurality of object tracking processes based at least in part upon the object tracking process best satisfying a condition; and
determining a position of the object based at least in part upon where the selected object tracking process tracked the at least one portion of the representation of the object in the one or more second images.

5. The computer-implemented method of claim 4 , wherein obtaining the data corresponding to where the at least one portion of the representation of the object is located in the one or more first images includes:
obtaining the one or more first images including the at least one portion of the representation of the object;
detecting the at least one portion of the representation of the object in the one or more first images; and
providing the data corresponding to where the at least one portion of the representation of the object is located in the one or more first images based upon where the at least one portion of the representation of the object is detected in the one or more first images.

6. The computer-implemented method of claim 4 , wherein determining the position of the object includes:
determining a first plurality of points of the at least one portion of the representation the object in the one or more first images respectively corresponding to a second plurality of points of the at least one portion of the representation of the object in the one or more second images;
calculating a fundamental matrix or an essential matrix using the first plurality of points and the second plurality of points; and
determining one or more three-dimensional points corresponding to the object by triangulation using a first camera matrix used to capture the one or more first images, a second camera matrix used to capture the one or more second images, the first plurality of points, and the second plurality of points.

7. The computer-implemented method of claim 4 , wherein the condition corresponds to at least one of a highest level of confidence, a lowest error rate, an average corresponding to where the plurality of types of object tracking processes tracked the at least one portion of the representation of the object in the one or more second images, and a majority corresponding to where the plurality of types of object tracking processes tracked the at least one portion of the representation of the object in the one or more second images.

8. The computer-implemented method of claim 4 , further comprising:
determining at least one of a motion or an orientation corresponding to one or more cameras used to capture the one or more second images; and
causing at least one of the plurality of types of object tracking processes to focus on at least one portion of the one or more second images to track the at least one portion of the representation of the object, the at least one portion of the one or more second images corresponding to the at least one of the motion or the orientation.

9. The computer-implemented method of claim 4 , further comprising:
determining an amount of ambient light of an environment in which one or more cameras used to capture the one or more second images are situated; and
weighting second data corresponding to at least one of the plurality of types of object tracking processes based at least in part upon the amount of ambient light.

10. The computer-implemented method of claim 4 , further comprising:
weighting respective data corresponding to each of the plurality of types of object tracking processes,
wherein the position of the object is further based at least in part upon each weight of the respective data corresponding to each of the plurality of types of object tracking processes.

11. The computer-implemented method of claim 10 , wherein the weight of the respective data corresponding to each of th

Assignees

• Amazon Tech Inc

Inventors

• Fotland David Allen

Classifications

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/30201

  Face · CPC title

• G06K9/00369

  Physics · mapped topic

• G06K9/3241 Primary

  Physics · mapped topic