Methods and apparatus for tracking objects using saliency

Patent metadata

Field	Value
Publication number	US-10657409-B2
Application number	US-201916291470-A
Country	US
Kind code	B2
Filing date	Mar 4, 2019
Priority date	Sep 19, 2014
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

Methods and apparatus for tracking and discerning objects using their saliency. In one embodiment of the present disclosure, the tracking of objects is based on a combination of object saliency and additional sources of signal about object identity. Under certain simplifying assumptions, the present disclosure allows for robust tracking of simple objects with limited processing resources. In one or more variants, efficient implementation of the methods described allow sensors (e.g., cameras) to be used on board a robot (or autonomous vehicle) on a mobile determining platform, such as to capture images to determine the presence and/or identity of salient objects. Such determination of salient objects allow for e.g., adjustments to vehicle or other moving object trajectory.

First claim

What is claimed is:

1. A system for tracking an object in a visual field, comprising:
a memory including computer readable instructions stored thereon; and
at least one processor configured to execute the computer readable instructions to,
generate a first image corresponding to a saliency image, the saliency image being based on a back projection image, the back projection image being the same size as an input image,
generate a heat map based on a distance function of the back projection image and the input image,
determine a mask corresponding to a region of the visual field based on a peak in the heat map, the heat map representing likelihood of presence of the object in the visual field based on a confidence value of the heat map, and
combine the saliency image and the mask in order to track the object in the visual field.

2. The system of claim 1 , wherein the back projection image is based on histogram of pixel values of the input image such that every pixel in the back project image includes the same value as pixel in the input image.

3. The system of claim 1 , wherein the generating of the saliency image based on difference between a pixel and an average value of pixels in a surrounding region of object in the visual field.

4. The system of claim 1 , wherein the at least one processor is further configured to,
generate the heat map based on a distance function of the back projection image and the input image.

5. The system of claim 1 , wherein the confidence value of the heat map include a first confidence value and a different second confidence value, the first confidence value of zero represents no change in a result, and the second confidence value of one represents significant change in the result.

6. The system of claim 1 , wherein the heat map is representative of a weighted sum of a plurality of individual heat maps.

7. The system of claim 6 , wherein the plurality of individual heat maps includes at least a first heat map and a second heat map, the first heat map corresponds to exclude certain positions as containing the object in the visual field, and the second heat map corresponds to distance detection threshold of the object.

8. A method for tracking an object in a visual field, comprising:
generating a first image corresponding to saliency image, the saliency image being based on a back projection image being the same size as an input image;
generating a heat map based on a distance function of the back projection image and the input image;
determining a mask corresponding to a region of the visual field based on a peak in the heat map, the heat map representing likelihood of presence of the object in the visual field based on a confidence value of the heat map; and
combining the saliency image and the mask in order to track the object in the visual field.

9. A non-transitory computer readable medium having computer readable instructions stored thereon, that when executed by at least one processor configure the at least one processor to,
generate a first image corresponding to saliency image, the saliency image being based on a back projection image being the same size as an input image;
generate a heat map based on a distance function of the back projection image and the input image;
determine a mask corresponding to a region of the visual field based on a peak in the heat map, the heat map representing likelihood of presence of the object in the visual field based on a confidence value of the heat map; and
combine the saliency image and the mask in order to track the object in the visual field.

Assignees

• Brain Corp

Inventors

• Piekniewski Filip
• Richert Micah

Classifications

• G06T7/292

  Multi-camera tracking · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• G06T2200/04

  involving 3D image data · CPC title