Salient features tracking apparatus and methods using visual initialization

Patent metadata

Field	Value
Publication number	US-10055850-B2
Application number	US-201514637138-A
Country	US
Kind code	B2
Filing date	Mar 3, 2015
Priority date	Sep 19, 2014
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

Apparatus and methods for detecting and utilizing saliency in digital images. In one implementation, salient objects may be detected based on analysis of pixel characteristics. Least frequently occurring pixel values may be deemed as salient. Pixel values in an image may be compared to a reference. Color distance may be determined based on a difference between reference color and pixel color. Individual image channels may be scaled when determining saliency in a multi-channel image. Areas of high saliency may be analyzed to determine object position, shape, and/or color. Multiple saliency maps may be additively or multiplicative combined in order to improve detection performance (e.g., reduce number of false positives). Methodologies described herein may enable robust tracking of objects utilizing fewer determination resources. Efficient implementation of the methods described below may allow them to be used for example on board a robot (or autonomous vehicle) or a mobile determining platform.

First claim

What is claimed is:

1. A non-transitory computer-readable apparatus comprising a storage medium having instructions embodied thereon, the instructions being executable by a processing apparatus to provide a tracking indication to a robotic platform, wherein the instructions, when executed by the processing apparatus, are configured to:
for an image comprising a plurality of pixels and characterized by a color space comprising at least a chromaticity dimension, individual ones of the pixels characterized by one or more pixel characteristics comprising information related to at least the chromaticity dimension, determine at least one pixel occurrence statistics parameter based at least on analysis of the one or more pixel characteristics, the at least one pixel occurrence statistics parameter corresponding to a reference chromaticity within the color space of the image;
determine a distance measure between chromaticity of individual pixels within the image and the reference chromaticity, wherein the determination of the reference chromaticity is based on the pixel occurrence statistics parameter, the reference chromaticity being configured to increase a discriminability between a distance measure associated with pixels corresponding to a salient area of the image and a distance measure corresponding to a background area of the image;
detect a location of the salient area in the image corresponding to saliency when the at least one pixel occurrence statistics parameter breaches a threshold level, the threshold level being determined by a given physical task of the robotic platform.

2. The computer readable apparatus of claim 1 , wherein:
the image comprises a representation of an object; and
the provision of the tracking indication comprises provision of the tracking indication so as to enable tracking of the object throughout a plurality of images occurring subsequent to the image, individual ones of the plurality of images comprising representations of the object.

3. The computer readable apparatus of claim 1 , wherein:
the image comprises one or more channels, individual ones of the one or more channels being characterized by the one or more pixel characteristics; and
the one or more pixel characteristics are selected from the group consisting of a pixel color and a pixel luminance.

4. The computer readable apparatus of claim 3 , wherein the at least one pixel occurrence statistics parameter is configured based on a determination of a likelihood of occurrence of a value of a pixel characteristic within the image.

5. The computer readable apparatus of claim 4 , wherein:
the determination of the likelihood is configured based at least on a determination of a histogram of a plurality of values of the pixel characteristic, the histogram comprising a plurality of bins, individual ones of the plurality of bins being characterized by a characteristic value range;
a value of a given bin of the plurality of bins is determined based at least on a comparison between a value of the pixel characteristic and the characteristic value range for the given bin; and
a value of the threshold is further determined based at least on an evaluation of the histogram.

6. The computer readable apparatus of claim 1 , wherein:
the location of the salient area is determined based at least on the spatial saliency distribution meeting or exceeding the threshold;
the at least one pixel parameter is configured based at least on a distribution of pixel values within the image, the distribution of pixel values characterized by a parameter; and
the saliency determination of the location of the salient area comprises at least an evaluation of the at least one pixel parameter relative to the threshold.

7. The computer readable apparatus of claim 6 , wherein a value of the threshold is determined based at least on a determination of saliency associated with another image preceding the image.

8. The computer readable apparatus of claim 6 , wherein a value of the threshold is determined based at least on a success rate of an object detection in one or more images preceding the image.

9. The computer readable apparatus of claim 1 , wherein:
the salient area is characterized by a first rate of pixel occurrence; and
the instructions are further configured to, when executed by the processor apparatus, determine a spatial saliency distribution of the plurality of pixels based on the at least one pixel occurrence statistics parameter, wherein the spatial saliency distribution comprises a background area characterized by a second rate of pixel occurrence, the second rate of pixel occurrence being greater than the first rate of pixel occurrence.

10. The computer readable apparatus of claim 1 , wherein the increased discriminability is based at least on a greater difference between (i) the distance measure associated with pixels corresponding to the salient area of the image, and (ii) the distance measure corresponding to the background area of the image when compared to respective distance measures determined based on another reference chromaticity, the another reference chromaticity determined based at least on pixels disposed outside the salient area of the image.

11. The computer readable apparatus of claim 1 , wherein:
the tracking of the object comprises detection of the object in the image; and
the reference chromaticity is configured to improve the detection of the object, the improvement of the detection characterized by fewer false positives of object detection compared to object detection effectuated absent determination of the reference chromaticity based on the pixel occurrence statistics parameter.

12. The computer readable apparatus of claim 1 , wherein the instructions are further configured to, when executed by the processor apparatus:
cause a robotic platform in communication with the processing apparatus to navigate an initial trajectory corresponding to a given task;
determine a spatial saliency distribution of the plurality of pixels based on the at least one pixel occurrence statistics parameter;
determine a saliency map from the spatial saliency distribution based on a deviation between (i) the one or more pixel characteristics of a given pixel of the image and (ii) respective one or more pixel characteristics of a reference pixel of the image;
provide the tracking indication that is configured to convey information related to the salient area to a tracking apparatus in communication with the non-transitory computer-readable storage medium; and
adjust the initial trajectory of the robotic platform based on the tracking indication that is configured to convey the information related to the salient area, the adjustment of the trajectory being configured to improve upon completion of the given task as compared with the initial trajectory.

13. A method of providing a tracking indication to a tracking apparatus, the method comprising:
initializing a navigation of a robotic apparatus along an initial trajectory associated with a given physical task of the robotic apparatus;
determining a pixel occurrence statistics parameter for an image comprising a plurality of pixels, each of the plurality of pixels being characterized by one or more pixel characteristics, based at least on an analysis of the one or more pixel characteristics;
determining a spatial saliency distribution of the plurality of pixels based on the pixel occurrence statistics parameter;
determining a location of a salient area in an image corresponding to saliency based on an aspect of a contiguous area of the saliency map meeting or exceeding a threshold that depends on the given physical task of th

Assignees

• Brain Corp

Inventors

• Piekniewski Filip
• Richert Micah
• Fisher Dimitry

Classifications

• Y10S901/47

  Optical · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T7/285

  using a sequence of stereo image pairs · CPC title