Apparatus and methods for saliency detection based on color occurrence analysis

Patent metadata

Field	Value
Publication number	US-9870617-B2
Application number	US-201514637191-A
Country	US
Kind code	B2
Filing date	Mar 3, 2015
Priority date	Sep 19, 2014
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

Apparatus and methods for detecting and utilizing saliency in digital images. In one implementation, salient objects may be detected based on analysis of pixel characteristics. Least frequently occurring pixel values may be deemed as salient. Pixel values in an image may be compared to a reference. Color distance may be determined based on a difference between reference color and pixel color. Individual image channels may be scaled when determining saliency in a multi-channel image. Areas of high saliency may be analyzed to determine object position, shape, and/or color. Multiple saliency maps may be additively or multiplicative combined in order to improve detection performance (e.g., reduce number of false positives). Methodologies described herein may enable robust tracking of objects utilizing fewer determination resources. Efficient implementation of the methods described below may allow them to be used for example on board a robot (or autonomous vehicle) or a mobile determining platform.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium having instructions embodied thereon, the instructions being executable by a processing apparatus to configure a process enabled to track an object in a plurality of images each comprised of pixels, wherein the instructions, when executed by the processing apparatus cause the process to:
evaluate a pixel chromaticity distribution for pixels of a given image of the plurality of images, the evaluation of the pixel chromaticity distribution comprising a reconstruction of the given image and an assignment of a saliency value within the reconstructed image;
based at least on the evaluation, identify a portion of pixels of the given image, the identification of the portion of pixels being based on the assigned saliency value within the reconstructed image;
provide positions corresponding respectively to individual pixels of the portion to the process; and
based on the provided positions, navigate a robotic apparatus in communication with the processing apparatus;
wherein:
the portion of pixels corresponds to a representation of the object in the given image;
the provided positions are configured to enable the process to determine a location of the object in a subsequent image of the plurality of images;
the evaluation of the pixel chromaticity distribution comprises a determination of a histogram of occurrence of pixel chromaticity within the given image.

2. The non-transitory computer-readable storage medium of claim 1 , wherein:
the pixel chromaticity distribution is configured to characterize a frequency of occurrence of pixel chromaticity within the given image; and
the identification of the portion of pixels is further based at least in part on the frequency of occurrence of pixel chromaticity breaching a threshold.

3. The non-transitory computer-readable storage medium of claim 2 , wherein identification of the portion of pixels of the given image comprises: (i) a determination of a spatial distribution of a difference measure between chromaticity of pixels of the given image and a reference chromaticity; and (ii) a comparison of the difference measure at a pixel location to a level.

4. The non-transitory computer-readable storage medium of claim 3 , wherein the reference chromaticity is determined based at least on evaluation of the frequency of occurrence of pixel chromaticity.

5. The non-transitory computer-readable storage medium of claim 4 , wherein:
the reference chromaticity is determined further based at least on a prevailing chromaticity of pixels satisfying a condition, and
the condition comprises the frequency of occurrence of pixel chromaticity being below the threshold.

6. The non-transitory computer-readable storage medium of claim 2 , wherein the configuration to enable the process further includes producing a back projected map comprising a plurality of entries, a value of a given entry at a given location being assigned a value corresponding to a chromaticity occurrence of a corresponding pixel at the given location within the given image.

7. The non-transitory computer-readable storage medium of claim 2 , wherein the portion of pixels corresponds to a pixel chromaticity occurrence being in the range comprising 0.1% to 5%, inclusive.

8. The non-transitory computer-readable storage medium of claim 2 , wherein:
the pixel chromaticity is characterized by first and second chromaticity channels; and
the histogram comprises a first plurality of bins characterizing the first chromaticity channel, and a second plurality of bins characterizing the second chromaticity channel.

9. The non-transitory computer-readable storage medium of claim 2 , wherein:
a number of bins of a first plurality of the pixel chromaticity is characterized by first and second chromaticity channels; and
the histogram comprises a first plurality of bins characterizing the first chromaticity channel, and a second plurality of bins characterizing the second chromaticity channel.

10. The non-transitory computer-readable storage medium of claim 9 , wherein:
values of the first and the second chromaticity channels are characterized by an 8-bit value; and
a number of at least one of the first plurality of bins and/or the second plurality of bins is selected from a range comprising 2 to 8 bins, inclusive.

11. The non-transitory computer-readable storage medium of claim 1 , wherein the evaluation of the pixel chromaticity distribution comprises a determination of a pixel chromaticity distribution model characterized by at least two parameters.

12. The non-transitory computer-readable storage medium of claim 1 , wherein the evaluation of the pixel chromaticity distribution comprises a determination of a pixel chromaticity distribution model, the determination of the model based at least on analysis of a portion of a plurality of images preceding the given image.

13. The non-transitory computer-readable storage medium of claim 1 , wherein the provision of the positions is configured to reduce a number of occurrences of false positives associated with detection of the object by the process in the another image, as compared to detection of the object by the process in the another image in the absence of the provision of the positions.

14. The non-transitory computer-readable storage medium of claim 1 , wherein:
the identification of the portion of pixels is further configured based at least on a determination of a low-pass filtered version of the difference measure; and
the configuration of the process further includes a determination of a shape parameter associated with a portion of the low-pass filtered version.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the portion of the low-pass filtered version is identified based at least on a greatest contiguous number of pixels.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the portion of the low-pass filtered version is identified based at least on a greatest cumulative saliency value, the greatest cumulative saliency value being determined by at least accumulation of a plurality of saliency values.

17. The non-transitory computer-readable storage medium of claim 14 , wherein the shape parameter comprises a circularity determined based at least on:
a determination of one or more contours associated with one or more contiguous pixel groups within the low-pass filtered version;
for individual ones of the one or more contours, a determination of a smallest enclosing circle; and
a ratio between an area of the individual ones of the one or more contours and an area of the smallest enclosing circle.

18. The non-transitory computer-readable storage medium of claim 1 , wherein:
the object comprises a target for the robotic apparatus; and
the tracking of the object is configured to cause a provision of a control command to the robotic apparatus to cause the robotic apparatus to approach the object.

19. The non-transitory computer-readable storage medium of claim 1 , wherein:
the object comprises a target object of a given color; and
the tracking of the object is configured to enable the robotic device to pick up the target object from a plurality of objects characterized by a plurality of colors, individual ones of the plurality of objects being characterized by a color of the plurality of colors.

20. A method of tracking an object in a sequence of images, the method comprising:
evaluating a pixel chromaticity distribution for pixels of a

Assignees

• Brain Corp

Inventors

• Piekniewski Filip
• Richert Micah
• Fisher Dimitry

Classifications

• Y10S901/47

  Optical · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T7/285

  using a sequence of stereo image pairs · CPC title