System for detecting salient objects in images

Patent metadata

Field	Value
Publication number	US-2018307935-A1
Application number	US-201815946693-A
Country	US
Kind code	A1
Filing date	Apr 5, 2018
Priority date	Mar 24, 2015
Publication date	Oct 25, 2018
Grant date	—

Abstract

Described is a system for detecting salient objects in images. During operation, the system maps an input image into a frequency domain having a spectral magnitude. The spectral magnitude is replaced with weights from a weight matrix W. The frequency domain is then transformed with the weights to a saliency map in the image domain, the saliency map having pixels with pixel values. A squaring operation is then performed on the saliency map by squaring the pixel values to generate a pixel-value altered saliency map. A final saliency map is generated by filtering the pixel-value altered saliency map. A number of devices may then be operated based on the saliency map.

First claim

What is claimed is:

1 . A system for detecting salient objects in images, the system comprising:
one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
mapping an input image into a frequency domain having a spectral magnitude;
replacing the spectral magnitude with weights from a weight matrix W;
transforming the frequency domain with the weights to a saliency map in the image domain, the saliency map having pixels with pixel values;
performing a squaring operation on the saliency map by squaring the pixel values to generate a pixel-value altered saliency map; and
generating a final saliency map by filtering the pixel-value altered saliency map.

2 . The system as set forth in claim 1 , further comprising an operation of controlling a device based on the saliency map.

3 . The system as set forth in claim 1 , further comprising operations of:
designating a region within the final saliency map as an object; and causing an autonomous vehicle to perform a maneuver to avoid collision with the object.

4 . The system as set forth in claim 1 , wherein in filtering the pixel-value altered saliency map, salient edges within the pixel-value altered saliency map are blurred into salient regions.

5 . The system as set forth in claim 1 , further comprising operations of:
classifying an object within the final saliency map; and displaying the classification on a display device.

6 . The system as set forth in claim 1 , further comprising an operation of generating the weight matrix W, such that the weight matrix W is a foreground weight matrix F divided by the background weight matrix B, wherein the foreground weight matrix F is an average spectral magnitude of foreground regions from a training set, and wherein the background weight matrix B is an average spectral magnitude of background images.

7 . The system as set forth in claim 1 , further comprising operations of:
generating the weight matrix W, such that the weight matrix W is a foreground weight matrix F divided by the background weight matrix B, wherein the foreground weight matrix F is an average spectral magnitude of foreground regions from a training set, and wherein the background weight matrix B is an average spectral magnitude of background images. designating a region within the final saliency map as an object; classifying an object within the final saliency map; displaying the classification on a display device; causing an autonomous vehicle to perform a maneuver to avoid collision with the object; and wherein in filtering the pixel-value altered saliency map, salient edges within the pixel-value altered saliency map are blurred into salient regions.

8 . The system as set forth in claim 1 , further comprising an operation of sending the final saliency map to a cell phone or central monitoring facility.

9 . A computer program product for detecting salient objects in images, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
mapping an input image into a frequency domain having a spectral magnitude;
replacing the spectral magnitude with weights from a weight matrix W;
transforming the frequency domain with the weights to a saliency map in the image domain, the saliency map having pixels with pixel values;
performing a squaring operation on the saliency map by squaring the pixel values to generate a pixel-value altered saliency map; and
generating a final saliency map by filtering the pixel-value altered saliency map.

10 . The computer program product as set forth in claim 9 , further comprising an operation of controlling a device based on the saliency map.

11 . The computer program product as set forth in claim 9 , further comprising operations of:
designating a region within the final saliency map as an object; and causing an autonomous vehicle to perform a maneuver to avoid collision with the object.

12 . The computer program product as set forth in claim 9 , wherein in filtering the pixel-value altered saliency map, salient edges within the pixel-value altered saliency map are blurred into salient regions.

13 . The computer program product as set forth in claim 9 , further comprising operations of:
classifying an object within the final saliency map; and displaying the classification on a display device.

14 . The computer program product as set forth in claim 9 , further comprising an operation of generating the weight matrix W, such that the weight matrix W is a foreground weight matrix F divided by the background weight matrix B, wherein the foreground weight matrix F is an average spectral magnitude of foreground regions from a training set, and wherein the background weight matrix B is an average spectral magnitude of background images.

15 . The computer program product as set forth in claim 9 , further comprising operations of:
generating the weight matrix W, such that the weight matrix W is a foreground weight matrix F divided by the background weight matrix B, wherein the foreground weight matrix F is an average spectral magnitude of foreground regions from a training set, and wherein the background weight matrix B is an average spectral magnitude of background images. designating a region within the final saliency map as an object; classifying an object within the final saliency map; displaying the classification on a display device; causing an autonomous vehicle to perform a maneuver to avoid collision with the object; and wherein in filtering the pixel-value altered saliency map, salient edges within the pixel-value altered saliency map are blurred into salient regions.

16 . The computer program product as set forth in claim 9 , further comprising an operation of sending the final saliency map to a cell phone or central monitoring facility.

17 . A method for detecting salient objects in images, the method comprising acts of:
mapping an input image into a frequency domain having a spectral magnitude; replacing the spectral magnitude with weights from a weight matrix W; transforming the frequency domain with the weights to a saliency map in the image domain, the saliency map having pixels with pixel values; performing a squaring operation on the saliency map by squaring the pixel values to generate a pixel-value altered saliency map; and generating a final saliency map by filtering the pixel-value altered saliency map.

18 . The method as set forth in claim 17 , further comprising an operation of controlling a device based on the saliency map.

19 . The method as set forth in claim 17 , further comprising operations of:
designating a region within the final saliency map as an object; and causing an autonomous vehicle to perform a maneuver to avoid collision with the object.

20 . The method as set forth in claim 17 , wherein in filtering the pixel-value altered saliency map, salient edges within the pixel-value altered saliency map are blurred into salient regions.

21 . The method as set forth in claim 17 , further comprising operations of:
classifying an object within the final saliency map; and displaying the classification on a display device.

Assignees

• Hrl Lab Llc

Inventors

• Rao Shankar R
• Horvath Andras

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G08G1/13

  the indicator being in the form of a map · CPC title

• G08G1/164

  Centralised systems, e.g. external to vehicles · CPC title