Pure convolutional neural network localization

Patent metadata

Field	Value
Publication number	US-9619735-B1
Application number	US-201615176328-A
Country	US
Kind code	B1
Filing date	Jun 8, 2016
Priority date	Jan 28, 2016
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

An approach is provided in which a knowledge manager processes an image using a convolutional neural network. The knowledge manager generates a pixel-level heat map of the image that includes multiple decision points corresponding to multiple pixels of the image. The knowledge manager analyzes the pixel-level heat map and detects sets of decision points that correspond to target objects. In turn, the knowledge manager marks regions of the heat map corresponding to the detected sets of per-pixel decision points, each of the regions indicating a location of the target objects.

First claim

The invention claimed is:

1. A method implemented by an information handling system that includes a memory and a processor, the method comprising:
generating a pixel-level heat map of an image using a convolutional neural network that comprises a plurality of convolutional layers and at least one 1×1 fully-connected convolutional layer, wherein the pixel-level heat map includes a plurality of per-pixel decision points corresponding to a plurality of pixels in the image;
detecting one or more sets of the plurality of per-pixel decision points that correspond to one or more target objects; and
marking one or more regions of the heat map that correspond to the detected one or more sets of per-pixel decision points, wherein each of the one or more regions indicate a location of the one or more target objects.

2. The method of claim 1 further comprising:
processing the image using the plurality of convolutional layers, resulting in a set of feature maps; and
processing the set of feature maps by the 1×1 fully-connected convolutional layer, resulting in the plurality of per-pixel decision points.

3. The method of claim 2 further comprising:
determining, for at least a selected one of the per-pixel decision points, whether the selected per-pixel decision point corresponds to a target class corresponding to at least one of the one or more target objects; and
assigning the target class to the selected per-pixel decision point based upon the determination.

4. The method of claim 3 wherein the plurality of per-pixel decision points are down sampled, and wherein the determining, the assigning, and the detecting are performed on the down sampled plurality of per-pixel decision points.

5. The method of claim 1 wherein, prior to the generating of the pixel-level heat map, the method further comprises:
training the convolutional neural network using a set of training images, wherein the training further comprises:
creating, by the plurality of convolutional layers, a plurality of pixel-level annotations from the set of training images; and
training on the plurality of pixel-level annotations at a plurality of scales.

6. The method of claim 1 wherein the detecting further comprises:
identifying a first one of the one or more regions corresponding to a first one of the one or more target objects; and
identifying a second one of the one or more regions corresponding to a second one of the one or more target objects.

7. The method of claim 6 wherein the first target object is a different object type than the second target object.

8. An information handling system comprising:
one or more processors;
a memory coupled to at least one of the processors; and
a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of:
generating a pixel-level heat map of an image using a convolutional neural network that comprises a plurality of convolutional layers and at least one 1×1 fully-connected convolutional layer, wherein the pixel-level heat map includes a plurality of per-pixel decision points corresponding to a plurality of pixels in the image;
detecting one or more sets of the plurality of per-pixel decision points that correspond to one or more target objects; and
marking one or more regions of the heat map that correspond to the detected one or more sets of per-pixel decision points, wherein each of the one or more regions indicate a location of the one or more target objects.

9. The information handling system of claim 8 wherein at least one of the one or more processors perform additional actions comprising:
processing the image using the plurality of convolutional layers, resulting in a set of feature maps; and
processing the set of feature maps by the 1×1 fully-connected convolutional layer, resulting in the plurality of per-pixel decision points.

10. The information handling system of claim 9 wherein at least one of the one or more processors perform additional actions comprising:
determining, for at least a selected one of the per-pixel decision points, whether the selected per-pixel decision point corresponds to a target class corresponding to at least one of the one or more target objects; and
assigning the target class to the selected per-pixel decision point based upon the determination.

11. The information handling system of claim 10 wherein the plurality of per-pixel decision points are down sampled, and wherein the determining, the assigning, and the detecting are performed on the down sampled plurality of per-pixel decision points.

12. The information handling system of claim 8 wherein, prior to the generating of the pixel-level heat map, at least one of the one or more processors perform additional actions comprising:
training the convolutional neural network using a set of training images, wherein the training further comprises:
creating, by the plurality of convolutional layers, a plurality of pixel-level annotations from the set of training images; and
training on the plurality of pixel-level annotations at a plurality of scales.

13. The information handling system of claim 8 wherein at least one of the one or more processors perform additional actions comprising:
identifying a first one of the one or more regions corresponding to a first one of the one or more target objects; and
identifying a second one of the one or more regions corresponding to a second one of the one or more target objects.

14. The information handling system of claim 6 wherein the first target object is a different object type than the second target object.

15. A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to perform actions comprising:
generating a pixel-level heat map of an image using a convolutional neural network that comprises a plurality of convolutional layers and at least one 1×1 fully-connected convolutional layer, wherein the pixel-level heat map includes a plurality of per-pixel decision points corresponding to a plurality of pixels in the image;
detecting one or more sets of the plurality of per-pixel decision points that correspond to one or more target objects; and
marking one or more regions of the heat map that correspond to the detected one or more sets of per-pixel decision points, wherein each of the one or more regions indicate a location of the one or more target objects.

16. The computer program product of claim 15 wherein the information handling system performs additional actions comprising:
processing the image using the plurality of convolutional layers, resulting in a set of feature maps; and
processing the set of feature maps by the 1×1 fully-connected convolutional layer, resulting in the plurality of per-pixel decision points.

17. The computer program product of claim 16 wherein the information handling system performs additional actions comprising:
determining, for at least a selected one of the per-pixel decision points, whether the selected per-pixel decision point corresponds to a target class corresponding to at least one of the one or more target objects; and
assigning the target class to the selected per-pixel decision point based upon the determination.

18. The computer program product of claim 17 wherein the plurality of per-pixel decision points are down sampled, and wherein the determining, the assigning, and the detecting are performed on the down sampled

Assignees

• IBM

Inventors

• Lineback Nicholas A
• Ranzinger Michael S

Classifications

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06F18/24 Primary

  Classification techniques · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T7/0081

  Physics · mapped topic