Combining convolution and deconvolution for object detection

Patent metadata

Field	Value
Publication number	US-10628705-B2
Application number	US-201815940907-A
Country	US
Kind code	B2
Filing date	Mar 29, 2018
Priority date	Mar 29, 2018
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Provided are systems, methods, and computer-readable medium for operating a neural network. In various implementations, the neural network can receive an input image that includes an object to be identified. The neural network can generate a plurality of initial feature maps using a convolution layers, wherein a first initial feature maps is generated using the input image. The neural network can generate an up-sampled feature map using a de-convolution layer that takes an initial feature map as input, where the up-sampled feature map has a same resolution as the previous initial feature map. The neural network can combine the up-sampled feature map and the previous initial feature map, and use the combined feature map to more accurate identify the object.

First claim

What is claimed is:

1. A method for object identification, comprising:
receiving an input image including a representation of an object to be identified;
generating a plurality of feature maps using a plurality of convolution layers of a neural network, wherein a first feature map of the plurality of feature maps is generated from the input image, and wherein one or more feature maps of the plurality of feature maps are generated from one or more previous feature maps of the plurality of feature maps;
generating, using a de-convolution layer of the neural network, an up-sampled feature map from a feature map of the one or more feature maps, wherein the up-sampled feature map has a same resolution as a previous feature map of the plurality of feature maps;
combining the previous feature map and the up-sampled feature map to produce a combined feature map; and
identifying the object represented in the input image using the combined feature map.

2. The method of claim 1 , wherein the previous feature map and the up-sampled feature map are combined using a concatenation operation.

3. The method of claim 1 , wherein the previous feature map and the up-sampled feature map are combined using a maximum value operation.

4. The method of claim 1 , wherein the combined feature map has a greater depth than the previous feature map.

5. The method of claim 1 , wherein the combined feature map has a same depth as the previous feature map.

6. The method of claim 1 , wherein the up-sampled feature map is generated without using a rectified linear unit operation.

7. The method of claim 1 , wherein the plurality of convolution layers perform a convolution to produce the plurality of feature maps.

8. The method of claim 1 , wherein the de-convolution layer performs a deconvolution on the feature map to produce the up-sampled feature map.

9. The method of claim 1 , wherein identifying the object further includes using a highest level feature map generated by the neural network.

10. The method of claim 1 , wherein identifying the object further includes categorizing the combined feature map using at least one fully-connected layer, wherein each node in the at least one fully-connected layer outputs a weighted sum that indicates a probable category.

11. The method of claim 1 , further comprising:
generating, using a second de-convolution layer of the neural network, a second up-sampled feature map from the combined feature map, wherein the second up-sampled feature map has a same resolution as a second previous feature map from the plurality of feature maps; and
combining the second previous feature map with the second up-sampled feature map to produce a second combined feature map, wherein identifying the object is further based on the second combined feature map.

12. The method of claim 1 , wherein the previous feature map is generated using a convolution layer from the plurality of convolution layers that precedes a convolution layer from the plurality of convolution layers used to generate the feature map.

13. An apparatus, comprising:
a memory configured to store an input image including a representation of an object to be identified; and
a processor configured to:
generate a plurality of feature maps using a plurality of convolution layers of a neural network, wherein a first feature map of the plurality of feature maps is generated from the input image, and wherein one or more feature maps of the plurality of feature maps are generated from one or more previous feature maps of the plurality of feature maps;
generate, using a de-convolution layer of the neural network, an up-sampled feature map from a feature map of the one or more feature maps, wherein the up-sampled feature map has a same resolution as a previous feature map of the plurality of feature maps;
combine the previous feature map and the up-sampled feature map to produce a combined feature map; and
identify the object represented in the input image using the combined feature map.

14. The apparatus of claim 13 , wherein the previous feature map and the up-sampled feature map are combined using a concatenation operation.

15. The apparatus of claim 13 , wherein the previous feature map and the up-sampled feature map are combined using a maximum value operation.

16. The apparatus of claim 13 , wherein the combined feature map has a greater depth than the previous feature map.

17. The apparatus of claim 13 , wherein the combined feature map has a same depth as the previous feature map.

18. The apparatus of claim 13 , wherein the up-sampled feature map is generated without using a rectified linear unit operation.

19. The apparatus of claim 13 , wherein the plurality of convolution layers perform a convolution to produce the plurality of feature maps.

20. The apparatus of claim 13 , wherein the de-convolution layer performs a deconvolution on the feature map to produce the up-sampled feature map.

21. The apparatus of claim 13 , wherein identifying the object further includes using a highest level feature map generated by the neural network.

22. The apparatus of claim 13 , wherein identifying the object further includes categorizing the combined feature map using at least one fully-connected layer, wherein each node in the at least one fully-connected layer outputs a weighted sum that indicates a probable category.

23. The apparatus of claim 13 , wherein the processor is further configured to:
generate, using a second de-convolution layer of the neural network, a second up-sampled feature map from the combined feature map, wherein the second up-sampled feature map has a same resolution as a second previous feature map from the plurality of feature maps; and
combine the second previous feature map with the second up-sampled feature map to produce a second combined feature map, wherein identifying the object is further based on the second combined feature map.

24. The apparatus of claim 13 , wherein the previous feature map is generated using a convolution layer from the plurality of convolution layers that precedes a convolution layer from the plurality of convolution layers used to generate the feature map.

25. A non-transitory computer-readable medium having stored thereon instructions that, when executed by a processor, cause the processor to perform operations including:
receiving an input image including a representation of an object to be identified;
generating a plurality of feature maps using a plurality of convolution layers of a neural network, wherein a first feature map of the plurality of feature maps is generated from the input image, and wherein one or more feature maps of the plurality of initial feature maps are generated from one or more previous feature maps of the plurality of feature maps;
generating, using a de-convolution layer of the neural network, an up-sampled feature map from a feature map of the one or more feature maps, wherein the up-sampled feature map has a same resolution as a previous feature map of the plurality of feature maps;
combining the previous feature map and the up-sampled feature map to produce a combined feature map; and
identifying the object represented in the input image using the combined feature map.

26. The non-transitory computer-readable medium of claim 25 , wherein the previous feature map and the up-sampled feature map are combined using a concatenation operation.

Assignees

• Qualcomm Inc

Inventors

• Jiang Bolan
• Li Xin
• Quan Shuxue
• Pan Yunke

Classifications

• G06K9/3241

  Physics · mapped topic

• G06N3/0454

  Physics · mapped topic

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06K9/6232 Primary

  Physics · mapped topic

• G06K9/66

  Physics · mapped topic