System and method for semantic segmentation using dense upsampling convolution (DUC)

Patent metadata

Field	Value
Publication number	US-9953236-B1
Application number	US-201715456219-A
Country	US
Kind code	B1
Filing date	Mar 10, 2017
Priority date	Mar 10, 2017
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

A system and method for semantic segmentation using dense upsampling convolution (DUC) are disclosed. A particular embodiment includes: receiving an input image; producing a feature map from the input image; performing a convolution operation on the feature map and reshape the feature map to produce a label map; dividing the label map into equal subparts, which have the same height and width as the feature map; stacking the subparts of the label map to produce a whole label map; and applying a convolution operation directly between the feature map and the whole label map without inserting extra values in deconvolutional layers to produce a semantic label map.

First claim

What is claimed is:

1. A system comprising:
a data processor; and
an image processing module, executable by the data processor, the image processing module being configured to perform semantic segmentation using a dense upsampling convolution (DUC) operation, the DUC operation being configured to:
receive an input image;
produce a feature map from the input image;
perform a convolution operation on the feature map and reshape the feature map to produce a label map;
divide the label map into equal subparts, which have the same height and width as the feature map;
stack the subparts of the label map to produce a whole label map; and
apply a convolution operation directly between the feature map and the whole label map without inserting extra values in deconvolutional layers to produce a semantic label map.

2. The system of claim 1 wherein the DUC operation is machine learnable.

3. The system of claim 1 wherein the DUC operation is configured to learn an array of upscaling filters to upscale the feature map into a final dense feature map of a desired size.

4. The system of claim 1 wherein the DUC operation is configured operate within a fully convolutional network (FCN).

5. The system of claim 1 wherein the DUC operation is performed at an original resolution, thereby enabling pixel-level decoding.

6. The system of claim 1 wherein the semantic label map is used by an autonomous control subsystem to control a vehicle without a driver.

7. A method comprising:
receiving an input image;
producing a feature map from the input image;
performing a convolution operation on the feature map and reshaping the feature map to produce a label map;
dividing the label map into equal subparts, which have the same height and width as the feature map;
stacking the subparts of the label map to produce a whole label map; and
applying a convolution operation directly between the feature map and the whole label map without inserting extra values in deconvolutional layers to produce a semantic label map.

8. The method of claim 7 wherein the method enables machine learning.

9. The method of claim 7 including learning an array of upscaling filters to upscale the feature map into a final dense feature map of a desired size.

10. The method of claim 7 wherein the method is configured operate within a fully convolutional network (FCN).

11. The method of claim 7 wherein the method is performed at an original resolution, thereby enabling pixel-level decoding.

12. The method of claim 7 wherein the semantic label map is used by an autonomous control subsystem to control a vehicle without a driver.

13. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive an input image;
produce a feature map from the input image;
perform a convolution operation on the feature map and reshape the feature map to produce a label map;
divide the label map into equal subparts, which have the same height and width as the feature map;
stack the subparts of the label map to produce a whole label map; and
apply a convolution operation directly between the feature map and the whole label map without inserting extra values in deconvolutional layers to produce a semantic label map.

14. The non-transitory machine-useable storage medium of claim 13 wherein the instructions are further configured to enable machine learning.

15. The non-transitory machine-useable storage medium of claim 13 wherein the instructions are further configured to learn an array of upscaling filters to upscale the feature map into a final dense feature map of a desired size.

16. The non-transitory machine-useable storage medium of claim 13 wherein the instructions are further configured to operate within a fully convolutional network (FCN).

17. The non-transitory machine-useable storage medium of claim 13 wherein the instructions are further configured to perform at an original resolution, thereby enabling pixel-level decoding.

18. The non-transitory machine-useable storage medium of claim 13 wherein the semantic label map is used by an autonomous control subsystem to control a vehicle without a driver.

Assignees

• TuSimple

Inventors

• Huang Zehua
• Chen Pengfei
• Wang Panqu

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06F18/24

  Classification techniques · CPC title