Instance segmentation method and apparatus

Patent metadata

Field	Value
Publication number	US-2022335619-A1
Application number	US-202217853799-A
Country	US
Kind code	A1
Filing date	Jun 29, 2022
Priority date	Dec 31, 2019
Publication date	Oct 20, 2022
Grant date	—

Abstract

An instance segmentation method and apparatus are provided. A to-be-trained segmentation network performs the following processing on each instance group that is in a sample original image and that is of pixels of a marked instance, where each instance group includes at least one marked instance, and the processing includes: predicting at least two different first basic feature maps and a first attention feature map corresponding to each first basic feature map; performing weighted processing on the at least two first basic feature maps and pixel values of respective first attention feature maps corresponding to the at least two first basic feature maps, to obtain a first feature fusion map; and training the to-be-trained segmentation network based on the first feature fusion map and the sample original image. A segmentation model can precisely determine pixels of an instance in an original image.

First claim

What is claimed is:

1 . An instance segmentation method, wherein the method comprises:
obtaining an original image; and inputting the original image into a segmentation network that has been trained, to obtain at least one feature fusion map corresponding to the original image, wherein the feature fusion map is used for marking pixels of an instance comprised in the original image, and each feature fusion map comprises at least one instance, wherein the segmentation network is trained in the following manner: performing, by a to-be-trained segmentation network, the following processing on each instance group that is in a sample original image and that is of pixels of a marked instance, wherein each instance group comprises at least one marked instance, and the processing comprises: predicting at least two different first basic feature maps and a first attention feature map corresponding to each first basic feature map; performing weighted processing on the at least two first basic feature maps and pixel values of respective first attention feature maps corresponding to the at least two first basic feature maps, to predict a first feature fusion map; and training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, wherein a size of the first attention feature map is the same as that of the first basic feature map, a pixel value of each pixel in the first attention feature map indicates a weight value of a pixel at a corresponding location in the first basic feature map corresponding to the first attention feature map, and there are pixels with different pixel values in the first attention feature map.

2 . The method according to claim 1 , wherein a value range of the pixel value of the first attention feature map is 0 to 1.

3 . The method according to claim 1 , wherein the sample original image is further marked with a bounding box, and the bounding box is used for identifying an instance; and
when the at least one feature fusion map corresponding to the original image is obtained, the method further comprises: obtaining a bounding box of the instance comprised in the original image.

4 . The method according to claim 3 , wherein the first basic feature map is a basic feature map of a bounding box image corresponding to the instance group; and
the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image comprises: training the to-be-trained segmentation network based on the first feature fusion map and the bounding box image.

5 . The method according to claim 1 , wherein before the inputting the original image into a segmentation network that has been trained, the method further comprises:
scaling the original image to a preset size in the segmentation network.

6 . The method according to claim 1 , wherein the size of the first basic feature map, the size of the first attention feature map, and a size of the first feature fusion map are all preset sizes in the to-be-trained segmentation network;
before the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, the method further comprises: performing scaling processing on the size of the first feature fusion map and/or a size of the sample original image, to enable the size of the first feature fusion map to be the same as the size of the sample original image.

7 . An instance segmentation apparatus, comprising a processor, a memory, and a transceiver, wherein
the memory stores a computer program or instructions; the transceiver is configured to receive and/or send a signal; and when the processor executes the computer program or the instructions, the apparatus is enabled to perform: obtaining an original image; and inputting the original image into a segmentation network that has been trained, to obtain at least one feature fusion map corresponding to the original image, wherein the feature fusion map is used for marking pixels of an instance comprised in the original image, and each feature fusion map comprises at least one instance, wherein the segmentation network is trained in the following manner: performing, by a to-be-trained segmentation network, the following processing on each instance group that is in a sample original image and that is of pixels of a marked instance, wherein each instance group comprises at least one marked instance, and the processing comprises: predicting at least two different first basic feature maps and a first attention feature map corresponding to each first basic feature map; performing weighted processing on the at least two first basic feature maps and pixel values of respective first attention feature maps corresponding to the at least two first basic feature maps, to predict a first feature fusion map; and training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, wherein a size of the first attention feature map is the same as that of the first basic feature map, a pixel value of each pixel in the first attention feature map indicates a weight value of a pixel at a corresponding location in the first basic feature map corresponding to the first attention feature map, and there are pixels with different pixel values in the first attention feature map.

8 . The apparatus according to claim 7 , wherein a value range of the pixel value of the first attention feature map is 0 to 1.

9 . The apparatus according to claim 7 , wherein the sample original image is further marked with a bounding box, and the bounding box is used for identifying an instance; and
when the at least one feature fusion map corresponding to the original image is obtained, the method further comprises: obtaining a bounding box of the instance comprised in the original image.

10 . The apparatus according to claim 9 , wherein the first basic feature map is a basic feature map of a bounding box image corresponding to the instance group; and
the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image comprises: training the to-be-trained segmentation network based on the first feature fusion map and the bounding box image.

11 . The apparatus according to claim 7 , wherein before the inputting the original image into a segmentation network that has been trained, the method further comprises:
scaling the original image to a preset size in the segmentation network.

12 . The method according to claim 7 , wherein the size of the first basic feature map, the size of the first attention feature map, and a size of the first feature fusion map are all preset sizes in the to-be-trained segmentation network;
before the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, the method further comprises: performing scaling processing on the size of the first feature fusion map and/or a size of the sample original image, to enable the size of the first feature fusion map to be the same as the size of the sample original image.

13 . A computer-readable storage medium, wherein the storage medium stores computer instructions; and when the computer instructions are executed by a computer, the computer is enabled to perform:
obtaining an original image; and inputting the original image into a segmentation network that has been trained, to obtain at least one feature fusion map corresponding to the original image, wherein the feature fusion map is used for marking pixels of an instance comprised in the original im

Assignees

• Huawei Tech Co Ltd

Inventors

• Sun Kunyang
• Chen Hao
• Shen Chunhua
• Yan Youliang
• Shao Bin
• Xu Songcen

Classifications

• G06V10/806 Primary

  of extracted features · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/82

  using neural networks · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title