Instance segmentation method and apparatus

Patent metadata

Field	Value
Publication number	US-12254636-B2
Application number	US-202217853799-A
Country	US
Kind code	B2
Filing date	Jun 29, 2022
Priority date	Dec 31, 2019
Publication date	Mar 18, 2025
Grant date	Mar 18, 2025

Abstract

An instance segmentation method and apparatus are provided. A to-be-trained segmentation network performs the following processing on each instance group that is in a sample original image and that is of pixels of a marked instance, where each instance group includes at least one marked instance, and the processing includes: predicting at least two different first basic feature maps and a first attention feature map corresponding to each first basic feature map; performing weighted processing on the at least two first basic feature maps and pixel values of respective first attention feature maps corresponding to the at least two first basic feature maps, to obtain a first feature fusion map; and training the to-be-trained segmentation network based on the first feature fusion map and the sample original image. A segmentation model can precisely determine pixels of an instance in an original image.

First claim

What is claimed is:

1. An instance segmentation method, wherein the method comprises:
obtaining an original image; and
inputting the original image into a segmentation network that has been trained; and
obtaining at least one feature fusion map corresponding to the original image, wherein the at least one feature fusion map is used for marking pixels of an instance comprised in the original image, and each feature fusion map comprises at least one instance, wherein the segmentation network is trained by operations comprising:
for each instance group that is in a sample original image, wherein the each instance group comprises at least one marked instance, performing, by a to-be-trained segmentation network, processing comprising:
predicting at least two different first basic feature maps and a first attention feature map corresponding to each of the at least two different basic feature maps;
performing weighted processing on the at least two first basic feature maps and pixel values of respective first attention feature maps corresponding to the at least two first basic feature maps, to predict a first feature fusion map; and
training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, wherein:
a size of the first attention feature map is the same as that of the first basic feature map corresponding to the first attention feature map, a pixel value of each pixel in the first attention feature map indicates a weight value of a pixel at a corresponding location in the first basic feature map corresponding to the first attention feature map, and there are pixels with different pixel values in the first attention feature map.

2. The method according to claim 1 , wherein a value range of the pixel value of the first attention feature map is 0 to 1.

3. The method according to claim 1 , wherein the sample original image is further marked with a bounding box, and the bounding box is used for identifying an instance; and
when the at least one feature fusion map corresponding to the original image is obtained, the method further comprises:
obtaining a bounding box of the instance comprised in the original image.

4. The method according to claim 3 , wherein the first basic feature map is a basic feature map of a bounding box image corresponding to the instance group; and
the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image comprises:
training the to-be-trained segmentation network based on the first feature fusion map and the bounding box image.

5. The method according to claim 1 , wherein before the inputting the original image into a segmentation network that has been trained, the method further comprises:
scaling the original image to a preset size in the segmentation network.

6. The method according to claim 1 , wherein the size of the first basic feature map, the size of the first attention feature map, and a size of the first feature fusion map are all preset sizes in the to-be-trained segmentation network;
before the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, the method further comprises:
performing scaling processing on the size of the first feature fusion map and/or a size of the sample original image, to enable the size of the first feature fusion map to be the same as the size of the sample original image.

7. An instance segmentation apparatus, comprising
at least one processor;
one or more memories coupled to the at least one processor and storing programming instructions for execution by the at least one processor to perform a method comprising:
obtaining an original image;
inputting the original image into a segmentation network that has been trained; and
obtaining at least one feature fusion map corresponding to the original image, wherein the feature fusion map is used for marking pixels of an instance comprised in the original image, and each feature fusion map comprises at least one instance, wherein
the segmentation network is trained by operations comprising
for each instance group that is in a sample original image and that is of pixels of a marked instance, wherein each instance group comprises at least one marked instance, performing, by a to-be-trained segmentation network, the following processing comprising:
predicting at least two different first basic feature maps and a first attention feature map corresponding to each of the at least two different first basic feature map;
performing weighted processing on the at least two first basic feature maps and pixel values of respective first attention feature maps corresponding to the at least two first basic feature maps corresponding to the at least two first basic feature maps, to predict a first feature fusion map; and
training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, wherein
a size of the first attention feature map is the same as that of the first basic feature map, a pixel value of each pixel in the first attention feature map indicates a weight value of a pixel at a corresponding location in the first basic feature map corresponding to the first attention feature map, and there are pixels with different pixel values in the first attention feature map.

8. The apparatus according to claim 7 , wherein a value range of the pixel value of the first attention feature map is 0 to 1.

9. The apparatus according to claim 7 , wherein the sample original image is further marked with a bounding box, and the bounding box is used for identifying an instance; and
when the at least one feature fusion map corresponding to the original image is obtained, the method further comprises:
obtaining a bounding box of the instance comprised in the original image.

10. The apparatus according to claim 9 , wherein the first basic feature map is a basic feature map of a bounding box image corresponding to the instance group; and
the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image comprises:
training the to-be-trained segmentation network based on the first feature fusion map and the bounding box image.

11. The apparatus according to claim 7 , wherein before the inputting the original image into a segmentation network that has been trained, the method further comprises:
scaling the original image to a preset size in the segmentation network.

12. The apparatus according to claim 7 , wherein the size of the first basic feature map, the size of the first attention feature map, and a size of the first feature fusion map are all preset sizes in the to-be-trained segmentation network;
before the training the to-be-trained segmentation network based on the first feature fusion map and the sample original image, the method further comprises:
performing scaling processing on the size of the first feature fusion map and/or a size of the sample original image, to enable the size of the first feature fusion map to be the same as the size of the sample original image.

13. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions; and when the computer instructions are executed by a computer, the computer is enabled to perform method comprising:
obtaining an original image;
inputting the original image into a segmentation network that has been trained, and
obtaining at least one feature fusion map corresponding to the original image, wherein the feature fusion map is used for mark

Assignees

• Huawei Tech Co Ltd

Inventors

• Sun Kunyang
• Chen Hao
• Shen Chunhua
• Yan Youliang
• Shao Bin
• Xu Songcen

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/82

  using neural networks · CPC title