Method and apparatus for training semantic segmentation model, computer device, and storage medium

Patent metadata

Field	Value
Publication number	US-11398034-B2
Application number	US-201816759383-A
Country	US
Kind code	B2
Filing date	Jul 13, 2018
Priority date	Apr 20, 2018
Publication date	Jul 26, 2022
Grant date	Jul 26, 2022

Abstract

A method and apparatus for training a semantic segmentation model, a computer device, and a storage medium are described herein. The method includes: constructing a training sample set; inputting the training sample set into a deep network model for training; inputting the training sample set into a weight transfer function for training to obtain a bounding box prediction mask parameter; and constructing a semantic segmentation model.

First claim

What is claimed is:

1. A method for training a semantic segmentation model, comprising:
constructing a training sample set, wherein the training sample set comprises a plurality of first-category objects and a plurality of second-category objects, wherein the first-category objects are marked with bounding boxes and segmentation masks, and the second-category objects are marked with bounding boxes;
inputting the training sample set into a deep network model for training to obtain first bounding box parameters and first mask parameters of the first-category objects and second bounding box parameters of the second-category objects; and inputting the first bounding box parameters and the first mask parameters into a weight transfer function for training to obtain a bounding box prediction mask parameter; and
inputting the first bounding box parameters, the first mask parameters, the second bounding box parameters, and the bounding box prediction mask parameter into the deep network model and the weight transfer function to construct a semantic segmentation model;
wherein a category number of the second-category objects is greater than that of the first-category objects;
wherein the deep network model is a Mask-RCNN network model;
wherein an expression of the weight transfer function is:
ω seg c =τ(ω det c ;θ) w det c =[ w cls c ,w box c ]

wherein τ denotes a transfer function, ω cls denotes a weight of a category, ω box denotes a weight of a bounding box, ω det denotes a merged vector, θ denotes a learning parameter of an unknown category, and ω seg denotes the bounding box prediction mask parameter.

2. The method for training a semantic segmentation model according to claim 1 , wherein after the step of inputting the first bounding box parameters, the first mask parameters, the second bounding box parameters, and the bounding box prediction mask parameter into the deep network model and the weight transfer function to construct a semantic segmentation model, the method comprises:
inputting an image to be segmented into the semantic segmentation model to output a semantic segmentation result of the image to be segmented.

3. The method for training a semantic segmentation model according to claim 2 , wherein the step of inputting an image to be segmented into the semantic segmentation model to output a semantic segmentation result of the image to be segmented comprises:
inputting the image to be segmented into the semantic segmentation model, predicting bounding boxes of the first-category objects in the image to be segmented by using the first bounding box parameters, and predicting bounding boxes of the second-category objects in the image to be segmented by using the second bounding box parameters;
predicting mask parameters of the first-category objects in the image to be segmented by using the bounding boxes of the first-category objects and the bounding box prediction mask parameter, and predicting mask parameters of the second-category objects in the image to be segmented by using the bounding boxes of the second-category objects and the bounding box prediction mask parameter; and
performing semantic segmentation on the first-category objects and the second-category objects in the image to be segmented by using the mask parameters of the first-category objects and the mask parameters of the second-category objects in the image to be segmented.

4. The method for training a semantic segmentation model according to claim 1 , wherein the weight transfer function is a two-layer fully connected neural network, wherein the two fully connected layers have 5120 neurons and 256 neurons, respectively, and an activation function used is LeakyReLU.

5. A computer device, comprising a memory storing computer readable instructions and a processor, wherein a method for training a semantic segmentation model is implemented when the processor executes the computer readable instructions, and the method comprises:
constructing a training sample set, wherein the training sample set comprises first-category objects and second-category objects, wherein the first-category objects are marked with bounding boxes and segmentation masks, and the second-category objects are marked with bounding boxes;
inputting the training sample set into a deep network model for training to obtain first bounding box parameters and first mask parameters of the first-category objects and second bounding box parameters of the second-category objects; and inputting the first bounding box parameters and the first mask parameters into a weight transfer function for training to obtain a bounding box prediction mask parameter; and
inputting the first bounding box parameters, the first mask parameters, the second bounding box parameters, and the bounding box prediction mask parameter into the deep network model and the weight transfer function to construct a semantic segmentation model;
wherein a category number of the second-category objects is greater than that of the first-category objects;
wherein the deep network model is a Mask-RCNN network model;
wherein an expression of the weight transfer function is:
ω seg c =τ(ω det c ;θ) w det c =[ w cls c ,w box c ]

wherein τ denotes a transfer function, ω cls denotes a weight of a category, ω box denotes a weight of a bounding box, ω det denotes a merged vector, θ denotes a learning parameter of an unknown category, and ω seg denotes the bounding box prediction mask parameter.

6. The computer device according to claim 5 , wherein after the step of inputting, by the processor, the first bounding box parameters, the first mask parameters, the second bounding box parameters, and the bounding box prediction mask parameter into the deep network model and the weight transfer function to construct a semantic segmentation model, the method comprises:
inputting an image to be segmented into the semantic segmentation model to output a semantic segmentation result of the image to be segmented.

7. The computer device according to claim 6 , wherein the step of inputting, by the processor, an image to be segmented into the semantic segmentation model to output a semantic segmentation result of the image to be segmented comprises:
inputting the image to be segmented into the semantic segmentation model, predicting bounding boxes of the first-category objects in the image to be segmented by using the first bounding box parameters, and predicting bounding boxes of the second-category objects in the image to be segmented by using the second bounding box parameters;
predicting mask parameters of the first-category objects in the image to be segmented by using the bounding boxes of the first-category objects and the bounding box prediction mask parameter, and predicting mask parameters of the second-category objects in the image to be segmented by using the bounding boxes of the second-category objects and the bounding box prediction mask parameter; and
performing semantic segmentation on the first-category objects and the second-category objects in the image to be segmented by using the mask parameters of the first-category objects and the mask parameters of the second-category objects in the image to be segmented.

8. The computer device according to claim 5 , wherein the weight transfer function is a two-layer fully connected neural network, wherein the two fully connected layers have 5120 neurons and 256 neurons, respectively, and an activation function used is LeakyReLU.

9. A non-transitory computer readable storage medium storing computer readable instructions, wherein a method for training a semantic segmentation model is implemented when the computer readable instructions are execu

Assignees

• Ping An Tech Shenzhen Co Ltd

Inventors

• Wang Jianzong
• Wang Chenyu
• Ma Jin
• Xiao Jing

Classifications

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06T7/10 Primary

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title