Semantic segmentation network model uncertainty quantification method based on evidence inference

Patent metadata

Field	Value
Publication number	US-12293278-B2
Application number	US-202117453983-A
Country	US
Kind code	B2
Filing date	Nov 8, 2021
Priority date	Aug 31, 2021
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

A semantic segmentation network model uncertainty quantification method based on evidence inference. The method comprises the steps of constructing an FCN network model, and training the FCN network model by using a training data set to obtain a trained FCN network model for semantic segmentation of image data; transplanting a D-S theory of evidence to the trained FCN network model to obtain a reconstructed FCN network model; and inputting to-be-segmented image data into the reconstructed FCN network model, outputting a classification result of a to-be-segmented image by the FCN network model, and calculating a classification result uncertainty value of each pixel point by using the D-S theory of evidence index.

First claim

What is claimed is:

1. A semantic segmentation network model uncertainty quantification method based on evidence inference, comprising a non-transitory computer readable medium operable on a computer with memory for the semantic segmentation network model uncertainty quantification method, and comprising program instructions for executing the following steps of:
i) constructing an FCN network model, and training the FCN network model by using a training data set to obtain a trained FCN network model for semantic segmentation of image data;
transplanting a D-S theory of evidence to the trained FCN network model to obtain a reconstructed FCN network model;
ii) inputting to-be-segmented image data into the reconstructed FCN network model, outputting a classification result of a to-be-segmented image by the FCN network model, and calculating a classification result uncertainty value of each pixel point by using the D-S theory of evidence index;
iii) converting information of a real world into to-be-segmented images for computer visualizing, deep learning and neural network predicting based on the semantic segmentation network model; and
iv) realizing autonomous driving, drones, meta-universe, virtual reality (VR) and augmented reality (AR) based on results of the semantic segmentation network model uncertainty quantification method.

2. The method according to claim 1 , wherein the step of constructing the FCN network model comprises that:
up sampling/deconvolution operation is performed on feature maps reduced by a convolutional layer and a pooling layer in a convolutional neural network to realize recovery of feature space information and obtain a fully convolutional network FCN network model, and parameters of the FCN network model are trained, wherein the parameters of the FCN network model include an activation value of the feature map and the weight of a filter, parameter layers of the FCN network model are all convolutional layers, the size of a convolution kernel of the last convolutional layer is 1*1, and a fully connected layer of the FCN network model is not used for executing an image segmentation task.

3. The method according to claim 2 , wherein the step of training the FCN network model by using the training data set to obtain the trained FCN network model for semantic segmentation of the image data comprises that:
a known original image data set and label data are obtained, wherein preprocessed images in the original image data set are the same in size m*n, the training data set is formed through the original image data set and the label data, the training data set is input into the FCN network model, and parameters of an optimized classification model are automatically calculated through a loss function to obtain the trained FCN network model used for semantic segmentation of the image data.

4. The method according to claim 3 , wherein the step of transplanting the D-S theory of evidence to the trained FCN network model to obtain the reconstructed FCN network model comprises that:
the D-S theory of evidence is transplanted to the trained FCN network model, and an original evidence pool m jk is obtained through calculation based on the D-S theory of evidence; and then a Dempster combination rule is applied to the original evidence pool m jk , confidence m 1 and uncertainty measurement indexes of K categories are obtained by calculation, wherein the uncertainty measurement indexes are used for evaluating uncertainty of a predictive result, plausibility transformation is performed on the confidence m 1 of the K categories to obtain a normalized plausibility function P m , and an output result P(C k ) of the FCN network model is equivalent to the plausibility function P m (C k ) obtained based on an evidence inference method so as to obtain the reconstructed FCN network model.

5. The method according to claim 4 , wherein the step of inputting the to-be-segmented image data into the reconstructed FCN network model, and outputting the classification result of the to-be-segmented image by the FCN network model comprises that:
the to-be-segmented image data is input into the reconstructed FCN network model, in case of assuming to segment the to-be-segmented image data into K+1 categories and adding a ‘background’ category, the last convolutional layer of the FCN network model is set to include K filters, and the size and the number of channels of the last layer of feature maps of the FCN network model are set into n H , n W and n C respectively;
semantic segmentation is performed on the to-be-segmented image data through the reconstructed FCN network model, an activation value φ(X) is extracted from the last group of feature maps of the FCN network model, wherein φ(X) is an activation value of J dimension (J=n H *n W *n C ), and meanwhile, the weight w and the bias b of the filter of the last convolutional layer of the FCN network model are extracted; and an original evidence pool m jk are calculated through the activation value φ(X), the weight w and the bias b;
the Dempster combination rule is applied to the original evidence pool m jk , and the confidence m 1 of the K categories is obtained by calculation, wherein the output result P(C K ) of the FCN network model is as follows:
P ( C K )=softmax( w *φ( X )+ b )  (1-19)

predictive data of N*n H *n W *n C *(K+1) dimension is obtained, wherein N is the number of input picture samples, n H , n W and n C are the size: height, width and number of color channels of an original picture respectively, and K is the classification number of categories.

6. The method according to claim 5 , wherein the step of calculating the classification result uncertainty value of each pixel point by using the D-S theory of evidence index comprises that:
all activation values φ(X) of the last layer of feature maps of the FCN model and the weight w and bias b of the filter obtained by training are extracted, wherein in the D-S theory of evidence, the conflict represents the conflict degree between evidences, and a calculation method of the conflict F between two evidences m 1 and m 1 sees Formula (2-5);

a
.

F

=

∑

x
⋂
y

=
∅

m
1

(
x
)

×

m
2

(
y
)

=

1
-

∑

x

Assignees

• Univ Beijing Jiaotong

Inventors

• Wang Rui
• Liang Ci
• Zheng Wei
• Zhang Yumei

Classifications

• G06F18/2163

  Partitioning the feature space · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title