Unsupervised domain adaptation method, device, system and storage medium of semantic segmentation based on uniform clustering

Patent metadata

Field	Value
Publication number	US-11734390-B2
Application number	US-202117408441-A
Country	US
Kind code	B2
Filing date	Aug 22, 2021
Priority date	May 18, 2021
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

The present disclosure discloses an unsupervised domain adaptation method, a device, a system and a storage medium of semantic segmentation based on uniform clustering; first, a prototype-based source domain uniform clustering loss and an empirical prototype-based target domain uniform clustering loss are established, to reduce intra-class differences of pixels responding to the same category; meanwhile, the pixels with similar structures but different classes are driven away from each other, wherein they tend to be evenly distributed, increasing the inter-class distance and overcoming the problem that the category boundaries are unclear during the domain adaptation process; next, the prototype-based source domain uniform clustering loss and the empirical prototype-based target domain uniform clustering loss are integrated into an adversarial training framework, which reduces the domain difference between the source domain and the target domain, thus improving the accuracy of semantic segmentation.

First claim

What is claimed is:

1. An unsupervised domain adaptation method of semantic segmentation based on uniform clustering, comprising:
establishing a source domain dataset with labels and a target domain dataset without labels; the source domain dataset comprising source domain images and semantic labels of the source domain images, the target domain dataset comprising target domain images;
establishing an unsupervised domain adaptation network model; the unsupervised domain adaptation network model comprising a semantic segmentation network model for performing semantic segmentation on the source domain images and the target domain images and a discriminator model for adversarial training;
establishing an objective function of the unsupervised domain adaptation network model; the objective function of the unsupervised domain adaptation network model comprises semantic segmentation loss for monitoring performance of the semantic segmentation network model, a prototype-based source domain uniform clustering loss, an empirical prototype-based target domain uniform clustering loss, and an adversarial loss for monitoring the performance of the discriminator model; wherein the adversarial loss for monitoring the performance of the discriminator model is denoted as adv , and the calculation process is:

ℒ
adv

=

-

∑

h
,
w

H
,
W

log
⁡
(

1
-

D
⁡
(

I
s

(

h
,
w

)

)

)

+

log
⁡
(

D
⁡
(

I
t

(

h
,
w

)

)

)

wherein I s (h,w) is an entropy map of (h,w) position pixel x s (h,w) in the source domain image x s , I t (h,w) is an entropy map of (h,w) position pixel x t (h,w) in the target domain image x t , and D( ) denotes a domain probability that the discriminator model D determines that the input entropy map comes from the target domain;
the entropy map I s (h,w) , and entropy map I t (h,w) are calculated by the following equations:

I
s

(

h
,
w

)

=

-

∑
c
C

P
s

(

h
,
w
,
c

)

⁢
log
⁢

P
s

(

h
,
w
,
c

)

⁢

I
t

(

h
,
w

)

=

-

∑
c
C

P
^

t

(

h
,
w
,
c

)

⁢
log
⁢

P
^

t

(

h
,
w
,
c

)

obtaining a semantic segmentation network model of which parameters are optimized by using the source domain dataset and the target domain dataset, and using the objective function to optimize network parameters of the unsupervised domain adaptation network model;
detecting the target domain images to be detected by using the semantic segmentation network model of which the parameters are optimized, to obtain a semantic label of the target domain image.

2. The unsupervised domain adaptation method of semantic segmentation based on uniform clustering according to claim 1 , wherein the semantic segmentation network model uses ResNet as a bas

Assignees

• Univ Zhejiang

Inventors

• Yin Jianwei
• Su Ge
• Shang Yongheng
• Yang Yingchun
• Deng Shuiguang

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06F18/2193 Primary

  based on specific statistical tests · CPC title

• G06F18/2132

  based on discrimination criteria, e.g. discriminant analysis · CPC title