Method and system for unsupervised deep representation learning based on image translation

Patent metadata

Field	Value
Publication number	US-12567227-B2
Application number	US-202118274217-A
Country	US
Kind code	B2
Filing date	Nov 24, 2021
Priority date	Jan 29, 2021
Publication date	Mar 3, 2026
Grant date	Mar 3, 2026

Abstract

A system for unsupervised deep representation learning based on image translation is provided. The system includes an image translation transformation module used for performing a random translation transformation on an image and generating an auxiliary label; an image mask module connected with the image translation transformation module and used for applying a mask to the image after translation transformation; a deep neural network connected with the image mask module and used for predicting an actual auxiliary label of the image after the mask is applied and learning the deep representation of the image; a regression loss function module connected with the deep neural network and used for updating parameters of the deep neural network based on a loss function; and a feature extraction module connected with the deep neural network and used for extracting the representation of the image.

First claim

What is claimed is:

1 . A system for an unsupervised deep representation learning based on an image translation, comprising:
an image translation transformation module configured for performing a translation transformation on an image and generating an auxiliary label; an image mask module connected with the image translation transformation module and configured for applying a mask to the image after the translation transformation; a deep neural network connected with the image mask module and configured for predicting an actual auxiliary label of the image after the mask is applied and learning a deep representation of the image; a regression loss function module connected with the deep neural network and configured for updating parameters of the deep neural network based on a loss function; and a feature extraction module connected with the deep neural network and configured for extracting the deep representation of the image.

2 . The system for the unsupervised deep representation learning based on the image translation according to claim 1 , wherein as performing the translation transformation on the image by the image translation transformation module, the image after the translation transformation is represented as: (x i |t),
wherein an image dataset X={x i ∈ C×W×H } i=1 N comprising N samples is given, each image x i is represented by a matrix C×W×H, and C, W, H are a number of channels, a width of the image, and a height of the image, respectively; an image translation transformation function is represented as (⋅|t), and t=[t w , t h ] is a translation transformation parameter; t w ∈(−1,1) is a horizontal translation parameter, when t w ≥0, a width scale of translating rightwards is represented as the t w , that is, translating rightwards t w *W pixels, and when t w <0, a width scale of translating leftwards is represented as −t w , that is, translating leftwards (−t w *W) pixels; t h ∈(−1,1) is a vertical translation parameter, when t h ≥0, t h represents a height scale of translating downwards, that is, translating downwards t h *H pixels, and when t h <0, −t h represents a height scale of translating upwards, that is, translating upwards (−t h *H) pixels; and t represents the auxiliary label.

3 . The system for the unsupervised deep representation learning based on the image translation according to claim 2 , wherein the mask in the image mask module is represented as:
( t )= (1| T ∘ sign( t ))= (1|[ T w *sign( t w ), T h *sign( t h )])
wherein C×W×H represents a matrix C×W×H with all 1 elements; T=[T w , T h ] represents a maximum scale allowing the translation transformation; t=[t w , t h ] represents performing the translation transformation with the auxiliary label; and sign represents a symbolic function defined as: a process of applying the mask to the image (x i |t) after the translation transformation is (x i |t)∘ (t), that is, an image matrix after the translation transformation and a mask matrix are multiplied by corresponding elements.

4 . The system for the unsupervised deep representation learning based on the image translation according to claim 3 , wherein the loss function in the regression loss function module is represented as:

ℒ
=

1
N

⁢

∑

i
=
1

N



F
⁡
(

(

x
i

❘
t

)

∘

⁢

(
t
)

❘
Ω

)

-
t



2
2

wherein F(⋅|Ω) represents a mapping function of the deep neural network; Ω represents all training parameters of the deep neural network; N represents a number of training samples; and (t) represents the mask.

5 . The system for the unsupervised deep representation learning based on the image translation of claim 4 , wherein extracting the deep representation of the image by the feature extraction module is obtained by extracting a trained deep neural network.

6 . A method for an unsupervised deep representation learning based on an image translation, comprising:
S 1 , performing a translation transformation on an image and generating an auxiliary label; S 2 , applying a mask to the image after the translation transformation; S 3 , predicting an actual auxiliary label of the image after the mask is applied and learning a deep representation of the image; S 4 , updating parameters of a deep neural network based on a loss function; and S 5 , extracting the deep representation of the image; wherein as performing the translation transformation on the image in step S 1 , the image after the translation transformation is represented as: (x i |t), wherein an image dataset X={x i ∈ C×W×H } i=1 N comprising N samples is given, each image x i is represented by a matrix C×W×H, and C, W, H are a number of channels, a width of the image, and a height of the image, respectively; an image translation transformation function is represented as (⋅|t), and t=[t w , t h ] is a translation transformation parameter; t w ∈(−1,1) is a horizontal translation parameter, when t w ≥0, a width scale of translating rightwards is represented as t w , that is, translating rightwards t w *W pixels, and when t w <0, a width scale of translating leftwards is represented as −t w , that is, translating leftwards (−t w *W) pixels; t h ∈(−1,1) is a vertical translation parameter, when t h ≥0, t h represents a height scale of translating downwards, that is, translating downwards t h *H pixels, and when t h <0, −t h represents a height scale of translating upwards, that is, translating upwards (−t h *H) pixels; and t represents the auxiliary label.

7 . The method for the unsupervised deep representation learning based on the image translation according to claim 6 , wherein the mask in step S 2 is represented as:
( t )= (1| T ∘ sign( t ))= (1|[ T w *sign( t w ), T h *sign( t h )])

wherein C×W×H represents a matrix C×W×H with all 1 elements; T=[T w , T h ] represents a maximum scale allowing the translation transformation; t=[t w , t h ] represents performing the translation transformation with the auxiliary label; and sign

Assignees

• Univ Zhejiang Normal

Inventors

• Zhu Xinzhong
• Xu Huiying
• Guo Xifeng
• Dong Shihao
• Zhao Jianmin

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06T7/10

  Segmentation; Edge detection (motion-based segmentation G06T7/215) · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title