Image analysis system and operating method of the same

Patent metadata

Field	Value
Publication number	US-2022019846-A1
Application number	US-202117375650-A
Country	US
Kind code	A1
Filing date	Jul 14, 2021
Priority date	Jul 17, 2020
Publication date	Jan 20, 2022
Grant date	—

Abstract

Disclosed is an image analysis system which includes a first analyzer and a second analyzer. The first analyzer generates first features by encoding images through a first model and adjusts a weight of the first model based on the first features. The second analyzer generates second features based on the first features by encoding the images through a second model, classifies the second features into classes, respectively, and adjusts a weight of the second model based on mutual information and a correlation between a first class among the classes which features corresponding to an original image group from among the second features are classified as and a second class among the classes which features corresponding to an augmentation group of the original image group from among the second features are classified as.

First claim

What is claimed is:

1 . An image analysis system comprising:
a first analyzer configured to generate first features by encoding images through a first model and to adjust a weight of the first model based on the first features; and a second analyzer configured to generate second features based on the first features by encoding the images through a second model, to respectively classify the second features into classes, and to adjust a weight of the second model based on mutual information and a correlation between a first class among the classes which features corresponding to an original image group from among the second features are classified as and a second class among the classes which features corresponding to an augmentation group of the original image group from among the second features are classified as.

2 . The image analysis system of claim 1 , wherein the first analyzer groups a feature being the closest in value to a target feature from among the first features based on the first model whose weight is adjusted.

3 . The image analysis system of claim 2 , wherein the first analyzer transforms the feature being the closest in value to the target feature so as to be close in distance to the target feature in an embedding space.

4 . The image analysis system of claim 1 , wherein the first analyzer generates the first features by extracting features from the images, reducing a dimension of the features to a given dimension, and normalizing the dimension-reduced features to an embedding space.

5 . The image analysis system of claim 1 , wherein the first analyzer is configured to:
calculate a first probability that a feature being the most neighboring to a target feature from among the first features is classified as a sub-class and a second probability that the target feature is absent from the sub-class, based on a similarity between features corresponding to the original image group from among the first features and features corresponding to the augmentation group from among the first features; and adjust the weight of the first model based on the first probability and the second probability.

6 . The image analysis system of claim 1 , wherein the second analyzer is configured to:
transform a feature being the most neighboring in value to a target feature from among the second features based on the second model whose weight is adjusted, so as to be close in distance to the target feature; and classify the transformed features into the classes.

7 . The image analysis system of claim 1 , wherein the second analyzer decides an initial value for calculating the second features based on the first features.

8 . The image analysis system of claim 1 , wherein the second analyzer generates the second features by extracting features from the images, reducing a dimension of the features to a given dimension, and normalizing the dimension-reduced features to an embedding space based on the first features.

9 . The image analysis system of claim 1 , wherein the second analyzer is configured to:
calculate a first probability variable of the first class and a second probability variable of the second class; calculate a clustering loss based on a joint probability distribution between the first and second probability variables; and adjust the weight of the second model based on the clustering loss such that the mutual information increases.

10 . The image analysis system of claim 1 , wherein the second analyzer is configured to:
calculate a first probability that a target feature among the features corresponding to the augmentation group is classified as a target class and a second probability that a feature corresponding to the target feature from among the features corresponding to the original image group is not classified as the target class; and adjust the weight of the second model based on the first probability and the second probability.

11 . The image analysis system of claim 1 , wherein the second analyzer is configured to:
calculate a first loss based on the mutual information; calculate a second loss based on the correlation; and adjust the weight of the second model such that a sum of the first loss and the second loss to which a weight is given.

12 . The image analysis system of claim 1 , wherein a label is not given to the images.

13 . An operating method of an image analysis system, the method comprising:
generating first features by encoding original images through a first model; generating second features by encoding augmentation images of the original images through the first model; adjusting a weight of the first model based on the first features and the second features; generating third features by encoding the original images through a second model based on the first features; generate fourth features by encoding the augmentation images through the second model based on the second features; clustering the third and fourth features; and adjusting a weight of the second model based on mutual information and a correlation between the third and fourth features thus clustered.

14 . The method of claim 13 , wherein the adjusting of the weight of the first model includes:
deciding an initial value for generating the third features and the fourth features based on the first features and the second features.

15 . The method of claim 13 , wherein the generating of the first features or the generating of the second features includes:
extracting features from the original images or the augmentation images; reducing a dimension of the extracted features to a given dimension; and normalizing the dimension-reduced features to an embedding space.

16 . The method of claim 13 , wherein the generating of the third features or the generating of the fourth features includes:
extracting features from the original images or the augmentation images; reducing a dimension of the extracted features to a given dimension; and normalizing the dimension-reduced features to an embedding space based on the first and second features.

17 . The method of claim 13 , wherein the adjusting of the weight of the second model includes:
calculating a first loss based on a joint probability distribution between a first probability variable of a first class which each of the third features is classified as and a second probability variable of a second class which each of the fourth features is classified as; calculating a second loss according to an event that a third feature and a fourth feature corresponding to each other from among the third features and the fourth features are classified into different classes, based on the correlation between the third and fourth features thus clustered; and adjusting the weight of the second model such that a weighted sum of the first loss and the second loss decreases.

Assignees

• Inst Basic Science

Inventors

• Cha Meeyoung
• Han Sungwon
• Park Sungwon
• Kim Sundong
• Park Sungkyu

Classifications

• G06F18/2321 Primary

  using statistics or function optimisation, e.g. modelling of probability density functions · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06T7/143

  involving probabilistic approaches, e.g. Markov random field [MRF] modelling · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06T2207/20081

  Training; Learning · CPC title