Automatic image analysis method for automatically recognizing at least one rare characteristic

Patent metadata

Field	Value
Publication number	US-12223700-B2
Application number	US-202017610098-A
Country	US
Kind code	B2
Filing date	May 7, 2020
Priority date	May 10, 2019
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

Disclosed is an automatic image analysis method that can be used to automatically recognise at least one rare characteristic in an image to be analysed. The method comprises a learning phase during which at least one convolutional neural network is trained to recognise characteristics, a parameter space of dimension n, in which n≥2, is constructed from at least one intermediate layer of the network, a presence probability function is determined for each characteristic in the parameter space from a projection of reference images in the parameter space. During a phase of analysing the image to be analysed, the method comprises a step of recognising the at least one rare characteristic in the image to be analysed on the basis of the presence probability function determined for the at least one rare characteristic.

First claim

The invention claimed is:

1. An automatic image analysis method, said method comprising a learning phase and an analysis phase, said method comprising:
during said learning phase:

a step of supervised deep learning of at least one convolutional neural network on the basis of a collection of images associated with annotations for generating at least one trained convolutional neural network, arranged in layers and forming a learning model configured to recognize at least one frequent characteristic in an image to be analyzed, said collection of images comprising images having said at least one frequent characteristic and images having at least one rare characteristic, a frequent characteristic corresponding to a characteristic presenting an appearance frequency on the collection of images which is equal or greater than a frequency threshold; a rare characteristic corresponding to a characteristic presenting an appearance frequency on the collection of images which is lower than a frequency threshold;
an annotation indicating whether a rare or frequent characteristic is present in an image of the collection of images;

a step of constructing a parameter space of dimension n, with n ≥2, each parameter originating from at least one intermediate layer of said at least one trained convolutional neural network and each parameter representing a dimension of said parameter space;
a step of determining a presence probability function in said parameter space for at least one rare characteristic on the basis of a projection, in said parameter space, of said images of the collection of images and annotations relating to said at least one rare characteristic;
during said analysis phase:

for an image to be analyzed, a step of recognizing said at least one rare characteristic in said image to be analyzed on the basis of the presence probability function determined for said at least one rare characteristic.

2. The analysis method as claimed in claim 1 , further comprising, during the learning phase, a step of determining a presence probability function in said parameter space for said at least one frequent characteristic on the basis of a projection, in said parameter space, of said images of the collection of images and annotations relating to said at least one frequent characteristic.

3. The analysis method as claimed in claim 2 , wherein the supervised deep learning step of said at least one convolutional neural network is supervised by said presence probability function of said at least one frequent characteristic.

4. The analysis method as claimed in claim 1 , wherein, during the supervised deep learning step, a plurality of convolutional neural networks is trained separately or jointly, with each of said convolutional neural networks being trained to recognize said at least one frequent characteristic.

5. The analysis method as claimed in claim 1 , wherein said parameter space is constructed on the basis of values of output parameters originating from at least one intermediate layer of said at least one convolutional neural network, said at least one intermediate layer being selected from among the penultimate layers of said at least one convolutional neural network.

6. The analysis method as claimed in claim 1 , comprising a step of determining an absence probability function in said parameter space for said at least one rare characteristic on the basis of a projection, in said parameter space, of said images of the collection of images and annotations relating to said at least one rare characteristic,
said at least one intermediate layer being selected based on the maximization, for at least one considered rare characteristic, of the Patrick-Fischer distance between the presence probability function of said at least one considered rare characteristic and the absence probability function of said at least one considered rare characteristic.

7. The analysis method as claimed in claim 1 , wherein the parameter space is a reduced parameter space and the step of constructing the reduced parameter space comprises a step of reducing the dimension of an initial parameter space at the output of said at least one intermediate layer of said at least one trained convolutional neural network.

8. The analysis method as claimed in claim 7 , wherein said step of reducing the dimension of the initial parameter space is based on a principal component analysis algorithm and/or on a t-SNE algorithm.

9. The analysis method as claimed in claim 1 , further comprising, for at least one rare characteristic, a step of determining, in the image to be analyzed, pixels responsible for recognizing said considered rare characteristic.

10. The analysis method as claimed in claim 8 , the recognition step comprising:
constructing a second reduced parameter space by reducing the parameter space at the output of said at least one intermediate layer of said at least one trained convolutional neural network;
projecting, in the second parameter space, the image to be analyzed in order to obtain a projected image;
obtaining reference projected images in the second parameter space by projecting reference images of the collection of images; and
estimating a probability that the image to be analyzed contains the rare characteristic, said probability being computed by regression on the basis of presence probabilities of the rare characteristic determined for the reference images for which the reference projected images are the nearest neighbors of the projected image.

11. The analysis method as claimed in claim 10 , wherein the second reduced parameter space is constructed by means of a principal component analysis applied to the parameter space at the output of said at least one intermediate layer and wherein the projection of an image in the second reduced parameter space is obtained by applying said at least one trained convolutional network to the considered image in order to obtain an output, then applying a projection function originating from the principal component analysis to this output.

12. The analysis method as claimed in claim 10 , wherein the presence probability of a rare characteristic in a reference image is obtained by projecting the reference image in the reduced parameter space constructed for the rare characteristic so as to obtain a reference projected image, then applying the presence probability function defined in the reduced parameter space to the reference projected image.

13. An analysis device configured to implement the steps of the analysis method as claimed in claim 1 .

Assignees

• Univ De Brest
• Inst Nat Sante Rech Med
• Univ Brest Bretagne Occidentale
• Inst Nat Sante Rech Med

Inventors

• Quellec Gwenolé

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06T2207/20081

  Training; Learning · CPC title