Device and method for producing a digital video classifier

Patent metadata

Field	Value
Publication number	US-12175663-B2
Application number	US-202117487070-A
Country	US
Kind code	B2
Filing date	Sep 28, 2021
Priority date	Sep 30, 2020
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

The present invention concerns a device for producing a “digital video classifier” configured to determine the quality of cleanliness of one or more segments of the digestive tube in a video capsule endoscopy (VCE) of a subject, comprising: a VCE allowing the acquisition of videos of segments of the digestive tube, video storage means, coupled with the VCE, an “image database” with images extracted from VCE exams, a “video database” with videos extracted from VCE exams, calculating means connected to the video storage means, and to the databases, and configured for performing a statistical learning for the generation of a “digital image classifier” from the “image database” and which classifies the images in adequate cleanliness or non-adequate cleanliness; generating the “digital video classifier” capable of classifying a video of a subject as being of adequate cleanliness or of non-adequate cleanliness of one or more segments of the digestive tube.

First claim

The invention claimed is:

1. A device for producing a digital video classifier configured to determine a quality of cleanliness of one or more segments of a digestive tube of a subject from a capsule endoscopy video of the subject digestive tube, comprising:
a data storage medium configured to store:
an image database with images extracted from video capsule endoscopy-VCE-exams and categorized into predetermined images categories including images with adequate cleanliness and images with non-adequate cleanliness; said predetermined image categories referring to a score determined by a visual analysis of the images based on at least one medical criterion, and
a video database with videos extracted from VCE exams categorized into predetermined video categories including videos with adequate cleanliness and videos with non-adequate cleanliness; said predetermined video categories referring to a score determined by a visual analysis of the videos, based on at least one medical criterion,
the at least one medical criterion being selected from: percentage of mucosa visualized, luminosity, presence of bubbles, presence of bile/chyme, presence of liquids and/or undigested debris,

one or several processors connected to the data storage medium, and configured for:
(A) performing a statistical learning for generation of a digital image classifier from the image database and which classifies the images in adequate cleanliness or non-adequate cleanliness;
(B) generating the digital video classifier, by the following steps:
a) extraction of all the successive images of each video from the video database;
b) automatic classification of the quality of all the successive images from each extracted video, in adequate cleanliness or non-adequate cleanliness, by the digital image classifier, and computing a video cleanliness score for each video according to a proportion of images whose quality is of adequate cleanliness;
c) automatic classifications of the videos into videos with adequate cleanliness and videos with non-adequate cleanliness, each of said classifications of the videos being done with one of a plurality of threshold values applied to the video cleanliness score;
d) computing a plurality of receiver operating characteristics-ROC-for each of the automatic classifications of the videos according to the predetermined video categories;
e) computing of a desired threshold value between 0% and 100% of the video cleanliness score allowing to obtain desired values in terms of ROC performance, and
f) validating the desired threshold of the video cleanliness score with a further set of videos from the video database, wherein the validation of said desired threshold comprises comparison of the receiver operating characteristic such as values of sensitivity and specificity obtained with a first subset of videos from the further set of videos with the receiver operating characteristic obtained with a second subset of videos from the further set of videos,

the digital video classifier being validated in step (f) when a variation of obtained receiver operating characteristic such as values of sensitivity and specificity is 2% maximum between the second subset of videos and the first subset of videos, for the desired threshold,
the value of this desired threshold of the video cleanliness score becoming, in combination with the digital image classifier, the digital video classifier capable of classifying a video of one or more segments of a subject digestive tube as being of adequate cleanliness or of non-adequate cleanliness.

2. The device for producing a digital video classifier according to claim 1 , wherein in step f) the two subsets of videos the video database are selected so as to have the same proportions of adequate cleanliness/non-adequate cleanliness videos according to the predetermined video categories.

3. The device for producing a digital video classifier according to claim 1 , wherein the computing a plurality of ROCs in the steps c) and d) comprises the following sub-steps:
variation between 0% and 100%, of the plurality of threshold values applied to the video cleanliness score corresponding to the proportion of images beyond which each video is considered adequate,
labeling of each video in a subset as adequate cleanliness video or not adequate cleanliness video depending on the threshold values,
for each given threshold value, counting a number of False Positives, True Positives, False Negatives and True Negatives in the videos of the first subset, by comparison of the video labels with the predetermined videos categories, to determine associated values of receiver operating characteristic associated with the given threshold value, and
plotting a ROC curve from the set of values obtained for each threshold value.

4. The device for producing a digital video classifier according to claim 1 , wherein the receiver operating characteristics includes true positive rate and false positive rate at various threshold settings.

5. The device for producing a digital video classifier according to claim 1 , wherein in step (A) learning is realized by a deep neural network architecture of the Convolutional Neural Network (CNN) type or of the Generative Adversarial Network (GAN) type.

6. The device for producing a digital video classifier according to claim 5 , wherein the calculating means perform, in the step (A), a statistical learning for the generation of the digital image classifier from the image database further comprising:
(i) a step of automatic learning according to a technique known as “deep neural networks” on a subset of the image database drawn at random, allowing the generation of a digital image classifier, and
(ii) a test step on the remaining images of the image database enabling specificity and sensitivity of the digital image classifier to be validated, by comparison of the automatic classification with the predetermined image categories of these remaining images.

7. The device for producing a digital video classifier according to claim 6 , wherein the digital image classifier is validated when at least the tested images have a specificity and a sensitivity at least equal to 90% compared to the predetermined image categories.

8. A method for producing a digital video classifier configured to determine a quality of cleanliness of one or more segments of a digestive tube of a subject from a capsule endoscopy video of the subject digestive tube, said method being executed by one or several processors connected to a data storage medium configured to store:
an image database with images extracted from video capsule endoscopy-VCE-exams and categorized into predetermined images categories including images with adequate cleanliness and images with non-adequate cleanliness; said predetermined image categories referring to a score determined by a visual analysis of the images based on at least one medical criterion, and
a video database with videos extracted from VCE exams categorized into predetermined video categories including videos with adequate cleanliness and videos with non-adequate cleanliness; said predetermined video categories referring to a score determined by a visual analysis of the videos, based on at least one medical criterion,
the at least one medical criterion being selected from: percentage of mucosa visualized, luminosity, presence of bubbles, presence of bile/chyme, presence of liquids and/or undigested debris,

said method comprising the following steps:
(A) performing a statistical learning for the generation of a “digital image classifier” from the “image database” and which classifies the images in adequate cleanliness or non-adequate cleanliness;
(B) generating the digital video classifier, by the following step

Assignees

• Hopitaux Paris Assist Publique
• Univ Sorbonne
• Centre Nat Rech Scient
• Ecole Nat Superieure De Lelectronique Et De Ses Applications
• Cy Energy Paris Univ
• Cy Cergy Paris Univ

Inventors

• Souchaud Marc
• Histace Aymeric
• Leenhardt Romain
• Dray Xavier

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T2207/30028

  Colon; Small intestine · CPC title

• G06T2207/10068

  Endoscopic image · CPC title