Method and apparatus for real-time detection of polyps in optical colonoscopy

Patent metadata

Field	Value
Publication number	US-11017526-B2
Application number	US-201716314817-A
Country	US
Kind code	B2
Filing date	Jun 19, 2017
Priority date	Jul 4, 2016
Publication date	May 25, 2021
Grant date	May 25, 2021

Abstract

A method for performing real-time detection and displaying of polyps in optical colonoscopy, includes a) acquiring and displaying a plurality of real-time images within colon regions to a video stream frame rate, each real-time image comprising a plurality of color channels; b) selecting one single color channel per real-time image for obtaining single color pixels; c) scanning the single color pixels across each the real-time image with a sliding sub-window; d) for each position of the sliding sub-window, extracting a plurality of single color pixels local features of the real-time image; e) passing the extracted single color pixels local features of the real-time image through a classifier to determine if a polyp is present within the sliding sub-window; f) real-time framing on display of colon regions corresponding to positions of the sliding sub-window wherein polyps are detected. A system for carrying out such a method is also provided.

First claim

The invention claimed is:

1. A method for performing real-time detection and displaying of polyps in optical video-colonoscopy, wherein the method comprises the steps of:
a) acquiring and displaying a plurality of real-time images within colon regions of a video stream frame rate, each real-time image comprising a plurality of color channels;
b) selecting only one single color channel for all the real-time images for obtaining single color pixels;
c) scanning the single color pixels across each said real-time image with a sliding sub-window;
d) for each position of said sliding sub-window, extracting local features from the single color pixels within the sliding sub-window of the real-time image, all the local features being only based on single-color pixels, a local feature being a function of neighboring single color pixels surrounding a given single color pixel;
e) passing the extracted local features of single color pixels of each position of the sliding sub-window through a classifier to determine if a Region of Interest (ROI), containing a polyp, is detected for at least one series of n successive images which comprises an image I f destined to display the ROI,
for each image of the series, spatial fusion of the sub-windows in which a same polyp is detected, for the sub-windows overlapping each other spatially on the image with at least x % of their size, to obtain ROI final in each successive image,
for each series of images, temporal fusion of the ROI final in only one ROI displayed , for the ROI final overlapping each other with at least y % of their size in the fixed referential of the images, real-time framing on display of colon regions corresponding to position of ROI displayed in the image I f , x and y being a non-zero number, and
the ROI being delimited by at least one sub-window generated by the sliding sub-window; and
f) real-time framing on display of colon regions corresponding to Regions Of Interest of said sliding sub-window wherein polyps are detected.

2. The method of claim 1 , wherein, for each image, the step c) of scanning is performed p times, with p being an odd number and greater than one, each time using a different size of the sliding window, the classifier is then applied to all p scans in order to decide whether a polyp is detected or not with a majority vote.

3. The method of claim 1 , wherein x is greater than or equal to 50, and y is greater than or equal to 70.

4. The method of claim 2 , wherein the ROI displayed is calculated for some series of n successive images, n being an odd number greater than or equal to 3, and a polyp is considered present in the ROI final if the polyp is detected at least (n+1)/2 times in the series.

5. The method of claim 1 , wherein the image I f is the final image of the series.

6. The method of claim 1 , wherein the time of the all the steps a), b), c), d), e), f) lasts less than 40 ms.

7. The method of claim 1 , wherein the scanning of the step c) is realized without polyp boundaries detection.

8. The method of claim 1 , wherein said single color channel is blue.

9. The method of claim 1 , wherein local features are chosen from the group comprising local binary patterns and Haar-like features.

10. The method of claim 1 , wherein each local feature is associated to a respective classifier, called weak classifier, the classifier used in step e) of the method comprising a sum of at least one hundred weak classifiers.

11. The method of claim 10 , wherein the classifier is based on a boosting algorithm.

12. The method of claim 11 , wherein said boosting algorithm is cascade Adaboost.

13. The method of claim 11 , further comprising a preliminary step of creating said classifier by active learning.

14. The method of claim 13 , wherein said active learning is carried out using a learning database or video comprising a sequence of images, wherein said images include ground truth images of known polyps, the active learning comprising the steps of:
s1) selecting an initial set of sub-images with and without polyps, extracted from a set of said images for training, and another set of said images for testing; and selecting one single color channel from all of said images for obtaining single color pixels for the sub-images used for training and the images for testing;
s2) extracting local features for training from the initial set of single-color sub-images used for training and local features for testing from the set of images for testing;
s3) computing a classifier based on the boosting algorithm applied on the local features of the initial set of sub-images used for training, and testing the first classifier on the local features of the sub-set of images for testing;
s4) for each sliding sub-window considered on the images used for testing, identifying false positive detection cases of polyps by said classifier during said testing of step s4), and creating an additional set of sub-images which present the false positive detection cases; and
s5) using said false positive detection cases of polyps detected in the additional set of sub-images, to re-compute the classifier based on the boosting algorithm applied on the local features of the initial set of sub-images and on the local features of the additional set of sub-images,
steps s4) to s5) being repeated a plurality of times to create a final classifier, and the classifier used in step e) of the method being said final classifier.

15. The method according to claim 1 , wherein said real-time images are acquired at a minimum frame rate of 24 images per second.

16. The method of claim 1 , wherein said sub-windows comprises n×m pixels, with n and m greater than or equal to 30, and wherein step d) comprises extracting at least 5 local features for each single color pixel of the sliding sub-window.

17. The method according to claim 1 , wherein the plurality of real-time images forms a high-definition or a standard definition video.

18. A system for real-time image detection and displaying of polyps in optical video-colonoscopy, comprising an input port for receiving a video stream, an image processor for processing images from said video stream and an output port for outputting processed images, wherein the image processor is configured for:
a) acquiring and displaying a plurality of real-time images within colon regions to a video stream frame rate, each real-time image comprising a plurality of color channels;
b) selecting only one single color channel for all the real-time images for obtaining single color pixels;
c) scanning the single color pixels across each said real-time image with a sliding sub-window;
d) for each position of said sliding sub-window, extracting local features from the single color pixels within the sliding sub-window of the real-time image, all the local features being based only on single-color pixels from the selected single color channel selected, a local feature being a function of neighbouring single color pixels surrounding a given single color pixel;
e) passing the extracted single color pixels local features of each sliding sub-window through a classifier to determine if a polyp is present within a region, called a Region Of Interest (ROI), for at least one series of n successive images with an image I f displaying the ROI of the sliding sub-windows,
for each image of the series, spatial fusion of the sub-windows in which a same polyp is detected, in each successive image, for the sub-windows overlapping each other spatially on the image with at least x % of their size, to obtain ROI final in eac

Assignees

• Centre National De La Rech Scientifique—Cnrs—
• Univ De Cergy Pontoise
• Ecole Nat Sup Electronique Application
• Hopitaux Paris Assist Publique
• Univ Sorbonne
• Centre Nat Rech Scient

Inventors

• Angermann Quentin
• Dray Xavier
• Romain Olivier
• Histace Aymeric

Classifications

• G06V10/7747

  Organisation of the process, e.g. bagging or boosting · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F18/254

  of classification results, e.g. of results related to same input data · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title