Endoscopic image enhancement using contrast limited adaptive histogram equalization (CLAHE) implemented in a processor

Patent metadata

Field	Value
Publication number	US-11330238-B2
Application number	US-202016992312-A
Country	US
Kind code	B2
Filing date	Aug 13, 2020
Priority date	May 17, 2015
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

Systems and methods of enhancing images use a contrast limited adaptive histogram equalization (CLAHE) algorithm in a field programmable gate array (FPGA). The images may be obtained by the imaging elements of a multiple imaging elements endoscope of an endoscopy system.

First claim

We claim:

1. A medical device system comprising:
a medical device configured to be inserted into a body of a patient, the medical device including a first viewing element; and
a control unit operatively coupled to the medical device, wherein the control unit comprises:
one or more processing boards that implement a contrast limited adaptive histogram equalization (CLAHE) algorithm to enhance images obtained by the first viewing element, wherein the one or more processing boards perform the steps of:

receiving a first video stream from the first viewing element,
building first histogram information based on a first portion of the first video stream,

building second histogram information based on a second portion of the first video stream,
converting the first histogram information into first cumulative distribution function values including applying a first clipping threshold to the first histogram information,
converting the second histogram information into second cumulative distribution function values including applying a second clipping threshold to the second histogram information, wherein the second clipping threshold is different from the first clipping threshold,
generating a first enhanced video stream based on the first cumulative distribution function values, and the second cumulative distribution function values, and sending an output video stream indicative of the first enhanced video stream to a display.

2. The endoscope system of claim 1 , wherein the first video stream is received by an adapter of the one or more processing boards, the adapter having a conversion module, and wherein the first video stream is converted from a first color space video stream to a second color space video stream by the conversion module.

3. The endoscope system of claim 1 , wherein the first plurality of frames are obtained from the first video stream using a frame grabber of the one or more processing boards.

4. The endoscope system of claim 3 , wherein the control unit further comprises a base board module, and wherein the base board module further comprises a random access memory module in communication with the frame grabber.

5. The endoscope system of claim 1 , wherein the one or more processing boards further perform the steps of:
obtaining a first plurality of frames from the first video stream, and
rearranging the first plurality of frames to generate a plurality of first delayed video streams, wherein the first plurality of frames are rearranged to generate the first plurality of delayed video streams using a fetching module of the one or more processing boards.

6. The endoscope system of claim 1 , wherein the first enhanced video stream is generated using an interpolation algorithm module of the one or more processing boards.

7. The endoscope system of claim 1 , wherein building first histogram information based on the first video stream includes splitting the first video stream into a first plurality of tiles, wherein units of the tile heights are in video lines and units of the tile widths are in pixels, wherein each of the first plurality of tiles overlaps with its neighboring tiles.

8. A method of enhancing images obtained by a medical device system using a contrast limited adaptive histogram equalization (CLAHE) process, wherein said medical device system comprises (a) one or more processing boards configured to implement said process, and (b) a medical device operatively coupled to the one or more processing boards, wherein the medical device is configured to be inserted into a body of a patient and includes a first viewing element, said method comprising the steps of:
receiving a first video stream from the first viewing element;
splitting the first video stream into a plurality of tiles, wherein each of the plurality of tiles overlaps with its neighboring tiles;
building histogram information based on the plurality of tiles;
converting the histogram information into cumulative distribution function values;
generating an enhanced video stream, using the cumulative distribution function values; and
displaying an output video stream indicative of the enhanced video stream.

9. The method of claim 8 , further comprising the steps of:
obtaining the first video stream as a first plurality of frames; and
rearranging the first plurality of frames to generate a first delayed video stream;
wherein generating an enhanced video stream includes using the first delayed video stream.

10. The method of claim 8 , wherein units of the tile heights are in video lines and units of the tile widths are in pixels.

11. The method of claim 8 , further comprising converting the first video stream from a first color space video stream to a second color space video stream, including the steps of:
performing YCbCr to RGB conversion using MAD instantiation;
performing a function similar to Y(x)=x{circumflex over ( )}2.40 using a LUT;
performing RGB to XYZ conversion using MAD instantiation;
executing a function similar to Y(x)=x{circumflex over ( )}0.33 using a LUT; and
performing XYZ to Lab conversion using MAD instantiation.

12. The method of claim 8 wherein converting the enhanced Lab color space video stream to the enhanced YCbCr color space video stream comprises the steps of:
performing Lab to XYZ conversion using MAD instantiation;
executing a function similar to Y(x)=x{circumflex over ( )}0.33 using a LUT;
performing XYZ to RGB conversion using MAD instantiation;
performing a function similar to Y(x)=x{circumflex over ( )}(1/2.40) using a LUT; and
performing RGB to YCbCr conversion using MAD instantiation.

13. A non-transient computer readable medium containing program instructions for causing a computer to perform a method of enhancing images obtained by a first viewing element in a medical device system using a contrast limited adaptive histogram equalization (CLAHE) process, wherein said medical device system comprises a processor configured to implement said process, and a medical device operatively coupled to the processor, the medical device being configured for insertion into a body of a patient and including the first viewing element, said method comprising the steps of:
receiving a first video stream from the first viewing element,
building first histogram information based on a first component of the first video stream,
building second histogram information based on a second component of the first video stream,
converting the first histogram information into first cumulative distribution function values including applying a first clipping threshold to the first histogram information,
converting the second histogram information into second cumulative distribution function values including applying a second clipping threshold to the second histogram information,
wherein the second clipping threshold is different from the first clipping threshold,
generating a first enhanced video stream based on the first cumulative distribution function values and the second cumulative distribution function values, and
sending an output video stream indicative of the first enhanced video stream to a display.

14. The computer readable medium of claim 13 , the method further comprising:
obtaining a first plurality of frames from the first video stream, and
rearranging the first plurality of frames to generate a plurality of first delayed video streams; wherein generating the first enhanced video stream includes using the plurality of first delayed video streams.

15. The computer readable medium of claim 13 , the method further comprising the step of converting the fi

Assignees

• Endochoice Inc

Inventors

• Sidar Itay
• Davidson Tal
• Kronman Achia
• Mor Lior
• Levy Idan

Classifications

• A61B1/000095

  for image enhancement · CPC title

• A61B1/0005

  combining images e.g. side-by-side, superimposed or tiled · CPC title

• G06T5/40

  using histogram techniques · CPC title

• H04N9/646 Primary

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title

• G06T2200/28

  involving image processing hardware · CPC title