Endoscopic image enhancement using contrast limited adaptive histogram equalization (CLAHE) implemented in a processor

Patent metadata

Field	Value
Publication number	US-10516865-B2
Application number	US-201615155814-A
Country	US
Kind code	B2
Filing date	May 16, 2016
Priority date	May 17, 2015
Publication date	Dec 24, 2019
Grant date	Dec 24, 2019

Abstract

Systems and methods of enhancing images use a contrast limited adaptive histogram equalization (CLAHE) algorithm in a field programmable gate array (FPGA). The images may be obtained by the imaging elements of a multiple imaging elements endoscope of an endoscopy system.

First claim

We claim:

1. An endoscope system comprising:
an endoscope including a first viewing element and a second viewing element;
a control unit operatively coupled to the endoscope by a cable, wherein the control unit comprises:
a base board module;
one or more processing boards that implement a contrast limited adaptive histogram equalization (CLAHE) algorithm to enhance images obtained by the first viewing element and the second viewing element, wherein the one or more processing boards perform the steps of:
receiving a first YCbCr color space video stream from the first viewing element and a second YCbCr color space video stream from the second viewing element,
converting the first YCbCr color space video stream to a first Lab color space video stream and the second YCbCr color space video stream to a second Lab color space video stream,
building and storing first histogram information based on the first Lab color space video stream,
building and storing second histogram information based on the second Lab color space video stream,
converting the first histogram information into first cumulative distribution function values including applying a first clipping threshold to the first histogram information,
converting the second histogram information into second cumulative distribution function values including applying a second clipping threshold to the second histogram information, wherein the second clipping threshold is different from the first clipping threshold,
obtaining a first plurality of Lab color space frames from the first Lab color space video stream and a second plurality of Lab color space frames from the second Lab color space video stream,
rearranging the first plurality of Lab color space frames to generate a plurality of first delayed Lab streams,
rearranging the second plurality of Lab color space frames to generate a plurality of second delayed Lab streams,
generating a first enhanced Lab stream based on the plurality of first delayed Lab streams and the first cumulative distribution function values,
generating a second enhanced Lab stream based on the plurality of second delayed Lab streams and the second cumulative distribution function values, and
sending output video streams indicative of the first enhanced Lab stream and the second enhanced Lab stream to a display.

2. The endoscope system of claim 1 , wherein the first YCbCr color space video stream and the second YCbCr color space video stream are received by an adapter of the one or more processing boards, the adapter having conversion modules, and wherein the first YCbCr color space video stream and the second YCbCr color space video stream are converted to the first Lab color space video stream and the second lab color space video stream by the conversion modules.

3. The endoscope system of claim 1 , wherein the first Lab color space frames and the second Lab color space frames are obtained from the first Lab color space video and the second Lab color space video stream using a frame grabber of the one or more processing boards.

4. The endoscope system of claim 3 , wherein said base board module further comprises DDR3 memory in communication with said frame grabber.

5. The endoscope system of claim 1 , wherein the first plurality of Lab color space frames and the second plurality of Lab color space frames are rearranged to generate the first plurality of delayed Lab streams and the second plurality of delayed Lab streams using a fetching module of the plurality of processing boards.

6. The endoscope system of claim 1 , wherein the histogram information is converted into cumulative distribution function values, and the first enhanced Lab stream and the second enhanced Lab stream are, generated using an interpolation algorithm module of the plurality of processing boards.

7. A method of enhancing images obtained by a plurality of viewing elements in an endoscope system using a contrast limited adaptive histogram equalization (CLAHE) process, wherein said endoscope system comprises (a) a control unit having a baseboard module including one or more processing boards configured to implement said process, and (b) an endoscope operatively coupled to the control unit by a cable, the endoscope including the plurality of viewing elements, said method comprising the steps of:
receiving a plurality of YCbCr color space video streams from each of the plurality of viewing elements;
converting the plurality of YCbCr color space video streams to Lab color space video streams;
building and storing histogram information based on the plurality of Lab color space video streams including splitting the plurality of Lab color space video streams into a plurality of tiles, wherein units of the tile heights are in video lines and units of the tile widths are in pixels, wherein each of the plurality of tiles overlaps with its neighboring tiles;
converting the histogram information into cumulative distribution function values;
obtaining said Lab color space video streams as Lab color space frames;
rearranging Lab color space frame data to generate delayed Lab streams;
generating a plurality of enhanced Lab streams, using the plurality of delayed Lab streams and the cumulative distribution function values; and
displaying output video streams indicative of the plurality of enhanced Lab streams.

8. The method of claim 7 , wherein converting said YCbCr color space streams to Lab color space video streams comprises the steps of:
performing YCbCr to RGB conversion using MAD instantiation;
performing a function similar to Y(x)=x{circumflex over ( )}2.40 using a LUT;
performing RGB to XYZ conversion using MAD instantiation;
executing a function similar to Y(x)=x{circumflex over ( )}0.33 using a LUT; and
performing XYZ to Lab conversion using MAD instantiation.

9. The method of claim 7 , further comprising converting said enhanced Lab streams to enhanced YCbCr streams.

10. The method of claim 9 , wherein converting said enhanced Lab streams to enhanced YCbCr streams comprises the steps of:
performing Lab to XYZ conversion using MAD instantiation;
executing a function similar to Y(x)=x{circumflex over ( )}0.33 using a LUT;
performing XYZ to RGB conversion using MAD instantiation;
performing a function similar to Y(x)=x{circumflex over ( )}(1/2.40) using a LUT; and
performing RGB to YCbCr conversion using MAD instantiation.

11. A non-transient computer readable medium containing program instructions for causing a computer to perform a method of enhancing images obtained by three viewing elements in an endoscope system using a contrast limited adaptive histogram equalization (CLAHE) process, wherein said endoscope system comprises (a) a control unit with a baseboard module including a processor configured to implement said process, and (b) an endoscope operatively coupled to the control unit by a cable, the endoscope including the three viewing elements, said method comprising the steps of:
receiving three YCbCr color space video streams from the three viewing elements with an adapter on said processor, each of the three viewing elements providing one of said three YCbCr color space video streams;
converting each of the three YCbCr color space video streams to corresponding three Lab color space video streams within said adapter;
building and storing histogram information based on the three Lab color space video streams within a histogram module on said processor;
converting the histogram information into cumulative distribution function values within the histogram module;
transmitting the three Lab color space video streams to a frame grabber on said processor in a form of a video stream obtained from a single vi

Assignees

• Endochoice Inc

Inventors

• Sidar Itay
• Davidson Tal
• Kronman Achia
• Mor Lior
• Levy Idan

Classifications

• G06T5/40

  using histogram techniques · CPC title

• G06T2200/28

  involving image processing hardware · CPC title

• G06T5/20

  using local operators · CPC title

• A61B1/0005

  combining images e.g. side-by-side, superimposed or tiled · CPC title

• H04N9/646 Primary

  for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters · CPC title