De-ghosting of images captured using a capsule camera

Patent metadata

Field	Value
Publication number	US-10015372-B2
Application number	US-201615335301-A
Country	US
Kind code	B2
Filing date	Oct 26, 2016
Priority date	Oct 26, 2016
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

A method of reducing ghost in images captured using a capsule endoscope while travelling in the gastrointestinal (GI) tract. The captured images contain ghost caused by reflections of multiple light sources by capsule housing of the capsule endoscope. The method derive, from the plurality of images, a ghost model comprising multiple ghost coefficients for relating light energies from the multiple light sources for a given image with ghost signals at multiple pixel locations for the given image. De-ghosted images are generated by compensating the plurality of images using estimated ghost signals based on derived ghost coefficients and the light energies from the multiple light sources. The process of deriving, from the plurality of images, the ghost model comprises removing any sensor gamma or any other non-linearity in pixel values of the plurality of images associated with the light energy.

First claim

The invention claimed is:

1. A method of reducing ghosts in images captured using a capsule endoscope, wherein captured images contain ghosts caused by reflections off capsule housing of the capsule endoscope by one or more light sources or one or more objects within the capsule endoscope illuminated by said one or more light sources, the method comprising:
receiving a plurality of images captured by a camera of the capsule endoscope, wherein the capsule endoscope uses one or more light sources to illuminate objects in a field of view of the camera;
deriving, from the plurality of images, a ghost model comprising ghost coefficients for relating light energies from said one or more light sources for a given image with ghost signals at multiple pixel locations for the given image; and
generating de-ghosted images by compensating the plurality of images using estimated ghost signals based on derived ghost coefficients and the light energies from said one or more light sources.

2. The method of claim 1 , wherein said deriving, from the plurality of images, the ghost model comprises removing any sensor gamma or any other non-linearity in pixel values of the plurality of images associated with the light energy.

3. The method of claim 1 , wherein each captured image is modelled as a sum of first pixel values corresponding to true pixel values and ghost signals at respective pixel locations within each picture, and wherein the true pixel value at a given pixel location is estimated by a sum of weighted first pixel values at neighboring locations of the given pixel location to derive an estimated true pixel value.

4. The method of claim 3 , wherein the true pixel value at the given pixel location is estimated by an average of the first pixel values at neighboring locations of the given pixel location.

5. The method of claim 3 , wherein the ghost coefficients at the given pixel location are estimated by minimizing errors between estimated pixel values and true pixel values calculated over multiple frames.

6. The method of claim 5 , wherein the errors between estimated pixel values and true pixel values correspond to mean squared errors.

7. The method of claim 5 , wherein the ghost coefficients are allowed to be non-negative only and a non-negative least squares process is used to derive the ghost coefficients.

8. The method of claim 5 , wherein the ghost coefficients are initially set to all zeros, and following iterative procedure is performed:
a) determining the ghost coefficients by minimizing errors between estimated pixel values and true pixel values calculated over multiple frames;
b) updating the estimated pixel values according to the ghost coefficients determined in step a); and
c) if a stopping criterion is satisfied, terminating the iteration procedure; and otherwise, going to step a).

9. The method of claim 3 , wherein the neighboring locations of the given pixel location include at least one pixel above, below, left and right of the given pixel location respectively.

10. The method of claim 3 , wherein the neighboring locations of the given pixel location include eight pixels around the given pixel location with horizontal distance plus vertical distance equal to two.

11. The method of claim 1 , wherein if a derived ghost coefficient has a negative value, the derived ghost coefficient is set to a non-negative value.

12. The method of claim 1 , wherein the light energies from said one or more light sources for the given image is retrieved from data stored during capturing the given image.

13. The method of claim 1 , wherein the images captured correspond to color images having multiple color components, and wherein said reducing ghosts is performed prior to de-mosaicking the multiple color components.

14. The method of claim 13 , wherein said reducing ghost is applied to each of the multiple color components separately.

15. An apparatus for reducing ghosts in images captured using a capsule endoscope, wherein captured images contain ghosts caused by reflections off capsule housing of the capsule endoscope by one or more light sources or one or more objects within the capsule endoscope illuminated by said one or more light sources, the apparatus comprising one or more electronic circuits or processors arranged to:
receive a plurality of images captured by a camera of the capsule endoscope, wherein the capsule endoscope uses one or more light sources to illuminate objects in a field of view of the camera;
derive, from the plurality of images, a ghost model comprising ghost coefficients for relating light energies from said one or more light sources for a given image with ghost signals at multiple pixel locations for the given image; and
generate de-ghosted images by compensating the plurality of images using estimated ghost signals based on derived ghost coefficients and the light energies from said one or more light sources.

16. The apparatus of claim 15 , wherein each captured image is modelled as a sum of first pixel values corresponding to true pixel values and ghost signals at respective pixel locations within each picture, and wherein the true pixel value at a given pixel location is estimated by a sum of weighted first pixel values at neighboring locations of the given pixel location to derive an estimated true pixel value.

17. The apparatus of claim 16 , wherein the true pixel value at the given pixel location is estimated by an average of the first pixel values at neighboring locations of the given pixel location.

18. The apparatus of claim 16 , wherein the ghost coefficients at the given pixel location are estimated by minimizing errors between estimated pixel values and true pixel values calculated over multiple frames.

Assignees

• Capsovision Inc
• Capsovision Inc

Inventors

• Hadley Mark
• Wilson Gordon C

Classifications

• H04N23/555

  for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes · CPC title

• A61B1/0661

  Endoscope light sources · CPC title

• H04N2005/2255

  Electricity · mapped topic

• A61B1/041

  Capsule endoscopes for imaging · CPC title

• A61B1/00009

  of image signals during a use of endoscope · CPC title