Method and apparatus of sharpening of gastrointestinal images based on depth information

Patent metadata

Field	Value
Publication number	US-10943333-B2
Application number	US-201916682208-A
Country	US
Kind code	B2
Filing date	Nov 13, 2019
Priority date	Oct 16, 2015
Publication date	Mar 9, 2021
Grant date	Mar 9, 2021

Abstract

A method for sharpening gastrointestinal (GI) images and an apparatus thereof are disclosed. A first target distance between a first target region in a regular image and an imaging apparatus is determined, and a second target distance between a second target region in the regular image and the imaging apparatus is determined. One or more first filter parameters for a first de-blurring filter and one or more second filter parameters for a second de-blurring filter are selected from stored filter parameters according to the first and second target distances respectively. A first processed target region is generated by applying the first de-blurring filter to the first target region to improve sharpness of the first target region. A second processed target region is generated by applying the second de-blurring filter to the second target region to improve sharpness of the second target region.

First claim

The invention claimed is:

1. A method for processing gastrointestinal (GI) images, comprising:
receiving a regular image, wherein the regular image is captured using an imaging apparatus by projecting non-structured light onto a body lumen when the imaging apparatus is in the body lumen;
determining a first target distance for a first target region in the regular image, wherein the first target distance is between the first target region and the imaging apparatus;
selecting one or more first filter parameters for a first de-blurring filter from stored filter parameters according to the first target distance;
generating a first processed target region by applying the first de-blurring filter to the first target region to improve sharpness of the first target region;
determining a second target distance for a second target region in the regular image, wherein the second target distance is between the second target region and the imaging apparatus;
selecting one or more second filter parameters for a second de-blurring filter from the stored filter parameters according to the second target distance;
generating a second processed target region by applying the second de-blurring filter to the second target region to improve sharpness of the second target region; and
providing a processed regular image comprising the first processed target region and the second processed target region.

2. The method of claim 1 , wherein the first de-blurring filter and the second de-blurring filter corresponds to a deconvolution kernel.

3. The method of claim 1 , wherein the imaging apparatus corresponds to a capsule endoscope.

4. The method of claim 1 , wherein the imaging apparatus corresponds to a traditional endoscope with a flexible tube.

5. The method of claim 1 , wherein the imaging apparatus corresponds to a capsule endoscope with panoramic cameras and the regular image corresponds to a panoramic image comprising multiple individual images captured by the panoramic cameras covering different fields of view.

6. The method of claim 1 , wherein the first de-blurring filter and the second de-blurring filter corresponds to a Wiener filter by taking into account of a point spread function (PSF) associated with camera system and noise in the camera system.

7. The method of claim 1 , further comprising receiving one or more structured-light images for a corresponding regular image, wherein said one or more structured-light images are captured using the imaging apparatus by projecting structured light onto the body lumen when the imaging apparatus is in the body lumen.

8. The method of claim 7 , the target distance is derived based on said one or more structured-light images for the corresponding regular image.

9. The method of claim 1 , wherein the stored filter parameters are previously derived by capturing test pictures of one or more test patterns placed at various test distances in a range including a focus distance.

10. The method of claim 9 , wherein the test pictures of one or more test patterns correspond to a single test picture having a white dot at a center in black or gray background.

11. The method of claim 9 , wherein the test pictures of one or more test patterns correspond to multiple test pictures, each having a white dot at selected location in a black or gray background.

12. The method of claim 11 , wherein said one or more first filter parameters and said one or more second filter parameters are selected from the stored filter parameters based on the target distance and pixel location.

13. The method of claim 1 , wherein the stored filter parameters are previously derived for an individual imaging apparatus by capturing test pictures of one or more test patterns placed at various test distances from the individual imaging apparatus.

14. An apparatus for processing gastrointestinal (GI) images, comprising a processor configured to:
receive a regular image, wherein the regular image is captured using an imaging apparatus by projecting non-structured light onto a body lumen when the imaging apparatus is in the body lumen;
determine a first target distance for a first target region in the regular image, wherein the first target distance is between the first target region and the imaging apparatus;
select one or more first filter parameters for a first de-blurring filter from stored filter parameters according to the first target distance;
generate a first processed target region by applying the first de-blurring filter to the first target region to improve sharpness of the first target region;
determine a second target distance for a second target region in the regular image, wherein the second target distance is between the second target region and the imaging apparatus;
select one or more second filter parameters for a second de-blurring filter from the stored filter parameters according to the second target distance;
generate a second processed target region by applying the second de-blurring filter to the second target region to improve sharpness of the second target region; and
provide a processed regular image comprising the first processed target region and the second processed target region.

15. The apparatus of claim 14 , wherein the stored filter parameters are previously derived for an individual imaging apparatus by capturing test pictures of one or more test patterns placed at various test distances from the individual imaging apparatus.

16. A method for characterizing an imaging apparatus, comprising:
placing the imaging apparatus under a controlled environment;
capturing test pictures for a set of pre-defined patterns, by placing the set of pre-defined patterns in a scene of the imaging apparatus, at multiple test distances in a range including a focus distance using the imaging apparatus, wherein each test picture is captured by
projecting light onto one selected pre-defined pattern placed at a selected distance;
determining one or more parameters associated a target point spread function from each test picture, wherein the target point spread function characterizes image formation of the imaging apparatus at the selected distance; and
storing a set of said one or more parameters associated a set of point spread functions characterizing the image formation of the imaging apparatus at said multiple test distances for processing images captured subsequently.

17. The method of claim 16 , wherein the set of pre-defined patterns corresponds to a white dot at a center in a black or gray background.

18. The method of claim 16 , wherein the set of pre-defined patterns corresponds to multiple white dots in a black or gray background.

19. The method of claim 16 , wherein the set of pre-defined patterns corresponds to one white dot at a set of selected locations in a black or gray background.

20. The method of claim 16 , wherein the target point spread function corresponds to a Gaussian function.

21. The method of claim 20 , wherein the set of pre-defined patterns corresponds to a white dot at a set of selected locations in a black or gray background, and the set of said one or more parameters corresponds to a set of sigma values.

22. The method of claim 16 , wherein said placing the imaging apparatus under the controlled environment, said determining said one or more parameters and said storing the set of said one or more parameters are performed for each individual imaging apparatus.

23. An apparatus for characterizing an individual imaging

Assignees

• Capso Vision Inc
• Capsovision Inc

Inventors

• Wang Kang-Huai
• Wilson Gordon C
• Wu Chenyu

Classifications

• H04N23/555

  for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• A61B1/0605

  for spatially modulated illumination · CPC title