Method and apparatus of sharpening of gastrointestinal images based on depth information

Patent metadata

Field	Value
Publication number	US-11354783-B2
Application number	US-201916703765-A
Country	US
Kind code	B2
Filing date	Dec 4, 2019
Priority date	Oct 16, 2015
Publication date	Jun 7, 2022
Grant date	Jun 7, 2022

Abstract

A method and apparatus for sharpening gastrointestinal (GI) images are disclosed. A target distance between the target region and the imaging apparatus is determined for a target region in the regular image. One or more filter parameters of a de-blurring filter are selected from stored filter parameters according to the target distance. A processed target region is generated by applying the de-blurring filter to the target region to improve sharpness of the target region. A method for characterizing an imaging apparatus is also disclosed. The imaging apparatus is placed under a controlled environment. Test pictures for one or more test patterns are captured at multiple test distances in a range including a focus distance using the imaging apparatus. One or more parameters associated a target point spread function are determined from each test picture for characterizing image formation of the imaging apparatus at the selected distance.

First claim

The invention claimed is:

1. A method for processing gastrointestinal (GI) images, comprising:
receiving a regular image, wherein the regular image is captured using an imaging apparatus by projecting non-structured light onto a body lumen when the imaging apparatus is in the body lumen;
determining a target distance for a target region in the regular image, wherein the target distance is between the target region and the imaging apparatus;
determining one or more filter parameters of a de-blurring filter from stored test pictures or from stored filter parameters according to the target distance;
generating a first processed target region by applying the de-blurring filter to the target region to improve sharpness of the target region; and
providing a first processed regular image comprising the first processed target region.

2. The method of claim 1 , wherein the de-blurring filter corresponds to a deconvolution kernel.

3. The method of claim 1 , wherein the imaging apparatus corresponds to a capsule endoscope.

4. The method of claim 1 , wherein the imaging apparatus corresponds to a traditional endoscope with a flexible tube.

5. The method of claim 1 , wherein the imaging apparatus corresponds to a capsule endoscope with panoramic cameras and the regular image corresponds to a panoramic image comprising multiple individual images captured by the panoramic cameras covering different fields of view.

6. The method of claim 1 , wherein the de-blurring filter corresponds to a Wiener filter by taking into account of a point spread function (PSF) associated with camera system and noise in the camera system.

7. The method of claim 1 , further comprising:
applying a second de-blurring filter to the target region of the regular image to generate a second processed regular image, wherein the second de-blurring filter corresponds to a second target distance with a small deviation from the target distance.

8. The method of claim 7 , further comprising displaying the second processed regular image on a display device for a user to assess visual quality of the second processed regular image.

9. The method of claim 8 , further comprising:
applying a third de-blurring filter to the target region of the regular image to generate a third processed regular image, wherein the third de-blurring filter corresponds to a third target distance modified from the second target distance in a direction depending on a user indication of visual quality comparison between the first processed regular image and the second processed regular image.

10. The method of claim 1 , further comprising receiving one or more structured-light images for a corresponding regular image, wherein said one or more structured-light images are captured using the imaging apparatus by projecting structured light onto the body lumen when the imaging apparatus is in the body lumen.

11. The method of claim 10 , the target distance is derived based on said one or more structured-light images for the corresponding regular image.

12. The method of claim 1 , wherein the stored filter parameters are previously derived by capturing test pictures of one or more test patterns placed at various test distances in a range.

13. The method of claim 12 , wherein the test pictures of one or more test patterns correspond to a single test picture having a white dot at a center in black or gray background.

14. The method of claim 12 , wherein the test pictures of one or more test patterns correspond to multiple test pictures, each having a white dot at selected location in a black or gray background.

15. The method of claim 14 , wherein said one or more filter parameters of the de-blurring filter are selected from the stored filter parameters based on the target distance and pixel location.

16. The method of claim 12 , wherein the range includes a focus distance.

17. The method of claim 1 , wherein the stored filter parameters are previously derived for an individual imaging apparatus by capturing test pictures of one or more test patterns placed at various test distances from the individual imaging apparatus.

18. The method of claim 1 , wherein the stored test pictures are captured using the imaging apparatus by placing one or more test patterns at various test distances from the imaging apparatus in a range.

19. The method of claim 18 , wherein said one or more test patterns correspond to a single test picture having a white dot at a center in black or gray background.

20. The method of claim 18 , wherein said one or more test patterns correspond to multiple test pictures, each having a white dot at selected location in a black or gray background.

21. The method of claim 18 , wherein the range includes a focus distance.

22. An apparatus for processing gastrointestinal (GI) images, comprising a processor configured to:
receive a regular image, wherein the regular image is captured using an imaging apparatus by projecting non-structured light onto a body lumen when the imaging apparatus is in the body lumen;
determine a target distance for a target region in the regular image, wherein the target distance is between the target region and the imaging apparatus;
determine one or more filter parameters of a de-blurring filter from stored test pictures or from stored filter parameters according to the target distance;
generate a first processed target region by applying the de-blurring filter to the target region to improve sharpness of the target region; and
provide a first processed regular image comprising the first processed target region.

23. The apparatus of claim 22 , wherein the stored filter parameters are previously derived for an individual imaging apparatus by capturing test pictures of one or more test patterns placed at various test distances from the individual imaging apparatus.

24. A method for characterizing an individual imaging apparatus, comprising:
placing the individual imaging apparatus under a controlled environment;
capturing test pictures for one or more test patterns at multiple test distances in a range including a focus distance using the individual imaging apparatus, wherein each test picture is capture by projecting light onto a selected test pattern placed at a selected test distance; and
storing the test pictures or one or more parameters of a de-blurring filter associated with the individual imaging apparatus at said multiple test distances for processing images captured subsequently, wherein said one or more parameters of the de-blurring filter are derived from the test pictures at each test distance and the de-blurring filter corresponds to a point-spread function characterizing image formation of the individual imaging apparatus at the selected test distance.

25. The method of claim 24 , wherein said one or more test patterns correspond to a single test picture having a white dot at a center in black or gray background.

26. The method of claim 24 , wherein said one or more test patterns correspond to a single test picture having multiple white dots in black or gray background at pre-determined locations.

27. The method of claim 24 , wherein said one or more test patterns correspond to multiple test pictures, each having a white dot at selected location in a black or gray background.

28. The method of claim 24 , wherein the point-spread function corresponds to a Gaussian function.

Assignees

• Capso Vision Inc
• Capsovision Inc

Inventors

• Wang Kang-Huai
• Wilson Gordon C
• Wu Chenyu

Classifications

• A61B1/041 Primary

  Capsule endoscopes for imaging · CPC title

• A61B1/0605

  for spatially modulated illumination · CPC title

• A61B1/000095

  for image enhancement · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T2207/10068

  Endoscopic image · CPC title