Method and apparatus for area or volume of object of interest from gastrointestinal images

Patent metadata

Field	Value
Publication number	US-10346978-B2
Application number	US-201715669853-A
Country	US
Kind code	B2
Filing date	Aug 4, 2017
Priority date	Aug 4, 2017
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

A method and apparatus for estimating or measuring a physical area or physical volume of an object of interest in one or more images captured using an endoscope are disclosed. According to the present method, an object of interest in an image or images is determined. Also, distance information associated with the object of interest with respect to an image sensor of the endoscope is received. The physical area size or physical volume size of the object of interest is then determined based on the image or images, and the distance information.

First claim

The invention claimed is:

1. A method of estimating or measuring a physical area or physical volume of an object of interest in one or more images captured using a capsule endoscope, the method comprising:
receiving one or more images captured using a capsule endoscope when the capsule endoscope is inside a human gastrointestinal (GI) tract;
determining an object of interest in said one or more images, wherein the object of interest corresponds to a target object on a wall of lumen of the human GI tract;
deriving distance information associated with the object of interest with respect to the capsule endoscope by projecting structured light onto the wall of lumen of the human GI tract; and
determining physical area size or physical volume size of the object of interest based on said one or more images and the distance information, and wherein said determining physical area size or physical volume size of the object of interest comprises:
determining a first direction aligned with a longest straight line from one end of the object of interest to another end of the object of interest;
estimating a first physical dimension of the object of interest based on the longest straight line measured from said one or more images scaled by a magnification factor associated with optical configuration of an image sensor of the capsule endoscope; and
estimating a second physical dimension of the object of interest based on a second measured longest length of the object of interest in a second direction scaled by the magnification factor, wherein the second direction is perpendicular to the first direction; and
wherein the physical area size of the object of interest is proportional to a product of the first physical dimension of the object of interest and the second physical dimension of the object of interest.

2. The method of claim 1 , wherein said determining an object of interest comprising outlining a boundary of the object of interest by a user or an artificial intelligence process or jointly by the user and the artificial intelligence process.

3. The method of claim 2 , wherein said outlining a boundary of the object of interest is performed by the user using an input device to indicate the boundary of the object of interest in said one or more images displayed on a display device.

4. The method of claim 2 , wherein if the object of interest crosses a boundary of a current image into one or more neighboring images, the current image and said one or more neighboring images are stitched prior to said determining physical area size or physical volume size of the object of interest.

5. The method of claim 4 , wherein the object of interest is outlined in the current image and said one or more neighboring images to assist stitching the current image and said one or more neighboring images.

6. The method of claim 1 , wherein said determining an object of interest comprises outlining a boundary of the object of interest automatically using image processing.

7. The method of claim 1 , wherein said determining physical area size or physical volume size of the object of interest is performed automatically using computer executable codes executed on a computing device.

8. The method of claim 1 , wherein the object of interest corresponds to a lesion, pedunculated polyp, sessile serrated polyp, flat lesion or an infected area by Crohn's disease.

9. A method of estimating or measuring a physical area or physical volume of an object of interest in one or more images captured using a capsule endoscope, the method comprising:
receiving one or more images captured using a capsule endoscope when the capsule endoscope is inside a human gastrointestinal (GI) tract;
determining an object of interest in said one or more images, wherein the object of interest corresponds to a target object on a wall of lumen of the human GI tract;
deriving distance information associated with the object of interest with respect to the capsule endoscope by projecting structured light onto the wall of lumen of the human GI tract; and
determining physical area size or physical volume size of the object of interest based on said one or more images and the distance information; and
wherein a score board is generated for an anomaly as an indication of condition of the anomaly by using an aggregate number corresponding to an average area or percentage of one or more target objects of interest in one or more images representing a section of GI tract, and wherein said one or more target objects of interest correspond to one or more infected areas in the human GI tract associated with the anomaly.

10. The method of claim 9 , wherein the section of GI tract corresponds to a part of small bowel, a part of colon or both.

11. A method of estimating or measuring a physical area or physical volume of an object of interest in one or more images captured using a capsule endoscope, the method comprising:
receiving one or more images captured using a capsule endoscope when the capsule endoscope is inside a human gastrointestinal (GI) tract;
determining an object of interest in said one or more images, wherein the object of interest corresponds to a target object on a wall of lumen of the human GI tract;
deriving distance information associated with the object of interest with respect to the capsule endoscope by projecting structured light onto the wall of lumen of the human GI tract; and
determining physical area size or physical volume size of the object of interest based on said one or more images and the distance information; and
wherein an index number is generated for an anomaly as an indication of a disease state for the anomaly, and wherein the index number is proportional to a sum of infected areas in said one or more images in a GI section divided by a sum of total image areas in said one or more images in the GI section.

12. The method of claim 1 , wherein the object of interest corresponds to an anomaly and the physical volume size of the object of interest is derived based on a 3D mathematical model for the anomaly and measured 1D or 2D size information for the object of interest, and wherein the 3D mathematical model is used to predict depth or volume of the anomaly underneath a mucosal surface.

13. The method of claim 12 , wherein the 3D mathematical model corresponds to an ellipsoid, ovoid, sphere or disc.

14. A method of estimating or measuring a physical area or physical volume of an object of interest in one or more images captured using a capsule endoscope, the method comprising:
receiving one or more images captured using a capsule endoscope when the capsule endoscope is inside a human gastrointestinal (GI) tract;
determining an object of interest in said one or more images, wherein the object of interest corresponds to a target object on a wall of lumen of the human GI tract;
deriving distance information associated with the object of interest with respect to the capsule endoscope by projecting structured light onto the wall of lumen of the human GI tract; and
determining physical area size or physical volume size of the object of interest based on said one or more images and the distance information; and
wherein the physical area size or the physical volume size of the object of interest is derived based on the distance information derived at a set of point locations by projecting structured lights onto a wall of a lumen of the human GI tract;
wherein a triangle mesh is generated to cover a topographic surface of the object of interest, and wherein the triangle mesh consists of a set of triangles generated using the set of point locations; and
wherein an outline id

Assignees

• Capsovision Inc

Inventors

• Wang Kang-Huai
• Hadley Mark
• Wu Chenyu

Classifications

• G06T2207/30092

  Stomach; Gastric · CPC title

• G06T2207/10068

  Endoscopic image · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06T2207/20104

  Interactive definition of region of interest [ROI] · CPC title