Image processing device, operation method, and computer readable recording medium

Patent metadata

Field	Value
Publication number	US-12414675-B2
Application number	US-201916420492-A
Country	US
Kind code	B2
Filing date	May 23, 2019
Priority date	Jan 26, 2017
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

An image processing device includes a processor including hardware, the processor being configured to execute: estimating an operation state of a medical apparatus in a lumen of a subject based on information sequentially input from the medical apparatus, the medical apparatus including an imaging device, and an insertion portion inserted into the lumen, and the information including at least an intraluminal image obtained by the imaging device capturing an image inside the lumen; and performing processing corresponding to the operation state of the medical apparatus in the lumen based on an estimation result.

First claim

What is claimed is:

1. An image processing device comprising:
a processor configured to:
acquire a predetermined reference image of a turning back point where an endoscope changes a direction of travel from inserting to removing;
acquire a sequence of intraluminal images captured by an imaging device;
determine a degree of matching between each one of the sequence of intraluminal images and the predetermined reference image when each one of the sequence of intraluminal images is acquired, wherein determine the degree of matching comprises comparing each one of the sequence of intraluminal images with the predetermined reference image at pixel level when each one of the sequence of intraluminal images is acquired;
determine that the endoscope reached the turning back point when the degree of matching reached a predetermined reference value; and
output a signal for switching an operation state of the endoscope from an insertion state and a removal state in response to determining that the endoscope reached the turning back point.

2. The image processing device according to claim 1 ,
wherein the processor is configured to:
determine an operation state of the imaging device is an inserting state that indicates the imaging device is inserted into a lumen based on the sequence of intraluminal images; and
in response to determining that the imaging device is inserted into the lumen, measure an insertion time period in which the imaging device is inserted into the lumen.

3. The image processing device according to claim 2 ,
wherein the processor is configured to set, as the insertion time period, a period in which the endoscope is advancing from a time point when the endoscope is determined to be inserted into the lumen.

4. The image processing device according to claim 1 ,
wherein the processor is configured to:
in response to determining that the degree of matching reached the predetermined reference value, control the imaging device to obtain an image.

5. The image processing device according to claim 1 , wherein:
the processor is configured to acquire information necessary for a retracting operation; and
the retracting operation includes at least one of discovery, identification, and treatment of a specific region.

6. The image processing device according to claim 1 ,
wherein the processor is configured to detect a target lesion based on the intraluminal images and count number of lesions.

7. The image processing device according to claim 1 ,
wherein the processor is configured to, in response to determining that the endoscope reached the turning back point, perform at least one of: estimating a lumen position and identifying an organ imaged by the imaging device in the intraluminal images sequentially obtained.

8. The image processing device according to claim 7 ,
wherein the processor is configured to estimate the lumen position based on the intraluminal images.

9. The image processing device according to claim 7 ,
wherein the processor is configured to identify a region of an organ imaged by the imaging device in the intraluminal images sequentially obtained.

10. The image processing device according to claim 9 , wherein:
when the organ is a large intestine, the processor is configured to identify the region of the large intestine as one of a rectum, a sigmoid colon, a descending colon, a hepatic flexure, a transverse colon, a transverse colon, a splenic flexure, an ascending colon, and an ileocecum;
when the organ is an esophagus, the processor is configured to identify the region of the esophagus as one of an esophageal opening and a cardiac orifice;
when the organ is a stomach, the processor is configured to identify the region of the stomach as one of a gastric fundus, a gastric body, a pyloric zone, and a pylorus;
when the organ is a duodenum, the processor is configured to identify the region of the duodenum as one of a duodenal bulb and a papilla vater; and
when the organ is a small intestine, the processor is configured to identify the region of the small intestine as one of a jejunum and an ileum.

11. The image processing device according to claim 1 , wherein the processor is configured to:
control a display to display the operation state of the endoscope.

12. The image processing device according to claim 1 , wherein:
the predetermined reference image is an image of an observation target in a lumen.

13. The image processing device according to claim 1 , wherein: the insertion state and the removal state are operation states of the endoscope in a lumen.

14. The image processing device according to claim 13 , wherein: the insertion state is the operation state until a distal end of the endoscope reaches a target position after the distal end enters into the lumen; the removal state is the operation state until the distal end of the endoscope comes out of the lumen after the distal end reaches the target position inside the lumen.

15. The image processing device according to claim 1 , wherein under the insertion state, the endoscope advances towards an observation target that is an ileocecum.

16. A system comprising: the image processing device according to claim 1 ; and the endoscope comprising an insertion portion and the imaging device provided in the insertion portion.

17. A method for determining an operation state of an imaging device, the method comprising:
acquiring, by the processor, a predetermined reference image of a turning back point where an endoscope changes a direction of travel from inserting to removing;
acquiring a sequence of intraluminal images captured by the imaging device;
determining a degree of matching between each one of the sequence of intraluminal images and the predetermined reference image when each one of the sequence of intraluminal images is acquired, wherein determining the degree of matching comprises comparing each one of the sequence of intraluminal images with the predetermined reference image at pixel level when each one of the sequence of intraluminal images is acquired;
determining, by the processor, that the endoscope reached the turning back point when the degree of matching reached a predetermined reference value; and
outputting a signal for switching an operation state of the endoscope from an insertion state and a removal state in response to determining that the endoscope reached the turning back point.

18. A non-transitory computer readable recording medium on which an executable program is recorded, the executable program instructing a processor of an image processing device to determine an operation state of an imaging device by performing the steps of:
acquire a predetermined reference image of a turning back point where an endoscope changes a direction of travel from inserting to removing;
acquire a sequence of intraluminal images captured by the imaging device;
determine a degree of matching between each one of the sequence of intraluminal images and the predetermined reference image when each one of the sequence of intraluminal images is acquired, wherein determine the degree of matching comprises comparing each one of the sequence of intraluminal images with the predetermined reference image at pixel level when each one of the sequence of intraluminal images is acquired;
determine that the endoscope reached the turning back point when the degree of matching reached a predetermined reference value; and
output a signal for switching an operation state of the endoscope from an insertion state and a removal state in response to determining that

Assignees

• Olympus Corp

Inventors

• Kimura Mitsutaka
• Hayami Takehito
• Kono Takashi
• Kanda Yamato
• Takami Ryoji

Classifications

• A61B1/00154

  using guiding arrangements for insertion · CPC title

• A61B1/00002

  Operational features of endoscopes · CPC title

• A61B1/00055

  for alerting the user · CPC title

• G06T2207/30101

  Blood vessel; Artery; Vein; Vascular · CPC title

• A61B1/041

  Capsule endoscopes for imaging · CPC title