Processing device, endoscope system, and method for processing captured image

Patent metadata

Field	Value
Publication number	US-12433478-B2
Application number	US-202217896271-A
Country	US
Kind code	B2
Filing date	Aug 26, 2022
Priority date	Feb 27, 2020
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

A processing device includes a processor, the processor being configured to: acquire a captured image of an inside of a lumen; acquire lumen structure information indicating a structure of the lumen; determine whether the captured image can be analyzed or not and output analysis allowability/non-allowability information indicating whether the captured image is in an analysis allowable state or not based on the captured image and first determination criteria; associate the analysis allowability/non-allowability information with the structure of the lumen based on the analysis allowability/non-allowability information and the lumen structure information to identify an analyzable portion and an unanalyzable portion of the structure of the lumen; and determine that the identified unanalyzable portion is a missed portion based on position and orientation information of a distal end of an insertion section to be inserted in the lumen and second determination criteria.

First claim

What is claimed is:

1. A processing device comprising:
a processor comprising hardware, the processor being configured to:
acquire a captured image of an inside of a lumen;
acquire lumen structure information indicating a structure of the lumen;
determine whether the captured image can be analyzed or not and output analysis allowability/non-allowability information indicating whether the captured image is in an analysis allowable state or not based on the captured image and first determination criteria;
associate the analysis allowability/non-allowability information with the structure of the lumen based on the analysis allowability/non-allowability information and the lumen structure information to identify an analyzable portion and an unanalyzable portion of the structure of the lumen; and
determine that a portion of the identified unanalyzable portion determined to meet second determination criteria is a missed portion based on position and orientation information of a distal end of an insertion section to be inserted in the lumen,

wherein:
the lumen is an intestinal tract; and
in a case where the insertion section is inserted into a depth of the intestinal tract and then pulled out while capturing an image of the intestinal tract, the second determination criteria is criteria that a portion is unobservable unless reinserting the insertion section into the depth.

2. The processing device as defined in claim 1 , wherein the processor is configured to output the analysis allowability/non-allowability information based on whether the captured image can be analyzed by a computer or not.

3. The processing device as defined in claim 2 , wherein:
the captured image is a biological image, and
the processor is configured to output the analysis allowability/non-allowability information based on whether a classification process or a detection process can be executed by the computer or not, the classification process being for classifying the biological image into any of classes and the detection process being for detecting a region of interest from the biological image.

4. The processing device as defined in claim 1 , wherein the processor is configured to determine the captured image to be analyzable when a degree of motion of a subject in the captured image is no greater than a predetermined value and the motion is maintained for a prescribed time.

5. The processing device as defined in claim 1 , wherein the processor is configured to determine the captured image to be analyzable when quality of the captured image is greater than or equal to a predetermined reference value.

6. The processing device as defined in claim 1 , wherein the processor is configured to determine the captured image to be analyzable when the captured image is divided into a plurality of regions such that each size of a plurality of the regions is greater than or equal to a predetermined value.

7. The processing device as defined in claim 1 , wherein the processor is configured to acquire the lumen structure information based on the captured image.

8. The processing device as defined in claim 1 , wherein the processor is configured to:
acquire the position and orientation information of the distal end of the insertion section relative to the lumen from a sensor provided in the insertion section; and
acquire the lumen structure information based on the position and orientation information and the captured image.

9. The processing device as defined in claim 1 , wherein the processor is configured to:
set a plurality of feature points in each of a plurality of the captured images captured at two or more timings; and
determine correspondence of a plurality of the feature points in the captured images captured at the two or more timings to each other, whereby associating the analysis allowability/non-allowability information with the structure of the lumen.

10. The processing device as defined in claim 9 , wherein the processor is configured to perform at least one of:
a process of defining an analyzable region with the three or more feature points to identify arrangement of the analyzable region in the lumen, the analyzable region being a region in the captured image determined to be analyzable; and
a process of defining an unanalyzable region with the three or more feature points to identify arrangement of the unanalyzable region in the lumen, the unanalyzable region being a region in the captured image determined to be unanalyzable.

11. The processing device as defined in claim 10 , wherein the region defined by the three or more feature points is deformable between a shape in the captured image captured at a first timing and a shape in the captured image captured at a second timing.

12. The processing device as defined in claim 1 , wherein the processor is configured to:
determine that a portion of the structure of the lumen determined to be analyzable based on at least one captured image is the analyzable portion; and
determine that a portion of the structure of the lumen other than the analyzable portion is the unanalyzable portion.

13. The processing device as defined in claim 1 , wherein the processor is configured to estimate at least one of an area of the analyzable portion, a shape of the analyzable portion, an area of the unanalyzable portion, and a shape of the unanalyzable portion.

14. The processing device as defined in claim 13 , wherein:
the processor is configured to perform a process of acquiring at least one of an analyzability rate and a miss rate;
the analyzability rate is information indicating area ratio of the analyzable portion relative to an entire inner surface of the lumen or information indicating area ratio of the analyzable portion relative to a portion of the lumen determined to have been observed; and
the miss rate is information indicating area ratio of the unanalyzable portion relative to the entire inner surface of the lumen or information indicating area ratio of the unanalyzable portion relative to a portion of the lumen determined to have been observed.

15. The processing device as defined in claim 14 , wherein the processor is configured to acquire at least one of the analyzability rate and the miss rate by simplifying the shape of the unanalyzable portion.

16. The processing device as defined in claim 14 , wherein:
the lumen is an intestinal tract; and
the processor is configured to acquire at least one of the analyzability rate and the miss rate based on a reference shape of the intestinal tract.

17. The processing device as defined in claim 1 , wherein when the missed portion is present, the processor is configured to perform a guide process to capture the missed portion in the analysis allowable state.

18. An endoscope system comprising:
a processor comprising hardware; and
an imaging device configured to capture an image of an inside of a lumen,
wherein the processor is configured to:
acquire a captured image based on the imaging device;
acquire lumen structure information indicating a structure of the lumen;
determine whether the captured image can be analyzed or not and output analysis allowability/non-allowability information indicating whether the captured image is in an analysis allowable state or not based on the captured image and first determination criteria;
associate the analysis allowability/non-allowability information with the structure of the lumen based on the analysis allowability/non-allowability information and the lumen structure information to identify an analyzable portion and an unanalyzab

Assignees

• Olympus Corp

Inventors

• Hane Jun

Classifications

• G06T7/0012

  Biomedical image inspection · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/10068

  Endoscopic image · CPC title

• A61B1/00045

  Display arrangement · CPC title

• A61B1/00009 Primary

  of image signals during a use of endoscope · CPC title