Luminal structure calculation apparatus, creation method for luminal structure information, and non-transitory recording medium recording luminal structure information creation program

Patent metadata

Field	Value
Publication number	US-12299922-B2
Application number	US-202217888907-A
Country	US
Kind code	B2
Filing date	Aug 16, 2022
Priority date	Feb 19, 2020
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

A luminal structure calculation apparatus includes at least one processor including hardware. The processor acquires picked-up images at a plurality of points in time including a same site of an object acquired by an image pickup unit provided in an insertion section inserted into a lumen serving as the object and three-dimensional disposition including information concerning at least a part of a position or a direction of the image pickup unit and calculates a position of the same site based on the picked-up images at the plurality of points in time and the three-dimensional disposition to calculate a three-dimensional structure of the lumen.

First claim

What is claimed is:

1. A luminal structure calculation apparatus comprising:
at least one processor including hardware, wherein the at least one processor is configured to:
acquire picked-up images at a plurality of points in time including a plurality of feature points that are a same site of an object acquired by an image sensor provided in an insertion section inserted into a lumen serving as the object and three-dimensional disposition including information concerning at least a part of a position or a direction of the image sensor,
calculate a plurality of partial three-dimensional structures, which are parts of a three-dimensional structure of the lumen, including information concerning positions of the feature points based on the picked-up images at the plurality of points in time and the three-dimensional disposition,
determine, based on the three-dimensional disposition at a time when the respective partial three-dimensional structures are created, disposition of the plurality of partial three-dimensional structures, and
correct the disposition of the partial three-dimensional structures based on positions of the feature points common to the plurality of partial three-dimensional structures to calculate the three-dimensional structure of the lumen,
wherein the lumen includes a fixed portion fixed from an outside and a movable portion not fixed from the outside, and
when the feature points similar among the plurality of partial three-dimensional structures are extracted, the at least one processor is further configured to compare a difference between distances from the similar feature points to the fixed portion with a predetermined reference value to determine whether the similar feature points are the feature points common to the plurality of partial three-dimensional structures.

2. The luminal structure calculation apparatus according to claim 1 , wherein the at least one processor is configured to:
extract the plurality of feature points in the respective picked-up images,
calculate positions in a three-dimensional space of the respective feature points from positions on the picked-up images of the plurality of feature points and the three-dimensional disposition of the insertion section, and
calculate the three-dimensional structure of the lumen from the calculated positions of the respective feature points in the three-dimensional space.

3. The luminal structure calculation apparatus according to claim 2 , wherein the at least one processor is configured to:
extract the plurality of feature points reflected in common in the picked-up images at the plurality of points in time,
acquire the three-dimensional disposition information of the insertion section at the plurality of points in time, and
calculate a position in the three-dimensional space of the plurality of feature points based on the three-dimensional disposition information of the insertion section at the plurality of points in time and positions on the picked-up images of the plurality of feature points reflected in common in the picked-up images at the plurality of points in time.

4. The luminal structure calculation apparatus according to claim 1 , wherein
the at least one processor is configured to calculate a distance between the partial three-dimensional structure corresponding to the movable portion and the fixed portion and correct the disposition of the partial three-dimensional structures based on the distance between the partial three-dimensional structure and the fixed portion.

5. The luminal structure calculation apparatus according to claim 2 , wherein the at least one processor is configured to determine positions in the three-dimensional space of the respective feature points with error adjustment.

6. The luminal structure calculation apparatus according to claim 5 , wherein the at least one processor is configured to regard the disposition detected by a detection apparatus as known and determine the positions in the three-dimensional space of the respective feature points with the error adjustment by bundle adjustment.

7. The luminal structure calculation apparatus according to claim 2 , wherein the at least one processor is configured to calculate a position in the three-dimensional space of the feature point based on triangulation from the three-dimensional disposition information of the insertion section at the plurality of points in time and a position on the picked-up images of the feature points reflected in common in the picked-up images at the plurality of points in time.

8. The luminal structure calculation apparatus according to claim 1 , wherein the at least one processor is configured to determine a three-dimensional structure of the lumen based on an illuminance difference stereo image from an image in a shadow region in the picked-up images at the plurality of points in time including the same site of the object obtained by switching an illumination unit caused to emit illumination light in a plurality of illumination units provided at the distal end portion.

9. The luminal structure calculation apparatus according to claim 1 , wherein the at least one processor is configured to determine, using a predetermined extension and contraction ratio upper limit value of respective feature points set for each region of the lumen, whether common feature points reflected in the picked-up images in plurality are same.

10. The luminal structure calculation apparatus according to claim 9 , wherein, when a change amount of a position of a detected feature point is equal to or smaller than the predetermined extension and contraction ratio upper limit value, the at least one processor is configured to determine that the respective feature points reflected in the respective picked-up images are the same.

11. The luminal structure calculation apparatus according to claim 1 , wherein the at least one processor is configured to perform predetermined notification when a difference of a luminance value between two pixels or two pixel regions adjacent to each other in the picked-up images is equal to or larger than a predetermined value.

12. A creation method for luminal structure information comprising:
acquiring picked-up images of an object at a plurality of points in time including a plurality of feature points that are a same site of the object acquired by an image sensor provided in an insertion section inserted into a lumen serving as the object;
acquiring three-dimensional disposition including information concerning at least one of a position or a direction of the insertion section;
calculating a plurality of partial three-dimensional structures, which are parts of a three-dimensional structure of the lumen, including information concerning positions a of the feature points based on the picked-up images at the plurality of points in time and the three-dimensional disposition,
determining, based on the three-dimensional disposition at a time when the respective partial three-dimensional structures are created, disposition of the plurality of partial three-dimensional structures,
correcting the disposition of the partial three-dimensional structures based on positions of the feature points common to the plurality of partial three-dimensional structures to calculate the three-dimensional structure of the lumen, and
when the feature points similar among the plurality of partial three-dimensional structures are extracted, comparing a difference between distances from the similar feature points to the fixed portion with a predetermined reference value to determine whether the similar feature points are the feature points common to the plurality of partial three-dimensional structures.

Assignees

• Olympus Corp

Inventors

• Hane Jun

Classifications

• G06T2207/30028

  Colon; Small intestine · CPC title

• G06T2207/10068

  Endoscopic image · CPC title

• G06T7/55

  from multiple images · CPC title

• A61B1/06

  with illuminating arrangements · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title