Autonomous navigation and intervention in the gastrointestinal tract

Patent metadata

Field	Value
Publication number	US-12369997-B2
Application number	US-202217567665-A
Country	US
Kind code	B2
Filing date	Jan 3, 2022
Priority date	Jan 4, 2021
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

Implementations include herein are visual navigation strategies and systems for lumen center tracking comprising a high-level state machine for gross (i.e., left/right/center) region prediction and curvature estimation and multiple state-dependent controllers for center tracking, wall-avoidance and curve following. This structure allows a navigation system to navigate even under the presence of significant occlusion that occurs during turn navigation and to robustly recover from mistakes and disturbances that may occur while attempting to track the lumen center. This system comprises a high-level state machine for gross region prediction, a turn estimator for anticipating sharp turns, and several lower level controllers for heading adjustment.

First claim

What is claimed is:

1. An autonomous endoscope system comprising: a robotic endoscope device comprising a video camera, a microcontroller transmitting a control signal to control a movement of the robotic endoscope device; and
a navigation computing device, in response to receiving image data of a lumen from the video camera, to:
segment the image data by reducing the image data to a plurality of skeletonized segments of a predetermined pixel width, the skeletonized segments depicting an edge surrounding an interior surface of the lumen;
classify the segments of the segmented image data, the classification of the segments of the segmented image data corresponding to the skeletonized segments being one of a closed segment indicating an entirety of the edge is illustrated in the segmented image data, a segment intersecting with one or more of the edges of a frame of the image data indicating at least a portion of the edge extends beyond the one or more of the edges of the frame of the segmented image data, or an open segment indicating at least a portion of the edge is missing from the segmented image data;
estimate, based on the classification of the segments, a center point within the segmented image data corresponding to an estimated center of a lumen within the image data; and
instruct the microcontroller to control the robotic endoscope device based on the estimated center point within the segmented image data.

2. The autonomous endoscope system of claim 1 , further comprising:
an input device in communication with the navigation computing device, the input device transmitting a control instruction to the microcontroller.

3. The autonomous endoscope system of claim 1 wherein the segmented image data is classified as at least one of the closed segment, a boundary closed segment, a boundary crossing segment, or the open segment.

4. The autonomous endoscope system of claim 1 wherein estimating the center point is based on a classification of the segmented image data.

5. The autonomous endoscope system of claim 1 wherein the navigation computing device is further to:
combine a plurality of segmented image data by associating a weighted value to each of the plurality of segmented image data.

6. The autonomous endoscope system of claim 1 wherein instructing the microcontroller to control the robotic endoscope device comprises obtaining one or more control instructions from a state machine associated with a position of the robotic endoscope device.

Assignees

• Univ Colorado Regents
• Univ Colorado Regents

Inventors

• Rentschler Mark E
• Formosa Gregory
• Prendergast Joseph M
• Fulton Mitchell J

Classifications

• A61B1/041

  Capsule endoscopes for imaging · CPC title

• A61B1/00087

  Tools (A61B17/00234 takes precedence) · CPC title

• A61B1/000094

  extracting biological structures · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• A61B2034/2065

  Tracking using image or pattern recognition · CPC title