Surgical devices, systems, and methods using fiducial identification and tracking

Patent metadata

Field	Value
Publication number	US-12472002-B2
Application number	US-202117493920-A
Country	US
Kind code	B2
Filing date	Oct 5, 2021
Priority date	Sep 29, 2021
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

In general, devices, systems, and methods for fiducial identification and tracking are provided.

First claim

What is claimed is:

1 . A surgical system, comprising:
a first imaging device configured to gather a first image of a surgical site during performance of a surgical procedure, the first image being an undistorted visualization of the surgical site; a second imaging device configured to gather a second image of the surgical site during the performance of the surgical procedure, the second image being a distorted visualization of the surgical site; and a processor operatively coupled to the first and second imaging devices, the processor configured to:
receive information indicating the first image of the undistorted visualization of the surgical site and the second image of the distorted visualization of the surgical site;
identify a critical structure associated with the surgical site;
determine that the distorted visualization of the surgical site includes a portion of the identified critical structure;
based at least on the determination that the distorted visualization of the surgical site includes at least the portion of the identified critical structure, supplement the second image by filling in an element missing from the second image with an element present in the first image to reduce the distorted visualization of the second image;
generate a third image based on the supplemented second image; and
send an indication of the third image to a display, to cause the display to show the third image.

2 . The system of claim 1 , wherein the second image is distorted due to at least one of strain of tissue at the surgical site, compression of tissue at the surgical site, collapse of tissue at the surgical site, or obstruction of an image sensor of the second imaging device by at least one of tissue or fluid at the surgical site.

3 . The system of claim 1 , wherein the element missing from the second image associated with the portion of the identified critical structure.

4 . The system of claim 1 , wherein the supplementing further includes replacing a distorted element in the second image with an undistorted element in the first image.

5 . The system of claim 1 , wherein the processor is further configured to:
compare pixels of the first image with pixels of the second image to detect an irregularity in the second image; and correct the irregularity based on the pixels of the first image, wherein the second image is further supplemented based on the corrected irregularity.

6 . The system of claim 5 , wherein the first and second images are video images, each including a plurality of frames, and wherein the processor is further configured to:
obtain a first set of pixels in a first frame of the first image, wherein the first set of pixels is gathered within a predefined time period associated with a second frame of the second image; and replace a second set of pixels in the second frame of the second image with the first set of pixels in the first frame of the first image, wherein the second image is supplemented further based on the replaced second set of pixels.

7 . The system of claim 1 , wherein the first and second images are video images, each including a plurality of frames, and wherein the processor is further configured to:
identify a distorted frame in the second image; and correct the distorted frame in the second image using a frame in the first image that was gathered within a predefined time period of the distorted frame of the second image.

8 . The system of claim 1 , wherein the first imaging device is further configured to gather the first image using visible light, and wherein the second imaging device is further configured to gather the second image using non-visible light.

9 . The system of claim 1 , wherein the processor is further configured to:
obtain a first set of pixels in a first frame from the first image, wherein the first set of pixels is gathered within a predefined time period associated with a second frame; and merge a second set of pixels in the second frame of the second image with the first set of pixels in the first frame to obtain merged pixels, wherein the second image is supplemented further based on the merged pixels.

10 . The system of claim 1 , wherein a robotic surgical system includes the processor, and the first and second imaging devices are each configured to be releasably coupled to and controlled by the robotic surgical system.

11 . The system of claim 1 , wherein the first imaging device has a first point of view (POV) of the surgical site when the first image is gathered; and
the second imaging device has a second POV of the surgical site when the second image is gathered, wherein the second POV and the first POV are different.

12 . The system of claim 1 , wherein the processor is configured to supplement the second image further based on the distorted visualization of the surgical site being within a predetermined threshold distance of the second imaging device.

13 . The system of claim 1 , wherein the critical structure includes at least one of an anatomical structure, an artery, a vein, a tumor, or a foreign structure.

14 . A surgical method, comprising:
receiving information during performance of a surgical procedure, wherein the information indicates a first image of an undistorted visualization of a surgical site from a first surgical device and a second image of a distorted visualization of the surgical site; identifying a critical structure associated with the surgical site; determining that the distorted visualization of the surgical site includes a portion of the identified critical structure; based on the determination that the distorted visualization of the surgical site includes at least the portion of the identified critical structure, supplementing the second image by filling in an element missing from the second image with the element present in the first image to reduce the distorted visualization of the second image; generating a third image based on the supplemented second image; and sending an indication of the third image to a display, to cause the display to show the third image.

15 . A surgical system, comprising:
a first imaging device configured to gather a first image of an undistorted visualization of a surgical site from a first point of view (POV) during performance of a surgical procedure, wherein the first image is gathered via a common wavelength of light, and wherein the first imaging device is configured to be advanced to the surgical site through a first cannulated access device; a second imaging device configured to gather a second image of a distorted visualization of the surgical site from a second POV during the performance of the surgical procedure, wherein the second image is gathered via the common wavelength of light, and wherein the second imaging device is configured to be advanced to the surgical site through a second cannulated access device independently of the first imaging device; and a processor operatively coupled to the first and the second imaging devices, the processor configured to:
identify a critical structure associated with the surgical site;
determine that distorted visualization of the surgical site includes at least a portion of the critical structure; and
based on the determination that the distorted visualization of the surgical site includes the portion of the critical structure, supplement the second image by filling in an element missing from the second image with an element present in the first image to reduce the distorted visualization of the second image;
generate a third image based on the supplemented second image; and
send an

Assignees

• Cilag Gmbh Int

Inventors

• Shelton Iv Frederick E
• Scheib Charles J
• Deck Andrew C

Classifications

• A61B1/042

  characterised by a proximal camera, e.g. a CCD camera · CPC title

• A61B1/000095

  for image enhancement · CPC title

• G06T5/80

  Geometric correction · CPC title

• G06T5/77

  Retouching; Inpainting; Scratch removal · CPC title

• A61B2090/378

  using ultrasound · CPC title