Systems and methods for displaying medical video images and/or medical 3D models

Patent metadata

Field	Value
Publication number	US-12016737-B2
Application number	US-202217592270-A
Country	US
Kind code	B2
Filing date	Feb 3, 2022
Priority date	Aug 19, 2019
Publication date	Jun 25, 2024
Grant date	Jun 25, 2024

Abstract

Computer-assisted surgical systems and methods provide intraoperative playback of and interaction with recorded video images and/or a 3D model while displaying current video images. The methods and related surgical systems involve capturing, by a two-dimensional (2D) or three-dimensional (3D) video camera, current video images and recording the captured video images. A user interface displays the current video images and the recorded video images and/or the 3D model and enables a user to interact with either or both of them.

First claim

What is claimed is:

1. An intraoperative method comprising:
capturing, by a video camera, video images;
recording at least a portion of the video images, yielding recorded video images;
causing simultaneous display of at least a portion of the recorded video images and current video images captured by the video camera;
receiving a signal to highlight an anatomic structure;
determining positions of the anatomic structure in the at least a portion of the recorded video images; and
highlighting, in response to the signal to highlight the anatomic structure, the anatomic structure in the at least a portion of the recorded video images based on the positions of the anatomic structure, yielding a highlighted anatomic structure;
determining that the highlighted anatomic structure is outside of a window; and
in response to determining that the highlighted anatomic structure is outside of the window, causing display of an object in the window indicating a direction of a position of the highlighted anatomic structure or the direction, in the recorded video images, of a recorded video image that contains at least a portion of the highlighted anatomic structure.

2. The method of claim 1 , wherein the video camera is a laparoscopic video camera or an endoscopic video camera.

3. The method of claim 1 , further comprising receiving a signal indicating selection of at least one of the current video images,
wherein recording the at least a portion of the video images includes recording the selected at least one of the current video images.

4. The method of claim 1 , further comprising:
causing display of the current video images in a first window; and
causing display of the at least a portion of the recorded video images in a second window.

5. The method of claim 4 , wherein the second window is smaller than the first window, and
further comprising overlaying the second window on the first window.

6. The method of claim 5 , further comprising overlaying the second window on a corner portion or a quadrant of the first window.

7. The method of claim 1 , further comprising:
causing display of the current video images in a first window; and
causing display of the at least a portion of the recorded video images in a second window, which is smaller than the first window and which is overlaid on the first window.

8. The method of claim 1 , further comprising causing display of the at least a portion of the recorded video images as semitransparent video images overlaid on the current video images.

9. The method of claim 1 , further comprising causing display of forward or backward movement through the recorded video images in response to a signal to move through the recorded video images in a forwards or backwards direction, respectively, while causing display of the current video images.

10. A system comprising:
a video capturing device configured to capture video images;
a processor;
a memory coupled to the processor and configured to store at least a portion of the video images, yielding stored video images; and
a display configured to simultaneously display at least a portion of the stored video images and current video images captured by the video capturing device,
wherein the memory is further configured to store instructions, which, when executed by the processor, causes the processor to:
receive a signal to highlight an anatomic structure;
determine a position of the anatomic structure in the at least a portion of the stored video images; and
highlight, in response to the signal to highlight the anatomic structure, the anatomic structure in the at least a portion of the stored video images based on the positions of the anatomic structure, yielding a highlighted anatomic structure;
cause display of a window on the display;
determine that the highlighted anatomic structure is outside of the window; and
in response to determining that the highlighted anatomic structure is outside of the window, cause display of an object in the window indicating a direction of a position of the highlighted anatomic structure or the direction, in the stored video images, of a stored video image that contains at least a portion of the highlighted anatomic structure.

11. The system of claim 10 , wherein the video capturing device is a laparoscopic video camera or an endoscopic video camera.

12. The system of claim 10 , wherein the instructions, when executed by the processor, further cause the processor to receive a user selection of at least one of the current video images, and
wherein storing the at least a portion of the video images includes storing the user selection of the at least one of the current video images.

13. The system of claim 10 , wherein the instructions, when executed by the processor, further cause the processor to:
cause display of a first window and a second window on the display;
cause display of the current video images in the first window; and
cause display of the at least a portion of the stored video images in the second window.

14. The system of claim 13 , wherein the second window is smaller than the first window, and
wherein the instructions, when executed by the processor, further cause the processor to overlay the second window on the first window.

15. The system of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to overlay the second window on a corner portion or a quadrant of the first window.

16. The system of claim 10 , wherein the instructions, when executed by the processor, further cause the processor to:
receive a signal to move through the stored video images in a forwards or backwards direction; and
cause display of forward or backward movement through the stored video images in response to the signal to move through the stored video images in a forwards or backwards direction, respectively, while causing display of the current video images.

17. The system of claim 10 , wherein the instructions, when executed by the processor, further cause the processor to cause display of the at least a portion of the stored video images as semitransparent video images overlaid on the current video images.

18. A system comprising:
a video capturing device configured to capture video images;
a display configured to display the video images;
a processor; and
a memory configured to store the video images, yielding stored video images, and configured to store instructions, which, when executed by the processor, causes the processor to:
cause simultaneous display of a first window and a second window on the display;
cause display of current video images in the first window;
cause display of the stored video images in the second window;
receive a signal to move through the stored video images;
cause display of movement through the stored video images in response to the signal to move through the stored video images;
receive a signal to highlight an anatomic structure;
determine a position of the anatomic structure in the stored video images;
highlight, in response to the signal to highlight the anatomic structure, the anatomic structure in the stored video images based on the positions of the anatomic structure;
determine that the highlighted anatomic structure is outside of the second window; and
in response to determining that the highlighted anatomic structure is outside of the second window, cause display of an object in the second window indicating a direction of a position of the highlighted anatomic structure or the direction, in the stored video ima

Assignees

• Covidien Lp

Inventors

• Komp John W
• Cantrall Irena
• Rossetto Francesca
• Pierce Robert W
• Morrison Fiona A
• Florio Mark N

Classifications

• A61B1/00194

  adapted for three-dimensional imaging · CPC title

• A61B1/000094

  extracting biological structures · CPC title

• G02B23/2415

  Stereoscopic endoscopes · CPC title

• A61B1/05

  characterised by the image sensor, e.g. camera, being in the distal end portion · CPC title

• A61B1/00055

  for alerting the user · CPC title