Methods and systems for providing visuospatial information

Patent metadata

Field	Value
Publication number	US-11071593-B2
Application number	US-201715650253-A
Country	US
Kind code	B2
Filing date	Jul 14, 2017
Priority date	Jul 14, 2017
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

Methods and systems for providing feedback during a medical procedure. The 3D position and orientation of a tracked tool are determined, relative to a site of the medical procedure, based on tracking information received from a tracking system. The 3D position and orientation are mapped to a common coordinate space, to determine the 3D position and orientation relative to a field-of-view (FOV) of an optical camera that is capturing an optical image of the site. Navigational information associated with the 3D position and orientation is determined. A virtual representation of the navigational information overlaid on the FOV and displayed. The displayed virtual representation is updated when the 3D position and orientation of the tracked tool changes or when the FOV changes, in accordance with the change.

First claim

The invention claimed is:

1. A system for providing feedback during a medical procedure, the system comprising:
a tracking system configured to obtain tracking information about three-dimensional (3D) position and orientation of a tracked tool during the medical procedure;
the tracked tool coupled to tracking markers to enable tracking of the tracked tool by the tracking system;
a camera for capturing an optical image of a field-of-view (FOV) of a site and the tracked tool during the medical procedure;
a display for displaying the optical image of the FOV; and
an input mechanism for indicating selection of one or more virtual 3D points in the optical image of the FOV, the input mechanism being part of the tracked tool;
a processor coupled to receive input data from the tracking system, the input mechanism, and the camera, and coupled to transmit output data for display on the display, the processor being configured to:
determine the 3D position and orientation of the tracked tool, relative to the site, based on the tracking information;
receive selection of one or more 3D points from the input mechanism based on the 3D position and orientation of the tracked tool;
map the 3D position and orientation of the tracked tool and the one or more selected 3 D points to a common coordinate space, to determine the 3D position and orientation of the tracked tool and the one or more selected 3D points relative to the FOV;
determine navigational information associated with the 3D position and orientation of the tracked tool and the one or more selected 3D points, the navigational information including a measurement of distance or angle, wherein determining navigational information comprises determining a measurement between the 3D position and orientation of the tracked tool to a reference defined by at least one of the one or more selected 3D points in the site;
cause the display to display a virtual representation of the navigational information overlaid on the FOV of the site and the tracked tool, the navigational information being displayed as an indication of the measurement overlaid on the FOV; and
update the displayed virtual representation by:
when the 3D position and orientation of the tracked tool changes, updating the displayed virtual representation of the one or more selected 3D points in accordance with the changed 3D position and orientation of the tracked tool; or
when the FOV changes, updating the displayed virtual representation of the one or more selected 3D points to follow the changed FOV.

2. The system of claim 1 , wherein the tracking system is further configured to:
track position and orientation of the camera or a support of the camera; and
track position and orientation of the site;

wherein the processor is further configured to:
map between the 3D position and orientation of the tracked tool, the site and the FOV, using information from the tracking system; and
cause the display to update the displayed virtual representation to follow the changed FOV, based on mapping between the 3D position and orientation of the tracked tool, the site and the FOV, using information from the tracking system.

3. The system of claim 2 , wherein the processor is further configured to:
determine an orientation of the camera relative to a reference orientation, using information from the tracking system; and
cause the display to display information overlaid on the FOV indicating the orientation of the FOV relative to the reference orientation.

4. The system of claim 1 , wherein the processor is configured to:
receive selection of multiple 3D points indicated by the tracked tool;
determine a region of interest (ROI) based on the selected multiple 3D points;
determine navigational information associated with the ROI; and
cause the display to display a virtual representation of the ROI overlaid on the FOV, the virtual representation including the navigational information.

5. The system of claim 4 , wherein the navigational information comprises imaging data of the ROI, wherein the overlay on the FOV includes the imaging data only for the ROI.

6. The system of claim 4 , wherein the processor is further configured to modify visual display of the optical image of the FOV within the ROI.

7. The system of claim 1 , wherein the processor is further configured to:
receive selection of multiple 3D points indicated by the tracked tool;
determine a boundary of interest (BOI) based on the selected multiple 3D points;
cause the display to display a virtual representation of the BOI overlaid on the FOV; and
cause output of feedback indicating position and orientation of another tracked tool relative to the BOI.

8. The system of claim 4 , wherein the processor is further configured to:
determine navigational information by determining orientation of a longitudinal axis of the tracked tool; and
cause the display to display a virtual representation of the navigational information as a reference line overlaid on the FOV.

9. The system of claim 1 , wherein the measurement is a measurement of the distance between the 3D position of the tracked tool and the at least one of the one or more selected 3D points.

10. The system of claim 1 , wherein the at least one of the one or more selected 3D points defines part of a reference line, and the measurement is a measurement of an angle between the 3D orientation of the tracked tool and the reference line.

11. The system of claim 1 , wherein the at least one of the one or more selected 3D points defines part of a reference depth plane, and the measurement is a measurement of a depth difference between the 3D position of the tracked tool and the reference depth plane.

12. The system of claim 1 , wherein the processor is further configured to:
receive planning information defining a planned trajectory or planned target;
wherein the navigational information is extracted from the planning information and displayed in association with the 3D position and orientation of the tracked tool overlaid on the FOV.

13. The system of claim 1 , wherein the processor is further configured to:
cause the display to display a virtual representation of the 3D position and orientation of the tracked tool overlaid on a display of imaging data.

14. The system of claim 13 , wherein the processor is further configured to:
cause the display to display a portion of the FOV, corresponding to the 3D position and orientation of the tracked tool, overlaid on the imaging data.

15. The system of claim 1 , wherein the system comprises an additional input mechanism for selecting an additional virtual 3D point via user interaction with the displayed FOV, and wherein the processor is further configured to:
receive selection of the additional virtual 3D point via user interaction with the displayed FOV.

16. A method for providing feedback during a medical procedure, the method comprising:
determining a 3D position and orientation of a tracked tool, relative to a site of the medical procedure, based on tracking information received from a tracking system that is tracking the tracked tool;
receiving selection of one or more 3D points from an input mechanism based on the 3D position and orientation of the tracked tool, the input mechanism being part of the tracked tool;
mapping the 3D position and orientation of the tracked tool and the one or more selected 3D points to a common coordinate space, to determine the 3D position and orientation of the tracked tool and the one or more selected 3D points relative to a field- of-view (FOV) of a camera that is capturi

Assignees

• Abhari Kamyar
• Mclachlin Stewart David
• Hynna Kai Michael
• Sela Gal
• Shoup Jared Rowland
• Bulk Michael Peter
• Dyer Kelly Noel
• Synaptive Medical Inc

Inventors

• Abhari Kamyar
• Mclachlin Stewart David
• Hynna Kai Michael
• Sela Gal
• Shoup Jared Rowland
• Bulk Michael Peter
• Dyer Kelly Noel

Classifications

• A61B34/25 Primary

  User interfaces for surgical systems · CPC title

• A61B17/3421

  Cannulas · CPC title

• A61B2010/045

  Needles (pointed biopsy devices with vacuum aspiration A61B10/0283) · CPC title

• A61B34/20 Primary

  Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis · CPC title

• A61B2090/365

  augmented reality, i.e. correlating a live optical image with another image · CPC title