Methods and systems for display of patient data in computer-assisted surgery

Patent metadata

Field	Value
Publication number	US-11737850-B2
Application number	US-202117410024-A
Country	US
Kind code	B2
Filing date	Aug 24, 2021
Priority date	Sep 9, 2016
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

Methods and systems for performing computer-assisted image-guided surgery, including robotically-assisted surgery. A method of displaying image data includes displaying image data of a patient on a handheld display device, tracking the handheld display device using a motion tracking system, and modifying the image data displayed in response to changes in the position and orientation of the handheld display device. Further embodiments include a sterile case for a handheld display device, display devices on a robotic arm, and methods and systems for performing image-guided surgery using multiple reference marker devices fixed to a patient.

First claim

The invention claimed is:

1. A method of obtaining and displaying patient-specific image data, the method comprising:
obtaining image data of a patient using an imaging device;
displaying the image data of the patient on a display screen of a handheld display device;
tracking, with a motion tracking device, movement of the handheld display device;
supporting an end effector relative to the patient with a robotic arm coupled to the imaging device;
tracking, with the motion tracking device coupled to the imaging device, movement of the end effector; and
modifying at least a portion of the image data displayed on the display screen in response to movement of the handheld display device and the end effector with respect to the patient, wherein the image data indicates a pose of the end effector of the robotic arm.

2. The method of claim 1 , wherein the image data comprises a portion of a three-dimensional dataset of anatomy of the patient, and the three-dimensional dataset comprises at least one of x-ray computed tomography (CT) reconstruction image data and magnetic resonance (MR) image data.

3. The method of claim 2 , wherein displaying image data on the display screen of the handheld display device comprises:
determining a first pose of the handheld display device with respect to the patient; and
displaying a first portion of the three-dimensional dataset of anatomy of the patient on the display screen based on the pose of the handheld display device.

4. The method of claim 3 , further comprising: determining a second pose of the handheld display device with respect to the patient.

5. The method of claim 4 , wherein displaying the first portion of the three-dimensional dataset comprises: displaying at least one two-dimensional slice of the three-dimensional dataset of anatomy of the patient in at least one first plane defined by the first pose of the handheld display device with respect to the patient; and
wherein modifying at least a portion of the image data displayed on the display screen comprises: displaying the at least one two-dimensional slice of the three-dimensional dataset of anatomy of the patient in at least one second plane defined by the second pose of the handheld display device with respect to the patient.

6. The method of claim 4 , wherein displaying the first portion of the three-dimensional dataset comprises: displaying a three-dimensional rendering of anatomy of the patient in a first view based on the first pose of the handheld display device with respect to the patient; and
wherein modifying at least a portion of the image data displayed on the display screen comprises displaying the three-dimensional rendering of anatomy of the patient in a second view based on the second pose of the handheld display device with respect to the patient.

7. The method of claim 2 , further comprising: displaying a camera image of the patient obtained by a camera on the handheld display device on the display screen, wherein the portion of a three-dimensional dataset of anatomy of the patient is displayed overlaying the camera image.

8. The method of claim 7 , wherein the portion of the three-dimensional dataset comprises a three-dimensional volume rendering of a portion anatomy of the patient visible in the camera image.

9. The method of claim 8 , further comprising performing a calibration process to match the three-dimensional volume rendering to a view of the patient in the camera image.

10. The method of claim 9 , further comprising updating the three-dimensional volume rendering based on movement of the camera relative to the patient.

11. The method of claim 1 , further comprising registering at least a first portion of the image data in a first patient coordinate system associated with a first reference marker device fixed to a bony structure of the patient at a first location.

12. The method of claim 11 , further comprising:
registering at least a second portion of the image data to a second patient coordinate system associated with a second reference marker device fixed to a bony structure of the patient at a second location that is different than the first location; and
selecting between a display of patient images registered to the first patient coordinate system and a display of patient images registered to the second patient coordinate system in an image guided surgery system based on a proximity to the first and second locations.

13. The method of claim 1 , wherein the handheld display device comprises at least one of a tablet computer, a smartphone, and a pendant controller.

14. A surgical system comprising:
an imaging device;
a motion tracking system comprising a motion tracking device;
a robotic arm coupled to the imaging device, the robotic arm supporting an end effector; and
a handheld display device configured to display image data obtained by the imaging device;
wherein the end effector and the handheld display device each include a respective plurality of markers mounted in a pre-determined geometric pattern to enable tracking via the motion tracking device.

15. The surgical system of claim 14 , wherein the markers include spherically-shaped reflectors; and
wherein the pre-determined geometric pattern is asymmetric such that the pattern is different when viewed from different angles.

16. The surgical system of claim 14 , wherein the imaging device includes a positioning system coupling the robotic arm with the imaging device.

17. The surgical system of claim 14 , further comprising a controller in communication with the handheld display device and configured to detect a motion of the handheld display device with respect to a viewing position and to modify at least one image shown on the handheld display device in response to the detected motion.

18. The surgical system of claim 17 , wherein the controller is configured to maintain at least one image on the handheld display such that it remains visible from the viewing position while the handheld display device moves.

19. The surgical system of claim 14 , wherein the motion tracking device is coupled to the imaging device.

20. The surgical system of claim 14 , further including a sterile case supporting the handheld display device.

Assignees

• Mobius Imaging Llc

Inventors

• Gregerson Eugene A
• Coppen Scott
• Furlong Todd
• Daley Edward
• Stanton Russell
• Harris Adeline
• Sebring Paul

Classifications

• A61B90/37 Primary

  Surgical systems with images on a monitor during operation · CPC title

• A61B5/055 Primary

  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging · CPC title

• A61B5/1127

  using markers · CPC title

• A61B6/032

  Transmission computed tomography [CT] · CPC title

• A61B6/461

  Displaying means of special interest · CPC title