Leveraging two-dimensional digital imaging and communication in medicine imagery in three-dimensional extended reality applications

Patent metadata

Field	Value
Publication number	US-2025032210-A1
Application number	US-202418908897-A
Country	US
Kind code	A1
Filing date	Oct 8, 2024
Priority date	May 8, 2020
Publication date	Jan 30, 2025
Grant date	—

Abstract

A surgical system includes an XR headset and an XR headset controller. The XR headset is configured to be worn by a user during a surgical procedure and includes a see-through display screen configured to display an XR image for viewing by the user. The XR headset controller is configured to receive a plurality of two-dimensional (“2D”) image data associated with an anatomical structure of a patient. The XR headset controller is further configured to generate a first 2D image from the plurality of 2D image data based on a pose of the XR headset. The XR headset controller is further configured to generate a second 2D image from the plurality of 2D image data based on the pose of the XR headset. The XR headset controller is further configured to generate the XR image by displaying the first 2D image in a field of view of a first eye of the user and displaying the second 2D image in a field of view of a second eye of the user.

First claim

What is claimed is:

1 . A method of operating an XR headset in a surgical system, the method comprising:
providing a 3D volume of an anatomical structure of a 3D patient scan from a medical imaging device, the 3D volume containing both bone anatomy and non-bone anatomy surrounding the bone anatomy and which has not been transformed into a 3D mesh; generating, by an XR headset controller, a first 2D image representing an MIP (maximal intensity projection) view from a perspective of a left eye from the provided 3D volume based on a pose of the XR headset; generating, by the XR headset controller, a second 2D image representing an MIP view from a perspective of a right eye from the provided 3D volume based on a pose of the XR headset; generating, by the XR headset controller, a XR image by displaying the first 2D image in a field of view of the left eye of the user and displaying the second 2D image in a field of view of the right eye of the user such that the displayed first and second 2D images provide a depth perception.

2 . The method of claim 1 , wherein providing a 3D volume of an anatomical structure of a 3D patient scan includes providing a plurality of 2D image data as a plurality of 2D scans of the anatomical structure.

3 . The method of claim 1 , wherein providing a 3D volume of an anatomical structure of a 3D patient scan includes providing a plurality of 2D slices of a DICOM scan.

4 . The method of claim 1 , wherein generating a first 2D image includes projecting the 3D volume on a plane at an orientation defined by the pose of the XR headset.

5 . The method of claim 1 , further comprising determining the pose of the XR headset based on an array of tracking markers on the XR headset as viewed by a camera tracking system.

6 . The method of claim 1 , further comprising combining by the XR headset the generated XR image with a real-world scene for augmented reality viewing by the user.

7 . The method of claim 6 , wherein the XR headset includes an at least partially transparent screen to combine the generated XR image with the real-world scene.

8 . The method of claim 1 , further comprising:
responsive to generating the first 2D image, adjusting a window or level of the first 2D image.

9 . The method of claim 1 , wherein the XR headset includes a set of cameras, the method further comprising when a dynamic reference array of a surgical tool comes within a field of view of the cameras, displaying a graphical representation of the surgical tool on in relation to the generated XR image.

10 . The method of claim 5 , wherein the camera tracking system is further configured to track a pose of a surgical tool, wherein the method of generating the XR image includes a graphical representation of the surgical tool based on the pose of the surgical tool relative to the anatomical structure.

11 . A method of operating an XR headset in a surgical system, the method comprising:
providing a 3D volume of an anatomical structure of a 3D patient scan from a medical imaging device, the 3D volume containing both bone anatomy and non-bone anatomy surrounding the bone anatomy and which has not been transformed into a 3D mesh, the 3D volume including a plurality of 2D DICOM files defining a plurality of 2D scans at various depths; generating, by an XR headset controller, a first 2D image representing an MIP (maximal intensity projection) view from a perspective of a left eye from the provided 3D volume based on a pose of the XR headset; generating, by the XR headset controller, a second 2D image representing an MIP view from a perspective of a right eye from the provided 3D volume based on a pose of the XR headset; generating, by the XR headset controller, a XR image by displaying the first 2D image in a field of view of the left eye of the user and displaying the second 2D image in a field of view of the right eye of the user such that the displayed first and second 2D images provide a depth perception.

12 . The method of claim 11 , wherein generating a first 2D image includes projecting the 3D volume on a plane at an orientation defined by the pose of the XR headset.

13 . The method of claim 11 , further comprising determining the pose of the XR headset based on an array of tracking markers on the XR headset as viewed by a camera tracking system.

14 . The method of claim 11 , further comprising combining by the XR headset the generated XR image with a real-world scene for augmented reality viewing by the user.

15 . The method of claim 14 , wherein the XR headset includes an at least partially transparent screen to combine the generated XR image with the real-world scene.

16 . The method of claim 11 , further comprising:
responsive to generating the first 2D image, adjusting a window or level of the first 2D image.

17 . The method of claim 11 , wherein the XR headset includes a set of cameras, the method further comprising when a dynamic reference array of a surgical tool comes within a field of view of the cameras, displaying a graphical representation of the surgical tool on in relation to the generated XR image.

18 . The method of claim 13 , wherein the camera tracking system is further configured to track a pose of a surgical tool, wherein the step of generating the XR image includes a graphical representation of the surgical tool based on the pose of the surgical tool relative to the anatomical structure.

Assignees

• Globus Medical Inc

Inventors

• Dulin Isaac
• CALLOWAY Thomas
• Johnson Norbert

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title