Surgeon head-mounted display apparatuses

Patent metadata

Field	Value
Publication number	US-12002171-B2
Application number	US-202217935585-A
Country	US
Kind code	B2
Filing date	Sep 27, 2022
Priority date	Feb 3, 2015
Publication date	Jun 4, 2024
Grant date	Jun 4, 2024

Abstract

An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

First claim

What is claimed is:

1. An augmented reality head mounted display (HMD) system comprising:
a housing;
an adjustable headband attached to the HMD housing and configured to adjustably grasp a user's head;
a partially transparent display screen extending downwardly from the housing;
a display module adapted to generate video images to be projected onto the display screen to be combined with real world images for viewing by the user;
a plurality of reference markers attached to the housing and positioned to be visible to a position tracking system;
a computer equipment configured to:
determine the location and orientation of the reference markers based on input from the position tracking system;
generate a three dimensional anatomical image of a patient;
rotate and scale the generated three dimensional anatomical image based on the determined location and orientation of the reference markers; and
display through the display module the rotated and scaled image on the display screen for viewing by the user.

2. The system of claim 1 , wherein the adjustable headband comprises:
an adjustable side band extending from the housing and laterally around the user's head;
a central band extending vertically from the housing and over the user's head.

3. The system of claim 2 , wherein the central band is adapted to be length adjustable.

4. The system of claim 3 , wherein one end of the central band is attached to the housing and the other end is attached to the adjustable side band.

5. The system of claim 4 , wherein the adjustable side band includes a first lateral band extending from one side of the housing and a second lateral band extending from the other side of the housing.

6. The system of claim 1 , wherein the adjustable headband comprises:
an adjustable side band extending from the housing and laterally around the user's head;
a central band extending vertically from the housing and over the user's head;
a side band knob for adjustably fixing the side band length; and
a central band knob for adjustably fixing the central band length.

7. The system of claim 1 , further comprising at least one front facing camera adapted to be connected to a gesture interpretation module that interprets hand gestures of the user.

8. The system of claim 1 , further comprising a plurality of patient tracking markers adapted to be attached to the patient and positioned to be visible to a position tracking system, wherein the computer equipment is configured to rotate and scale the generated three dimensional anatomical image based on the determined locations and orientations of the reference marker and the patient tracking markers.

9. An augmented reality head mounted display (HMD) system comprising:
a housing;
an adjustable headband attached to the HMD housing and configured to adjustably grasp a user's head;
a partially transparent display screen extending downwardly from the housing;
a display module adapted to generate video images to be projected onto the display screen to be combined with real world images for viewing by the user;
a plurality of patient tracking markers adapted to be attached to the patient and positioned to be visible to a position tracking system;
a computer equipment configured to:
determine the location and orientation of the patient tracking markers based on input from the position tracking system;
generate a three dimensional anatomical image of a patient;
rotate and scale the generated three dimensional anatomical image based on the determined location and orientation of the patient tracking markers; and
display through the display module the rotated and scaled image on the display screen for viewing by the user.

10. An augmented reality head mounted display (HMD) system comprising:
an HMD including:
a housing;
an adjustable headband attached to the HMD housing and configured to adjustably grasp a user's head;
a plurality of cameras configured to output video images;
a plurality of reference markers attached to the housing and positioned to be visible to a camera tracking system;
a partially transparent display screen extending downwardly from the housing;
a display module adapted to generate video images to be projected onto the display screen to be combined with real world images for viewing by the user;

a computer equipment configured to:
determine the location and orientation of the reference markers based on input from the camera tracking system;
generate a three dimensional anatomical image of a patient;
rotate and scale the generated three dimensional anatomical image based on the determined location and orientation of the reference markers; and
display through the display module the rotated and scaled image on the display screen for viewing by the user.

11. The system of claim 10 , wherein the adjustable headband comprises:
an adjustable side band extending from the housing and laterally around the user's head;
a central band extending vertically from the housing and over the user's head.

12. The system of claim 11 , wherein the central band is adapted to be length adjustable.

13. The system of claim 12 , wherein one end of the central band is attached to the housing and the other end is attached to the adjustable side band.

14. The system of claim 13 , wherein the adjustable side band includes a first lateral band extending from one side of the housing and a second lateral band extending from the other side of the housing.

15. The system of claim 10 , wherein the adjustable headband comprises:
an adjustable side band extending from the housing and laterally around the user's head;
a central band extending vertically from the housing and over the user's head;
a side band knob for adjustably fixing the side band length; and
a central band knob for adjustably fixing the central band length.

16. The system of claim 10 , further comprising at least one front facing camera adapted to be connected to a gesture interpretation module that interprets hand gestures of the user.

17. The system of claim 10 , further comprising a plurality of patient tracking markers adapted to be attached to the patient and positioned to be visible to the camera tracking system, wherein the computer equipment is configured to rotate and scale the generated three dimensional anatomical image based on the determined locations and orientations of the reference marker and the patient tracking markers.

Assignees

• Globus Medical Inc

Inventors

• JONES Kenneth Milton
• Popoolapade John
• CALLOWAY Thomas
• Lemoine Thierry
• Jutteau Christian
• Bruzy Christophe
• James Yannick
• LAGUARDA Joachim
• XING Dong-Mei Pei
• Gorges Sebastien
• Yarin Paul Michael

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A61B34/10

  Computer-aided planning, simulation or modelling of surgical operations · CPC title

• A61B90/37

  Surgical systems with images on a monitor during operation · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title