Augmented reality assisted joint arthroplasty

Patent metadata

Field	Value
Publication number	US-12491027-B2
Application number	US-202217647164-A
Country	US
Kind code	B2
Filing date	Jan 5, 2022
Priority date	Jul 9, 2019
Publication date	Dec 9, 2025
Grant date	Dec 9, 2025

Abstract

Systems and methods of providing augmented reality assisted surgery are disclosed. Certain aspects provide a method including generating a virtual scene comprising one or more virtual elements comprising one or more anatomical elements corresponding to one or more anatomy parts. The method further includes identifying one or more references in an actual physical scene comprising the one or more anatomy parts. The method further includes registering the virtual scene to the one or more references to generate an augmented scene. The method further includes displaying the augmented scene on a display device.

First claim

What is claimed is:

1 . A method of performing augmented reality assisted surgery, the method comprising:
generating a virtual scene comprising one or more virtual elements comprising one or more anatomical elements corresponding to one or more anatomy parts of a patient; acquiring one or more images of an actual physical scene comprising the one or more anatomy parts; automatically determining, based on the one or more images or another input, a step of a plurality of steps of a surgery the one or more images correspond to; limiting access to a library of augmentation element options during the step of the surgery to only show a subset of augmentation element options of the library of augmentation element options during the step of the surgery, the subset of augmentation element options corresponding to the step of the plurality of steps of the surgery; automatically identifying one or more references in the actual physical scene during the surgery; registering, during the step of the surgery, the virtual scene to the one or more references to generate an augmented scene comprising one or more augmentation elements configured based on one or more augmentation element options selected from the subset of augmentation element options; and displaying, during the surgery, the augmented scene as an overlay on an anatomy of the patient via a display device.

2 . The method of claim 1 , wherein the one or more references comprise a patient-specific guide placed on the one or more anatomy parts.

3 . The method of claim 1 , wherein the one or more images comprise at least one image of the one or more anatomy parts, and further comprising:
segmenting the at least one image to generate one or more virtual 3D models of the one or more anatomy parts.

4 . The method of claim 1 , further comprising:
acquiring one or more virtual 3D models of the one or more anatomy parts; determining one or more implant components based on the one or more virtual 3D models; and determining a size and a position for the one or more implant components based on the one or more virtual 3D models; wherein the one or more anatomical elements comprise a depiction of the one or more implant components having the determined size in the determined position.

5 . The method of claim 1 , wherein the one or more anatomical elements comprise portions of the one or more anatomy parts obscured in the one or more images.

6 . The method of claim 1 , wherein the one or more anatomical elements comprise highlights corresponding to the one or more anatomy parts.

7 . The method of claim 1 , wherein the one or more references comprise physical markers or objects in the actual physical scene.

8 . The method of claim 1 , further comprising acquiring one or more virtual 3D models of the one or more anatomy parts, wherein the one or more references are automatically determined by performing shape recognition on the one or more virtual 3D models and one or more images of the actual physical scene.

9 . The method of claim 1 , further comprising:
performing one or more measurements of the one or more anatomy parts based on the one or more images, wherein the one or more anatomical elements comprise the one or more measurements.

10 . The method of claim 1 , wherein the one or more anatomical elements comprise guidance for one or more steps of a surgical procedure, and the one or more anatomical elements are displayed in an order corresponding to the steps of the surgical procedure.

11 . The method of claim 1 , wherein the one or more anatomical elements comprise planned or simulated instrument trajectories for performing surgery on the one or more anatomy parts.

12 . The method of claim 1 , further comprising determining an alignment between an instrument in the one or more images and a planned trajectory for the instrument, wherein the one or more anatomical elements indicate the alignment.

13 . The method of claim 1 , further comprising:
generating at least one of the one or more anatomical elements that belong to the step based on the step.

14 . The method of claim 1 , wherein the library of augmentation element options comprises one or more options for configuring an augmentation element according to one or more of an augmentation element size, an augmentation element type, or an augmentation element position.

15 . The method of claim 1 , wherein the one or more references include one or more bones of the patient.

16 . The method of claim 1 , further comprising:
automatically tracking a position of an instrument in relation to a bone; and modifying the augmented scene with one or more instrument guiding elements.

17 . The method of claim 1 , further comprising:
updating the augmented scene based on determining occurrence of another step of the plurality of steps of the surgery.

18 . The method of claim 1 , further comprising:
identifying a relative movement between the display device and the one or more references; and updating a position of the one or more augmentations elements on the display device to track the relative movement.

19 . A non-transitory computer-readable medium having computer-executable instructions stored thereon, which, when executed by one or more processors of a computing device, cause the computing device to perform a method of providing augmented reality assisted surgery, the method comprising:
generating a virtual scene comprising one or more virtual elements comprising one or more anatomical elements corresponding to one or more anatomy parts of a patient; acquiring one or more images of an actual physical scene comprising the one or more anatomy parts; automatically determining, based on the one or more images or another input, a step of a plurality of steps of a surgery the one or more images correspond to; limiting access to a library of augmentation element options during the step of the surgery to only show a subset of augmentation element options of the library of augmentation element options during the step of the surgery, the subset of augmentation element options corresponding to the step of the plurality of steps of the surgery; automatically identifying one or more references in the actual physical scene during the surgery; registering, during the step of the surgery, the virtual scene to the one or more references to generate an augmented scene comprising one or more augmentation elements configured based on one or more augmentation element options selected from the subset of augmentation element options; and displaying, during the surgery, the augmented scene as an overlay on an anatomy of the patient via a display device.

20 . A computing device comprising:
one or more memories; and one or more processors configured to cause the computing device to:
generate a virtual scene comprising one or more virtual elements comprising one or more anatomical elements corresponding to one or more anatomy parts of a patient;
acquire one or more images of an actual physical scene comprising the one or more anatomy parts;
automatically determine, based on the one or more images or another input, a step of a plurality of steps of a surgery the one or more images correspond to;
limit access to a library of augmentation element options during the step of the surgery to only show a subset of augmentation element options of the library of augmentation element options during the step of the surgery, the subset of augmentation element options corresponding to the step of the plu

Assignees

• Mat Nv

Inventors

• Slagmolen Pieter
• Plessers Katrien
• Jonkergouw Filip
• Assche Anneleen Van
• Marien Rosalien
• Gajjar Durva
• Dela Ruelle Chloé
• Daemen Nele
• Wirix-Speetjens Roel
• Heyde Brecht
• Willekens Sanne
• Geebelen Ben

Classifications

• A61B2090/3937

  Visible markers · CPC title

• A61B2034/2055

  Optical tracking systems · CPC title

• A61B2034/252

  indicating steps of a surgical procedure · CPC title

• A61B2034/254

  being adapted depending on the stage of the surgical procedure · CPC title

• A61B2090/502

  Headgear, e.g. helmet, spectacles · CPC title