Automated instrument or component assistance using mixed reality in orthopedic surgical procedures

Patent metadata

Field	Value
Publication number	US-12266440-B2
Application number	US-202017117730-A
Country	US
Kind code	B2
Filing date	Dec 10, 2020
Priority date	Jun 19, 2018
Publication date	Apr 1, 2025
Grant date	Apr 1, 2025

Abstract

An example medical device system includes a plurality of surgical items; and a visualization device configured to present a mixed reality (MR) presentation to a user; and one or more processors configured to select a surgical item of the plurality of surgical items based on a surgical procedure and present, in the MR presentation, virtual information that identifies the selected surgical item among the plurality of surgical items, wherein the virtual information is presented on or adjacent a position of the selected surgical item visible via the visualization device.

First claim

What is claimed is:

1. A medical device system comprising:
a plurality of surgical items associated with an orthopedic surgical procedure, wherein the plurality of surgical items is identified in a surgical plan for a patient and wherein the surgical plan for the patient defines a sequence for using the plurality of surgical items over a course of the orthopedic surgical procedure; and
a visualization device configured to present a mixed reality (MR) presentation to a user; and
one or more processors configured to select a surgical item of the plurality of surgical items based on the orthopedic surgical procedure and present, in the MR presentation, virtual information that identifies the selected surgical item among the plurality of surgical items, wherein the virtual information is presented on or adjacent a position of the selected surgical item visible via the visualization device, and wherein the virtual information includes virtual elements that identify all of the surgical items in the plurality of surgical items while distinguishing a particular one of the surgical items corresponding to the selected surgical item, wherein the virtual information includes a color coordination scheme that distinguishes a first subset of the surgical items, a second subset of the surgical items, and a third subset of the surgical items, wherein the first subset of the surgical items corresponds to already used surgical items within the sequence defined in the surgical plan, the second subset of the surgical items corresponds to a currently used surgical item that is the selected surgical item within the sequence defined in the surgical plan, and the third subset of the surgical items corresponds to subsequently needed surgical items within the sequence defined in the surgical plan.

2. The medical device system of claim 1 , wherein the virtual elements identify two or more different surgical items with different colors.

3. The medical device system of claim 1 , wherein a different light color of one of the virtual elements distinguishes the particular one of the surgical items.

4. The medical device system of claim 1 , wherein a lighting effect of one of the virtual elements distinguishes the particular one of the surgical items.

5. The medical device system of claim 1 , wherein a lighting intensity of one of the virtual elements distinguishes the particular one of the surgical items.

6. The medical device system of claim 1 , wherein the plurality of surgical items comprises sounders for the orthopedic surgical procedure and the particular one of the surgical items comprises a current sounder to be used.

7. The medical device system of claim 1 , wherein the plurality of surgical items comprises sounders for the orthopedic surgical procedure and one or more punching tools for the orthopedic surgical procedure.

8. The medical device system of claim 1 , wherein the plurality of surgical items comprises sounders for an orthopedic shoulder repair surgery, one or more punching tools for the orthopedic surgical procedure, and one or more compacting tools for the orthopedic surgical procedure.

9. The medical device system of claim 1 , wherein the plurality of surgical items comprises sounders for the orthopedic surgical procedure, one or more punching tools for the orthopedic surgical procedure, one or more compacting tools for the orthopedic surgical procedure, and one or more planing tools for the orthopedic surgical procedure.

10. The medical device system of claim 1 , wherein the one or more processors are configured to select, based on a detected change in the orthopedic surgical procedure, a next surgical item, wherein the detected change causes a change in the sequence defined in the surgical plan.

11. The medical device system of claim 10 , wherein the one or more processors are configured to select, based on the detected change in the orthopedic surgical procedure and based on machine learning, the next surgical item.

12. The medical device system of claim 1 , wherein the virtual information is first virtual information and the one or more processors are configured to select, based on detection of the currently used surgical item, a next surgical item and present second virtual information that identifies the next surgical item.

13. A visualization device comprising:
a holographic screen comprising a holographic lens configured to present a mixed reality (MR) presentation to a user; and
one or more processors configured to:
select a surgical item of a plurality of surgical items, wherein the plurality of surgical items is associated with an orthopedic surgical procedure, wherein the plurality of surgical items is identified in a surgical plan for a patient, and wherein the surgical plan for the patient defines a sequence for using the plurality of surgical items over a course of the orthopedic surgical procedure, and
present, in the MR presentation, virtual information that identifies the selected surgical item among the plurality of surgical items, wherein the virtual information is presented on or adjacent a position of the selected surgical item visible via the visualization device and wherein the virtual information includes virtual elements that identify all of the surgical items in the plurality of surgical items while distinguishing a particular one of the surgical items corresponding to the selected surgical item, wherein the virtual information includes a color coordination scheme that distinguishes a first subset of the surgical items, a second subset of the surgical items, and a third subset of the surgical items, wherein the first subset of the surgical items corresponds to already used surgical items within the sequence defined in the surgical plan, the second subset of the surgical items corresponds a currently used surgical item that is the selected surgical item within the sequence defined in the surgical plan, and the third subset of the surgical items corresponds to subsequently needed surgical items within the sequence defined in the surgical plan.

14. The visualization device of claim 13 , wherein the virtual elements identify at least two different surgical items of the plurality of surgical items with different colors.

15. The visualization device of claim 13 , wherein a different light color of one of the virtual elements distinguishes the particular one of the surgical items.

16. The visualization device of claim 13 , wherein a lighting effect of one of the virtual elements distinguishes the particular one of the surgical items.

17. The visualization device of claim 13 , wherein a lighting intensity of one of the virtual elements distinguishes the particular one of the surgical items.

18. The visualization device of claim 13 , wherein the plurality of surgical items comprises sounders for the orthopedic surgical procedure and the particular one of the surgical items comprises a current sounder to be used.

19. The visualization device of claim 13 , wherein the plurality of surgical items comprises sounders for the orthopedic surgical procedure and one or more punching tools for the orthopedic surgical procedure.

20. The visualization device of claim 13 , wherein the plurality of surgical items comprises sounders for the orthopedic surgical procedure, one or more punching tools for an orthopedic shoulder repair surgery, and one or more compacting tools for the orthopedic surgical procedure.

21. The visualization device of claim 13 , wherein the plurality of surgical items comprises sounders for t

Assignees

• Howmedica Osteonics Corp

Inventors

• Poltaretskyi Sergii
• Cariou Damien
• Morvan Yannick
• Gaborit Vincent
• Dassonville Benjamin
• Chaoui Jean

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F18/2163

  Partitioning the feature space · CPC title

• A61B34/76

  Manipulators having means for providing feel, e.g. force or tactile feedback · CPC title

• G06F30/20

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title