Registration of surgical tool with reference array tracked by cameras of an extended reality headset for assisted navigation during surgery

Patent metadata

Field	Value
Publication number	US-11607277-B2
Application number	US-202016861741-A
Country	US
Kind code	B2
Filing date	Apr 29, 2020
Priority date	Apr 29, 2020
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

A camera tracking system is disclosed for computer assisted navigation during surgery. The camera tracking system is configured to identify a reference array tracked by a set of tracking cameras attached to an extended reality (XR) headset, and determine whether the reference array is registered as being paired with characteristics of one of a plurality of surgical tools defined in a surgical tool database. The camera tracking system is further configured to, based on the reference array being determined to not be registered and receiving user input, register the reference array to be paired with characteristics of one of the plurality of surgical tools selected based on the user input. The camera tracking system is further configured to provide a representation of the characteristics to a display device of the XR headset for display to the user.

First claim

What is claimed is:

1. A camera tracking system for computer assisted navigation during surgery, the camera tracking system configured to:
automatically initiate a registration process when a reference array of a surgical tool is brought into in a field of view of a set of tracking cameras attached to an extended reality (XR) headset;
automatically identify the reference array of the surgical tool for tracking by the set of tracking cameras attached to the extended reality (XR) headset adapted to be worn by a user, the set of tracking cameras of the XR headset adapted for tracking the surgical tools and supplementing a main navigation system having a plurality of cameras for tracking the same surgical tools and an anatomical structure of a patient;
automatically determine whether the identified reference array captured by the tracking cameras of the XR headset is registered as being paired with characteristics of one of the surgical tools defined in a surgical tool database;
based on the reference array being determined to not be registered, provide to the display device a list of at least some of the plurality of the surgical tools defined in the surgical tool database for the user to select the one of the surgical tools to be registered as paired with the reference array;
based on the reference array being determined to not be registered and receiving user input, register the reference array as paired with characteristics of one of the surgical tools selected based on the user input, wherein the user input is in the form of one or more hand gestures recognizable by the set of tracking cameras of the XR headset; and
provide a representation of the characteristics to a display device of the XR headset for display to the user.

2. The camera tracking system of claim 1 , further configured to:
determine the user's selection among the displayed list of the one of the surgical tools to be registered as paired with the reference array, based on tracking information from the set of tracking cameras indicating pose of a hand of the user in XR space relative to the displayed list.

3. The camera tracking system of claim 1 , further configured to:
receive tracking information from the set of tracking cameras indicating pose of the reference array relative to the XR headset;
determine a pose for where the representation of the characteristics is to be displayed relative to the reference array based on the tracking information; and
control the XR headset to display the representation of the characteristics with the determined pose relative to the reference array.

4. The camera tracking system of claim 1 , further configured to:
obtain tracking accuracy information characterizing accuracy at which the set of tracking cameras is presently tracking pose of the reference array; and
provide an indication of the tracking accuracy information to the display device of the XR headset for display to the user.

5. The camera tracking system of claim 1 , further configured to:
obtain a graphical representation of the one of the plurality of surgical tools in at least 2-dimensions;
receive tracking information from the set of tracking cameras indicating pose of the reference array relative to the XR headset;
determine a pose for the graphical representation based on the tracking information; and provide the graphical representation as the representation of the characteristics to the display device of the XR headset for display with the determined pose.

6. The camera tracking system of claim 1 , further configured to:
obtain a last verified date which indicates when the one of the surgical tools was last verified to not have structural deformation; and
provide an indication of the last verified date to the display device of the XR headset for display to the user.

7. The camera tracking system of claim 1 , further configured to:
determine when the reference array has been brought within an inspection region which is defined relative to location of the XR headset;
responsive to determining that the reference array has been brought within the inspection region, initiate the determination of whether the reference array is registered as being paired with characteristics of one of a plurality of surgical tools defined in the surgical tool database; and
limit operation to receive user input to register the reference array as paired with characteristics of one of the plurality of surgical tools defined in the surgical tool database, to occur only while the reference array remains within the inspection region.

8. The camera tracking system of claim 1 , further configured to:
determine when the reference array has been brought within an inspection region which is defined relative to location of the XR headset;
responsive to determining that the reference array has been brought within the inspection region, initiate the providing of the representation of the characteristics to the display device of the XR headset for display to the user; and
responsive to determining that the reference array has exited the inspection region, cease any ongoing providing of the representation of the characteristics to the display device of the XR headset for display to the user.

9. The camera tracking system of claim 1 , further configured to:
identify poses of a plurality of fiducials among the reference array; and
for each of the plurality of fiducials among the reference array having the identified poses, providing a graphical indicia for display by the display device at least partially overlapped with the identified pose of the fiducial.

10. The camera tracking system of claim 9 , further configured to:
when one of the fiducials in the reference array is a missing fiducial not having an identifiable pose, estimating a pose of the missing fiducial relative to the plurality of fiducials having the identified poses, and providing another graphical indicia for display by the display device at least partially overlapped with the estimated pose of the missing fiducial.

11. The camera tracking system of claim 1 , further configured to:
receive tracking information from the set of tracking cameras indicating pose of the reference array relative to the XR headset and pose of a hand of the user;
determine a hand-user-interface exclusion zone relative to the pose of the reference array based on the tracking information;
enable receipt of hand gesture input from the user based on the tracked pose of the hand indicated by the tracking information while the user's hand is determined to be entirely outside the hand-user-interface exclusion zone; and
disable receipt of hand gesture input from the user based on the tracked pose of the hand indicated by the tracking information while the user's hand is determined to be at least partially within the hand-user-interface exclusion zone.

12. A computer program product comprising a non-transitory computer readable medium storing program code executable by at least one processor of a camera tracking system for computer assisted navigation surgery to:
automatically initiate a registration process when a reference array of a surgical tool is brought into in a field of view of a set of tracking cameras attached to an extended reality (XR) headset;
automatically identify the reference array of the surgical tool for tracking by the set of tracking cameras attached to the extended reality (XR) headset adapted to be worn by a user when the reference array is brought into a field of view of the set of tracking cameras, the set of tracking cameras of the XR headset adapted for tracking the surgical tools and supplementing a main navigation system having a plurality of cam

Assignees

• Globus Medical Inc

Inventors

• CALLOWAY Thomas
• Dulin Isaac

Classifications

• A61B2034/2072

  Reference field transducer attached to an instrument or patient · CPC title

• A61B2090/3762

  using computed tomography systems [CT] · CPC title

• A61B2090/372

  Details of monitor hardware · CPC title

• A61B34/20 Primary

  Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis · CPC title

• A61B2090/3983

  Reference marker arrangements for use with image guided surgery · CPC title