Extended reality systems for visualizing and controlling operating room equipment

Patent metadata

Field	Value
Publication number	US-12201375-B2
Application number	US-202117476689-A
Country	US
Kind code	B2
Filing date	Sep 16, 2021
Priority date	Sep 16, 2021
Publication date	Jan 21, 2025
Grant date	Jan 21, 2025

Abstract

A camera tracking system receives patient reference tracking information indicating pose of a patient reference array tracked by a patient tracking camera relative to a patient reference frame. A local XR headset view pose transform is determined between a local XR headset reference frame and the patient reference frame. Remote reference tracking information is received indicating pose of a remote reference array tracked by a remote reference tracking camera. A remote XR headset view pose transform is determined between a remote XR headset reference frame of a remote XR headset and the remote reference array. A 3D computer image is transformed from a local pose determined using the local XR headset view pose transform to a remote pose determined using the remote XR headset view pose transform. The transformed 3D computer image is provided to the remote XR headset for display with the remote pose relative to the remote XR headset reference frame.

First claim

What is claimed is:

1. A camera tracking system comprising at least one processor operative to:
receive equipment reference tracking information indicating poses of medical equipments and a patient reference array tracked by a tracking camera relative to a reference frame;
determine an extended reality (XR) headset view pose transform between an XR headset reference frame of an XR headset and the reference frame using the equipment reference tracking information;
obtain operator-gesture tracking information from the tracking camera indicating movement of an object relative to the XR headset reference frame by an operator wearing the XR headset;
select an operational command from among a set of operational commands based on the operator-gesture tracking information; and
provide instructions to one of the medical equipments based on the operational command that is selected wherein the at least one processor is further operative to:
determine a gesture path relative to the XR headset reference frame based on processing the operator-gesture tracking information through the XR headset view pose transform,
select the operational command from among the set of operational commands based on identifying that the gesture path corresponds to a defined gesture associated with the operational command, wherein the operational commands in the set are associated with different shaped gesture paths,
wherein the at least one processor is further operative to:
select the operational command for relocating an end effector connected to a surgical robot arm that is movable under control of a surgical robot system, from among the set of operational commands based on the gesture path corresponding to the defined gesture associated with the operational command for relocating the end effector;
determine a present pose of the end effector based on end effector tracking information indicating pose of the end effector tracked by the tracking camera relative to the reference frame; and,
control movement of the end effector by the surgical robot system from the present pose to a target pose relative to the reference frame based on the operational command for relocating the end effector;
determine a planned end effector trajectory path from the present pose to the target pose based on at least a segment of the gesture path; and
control movement of the end effector by the surgical robot system to conform to the planned end effector trajectory path from the present pose to the target pose.

2. The camera tracking system of claim 1 , wherein:
the tracking camera is part of the XR headset; and
the reference frame is the same as the XR headset reference frame.

3. The camera tracking system of claim 1 , wherein the at least one processor is further operative to:
determine the planned end effector trajectory path based on tracking movement of fingers and/or a hand of the operator wearing the XR headset relative to the end effector.

4. The camera tracking system of claim 1 , wherein the at least one processor is further operative to:
select the operational command for relocating medical imaging equipment in a room under control of a computer system, from among the set of operational commands based on the gesture path corresponding to the defined gesture associated with the operational command for relocating the medical imaging equipment;
determine a present location in the room of the medical imaging equipment relative to the reference frame based on the equipment reference tracking information;
determine a target location in the room for the medical imaging equipment relative to the reference frame based on at least a segment of the gesture path; and
control movement of the medical imaging equipment by the computer system from the present location to the target location based on the operational command for relocating the medical imaging equipment.

5. The camera tracking system of claim 1 , wherein the at least one processor is further operative to:
obtain first graphical and/or textual information from a first one of the medical equipments, and second graphical and/or textual information from a second one of the medical equipments; and
display the first graphical and/or textual information through the XR headset with a pose in the XR headset reference frame defined to be adjacent to the first one of the medical equipments and display the second graphical and/or textual information through the XR headset with another pose in the XR headset reference frame defined to be adjacent to the second one of the medical equipment.

Assignees

• Globus Medical Inc

Inventors

• Robinson Michael
• CALLOWAY Thomas
• Dulin Isaac
• Hussain Mir

Classifications

• G06V40/107

  Static hand or arm · CPC title

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• A61B90/50

  Supports for surgical instruments, e.g. articulated arms · CPC title

• A61B90/361

  Image-producing devices, e.g. surgical cameras · CPC title