Augmented reality device for providing feedback to an acute care provider

Patent metadata

Field	Value
Publication number	US-12544179-B2
Application number	US-202318378701-A
Country	US
Kind code	B2
Filing date	Oct 11, 2023
Priority date	Mar 15, 2018
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

A spatially sensitive augmented reality system for providing resuscitative feedback in a mixed reality environment to an acute care provider during occurrence of a cardiac event in a patient includes a wearable augmented reality device having at least one three-dimensional sensor, a visual display for viewing by the acute care provider, and at least one processor. The at least one processor is configured to receive and process three-dimensional information of a scene of the cardiac event; produce a three-dimensional representation of a field of view of the acute care provider based on the processed three-dimensional information; identify one or more physical objects associated with the cardiac event in the three-dimensional representation; generate at least one virtual three-dimensional object within the three-dimensional representation; and generate an image of the virtual three-dimensional object within the three-dimensional representation on the visual display.

First claim

What is claimed is:

1 . A spatially sensitive augmented reality system for providing resuscitative information in a mixed reality environment to an acute care provider for treatment of a patient during an acute care event, the system comprising:
a wearable augmented reality device having at least one three-dimensional sensor, a visual display for viewing by the acute care provider, and at least one processor configured to:
receive and process three-dimensional information of a rescue scene of the acute care event to generate a three-dimensional representation of a field of view of the acute care provider;
analyze the three-dimensional representation to identify boundaries of one or more physical objects in the three-dimensional representation;
apply one or more spatially sensitive rules according to the identified boundaries of the one or more physical objects; and
cause the augmented reality device worn by the acute care provider to generate at least one virtual three-dimensional object within the three-dimensional representation of the rescue scene and display at least one image of the at least one virtual three-dimensional object on the visual display within the acute care provider's field of view and outside of the identified boundaries to provide emergency resuscitative guidance for the acute care provider, such that the at least one image of the at least one virtual three-dimensional object abides by the one or more spatially sensitive rules and is prevented from traversing the one or more identified boundaries.

2 . The spatially sensitive augmented reality system of claim 1 , wherein the one or more physical objects comprise the patient and the acute care provider, and
wherein the at least one processor is configured to apply the one or more spatially sensitive rules according to the identified boundaries of the patient and the acute care provider in order to prevent the at least one virtual three-dimensional object from traversing the identified boundaries of the patient and the acute care provider.

3 . The spatially sensitive augmented reality system of claim 1 , wherein the at least one virtual three-dimensional object comprises a dashboard display comprising at least one of physiological information for the patient, status information for the acute care provider, or information about the rescue scene.

4 . The spatially sensitive augmented reality system of claim 3 , wherein the dashboard display comprises a real-time video of structures of a cardiothoracic cavity of the patient obtained from an implanted or indwelling camera.

5 . The spatially sensitive augmented reality system of claim 3 , wherein information of the dashboard display is based at least in part on analysis of two dimensional images captured by at least one camera of the augmented reality device.

6 . The spatially sensitive augmented reality system of claim 1 , wherein the acute care event comprises an intubation procedure of the patient, and wherein the one or more physical objects comprise the patient and at least a portion of an endotracheal tube.

7 . The spatially sensitive augmented reality system of claim 6 , wherein the at least one processor is further configured to determine positional information about the endotracheal tube based on the analysis of the three-dimensional representation.

8 . The spatially sensitive augmented reality system of claim 7 , wherein the emergency resuscitation guidance for the acute care provider comprises visual feedback to guide the acute care provider in positioning of the endotracheal tube based, at least in part, on the determined positional information.

9 . The spatially sensitive augmented reality system of claim 6 , wherein the generated at least one virtual three-dimensional object comprises at least one virtual image of an indwelling portion of the endotracheal tube displayed on the visual display according to the one or more spatially sensitive rules and positioned relative to the patient.

10 . The spatially sensitive augmented reality system of claim 1 , wherein the at least one processor is configured to receive an instruction to reposition the at least one virtual three-dimensional object and, based on the received instruction, reposition the at least one virtual three-dimensional object within the three-dimensional representation of the rescue scene so that the image of the at least one virtual three-dimensional object is displayed, within the acute care provider's field of view, at a new location.

11 . The spatially sensitive augmented reality system of claim 10 , wherein receiving the instruction comprises analyzing the three-dimensional information to identify a gesture performed by the acute care provider and generate the instruction to reposition the at least one virtual three-dimensional object based on the instruction.

12 . The spatially sensitive augmented reality system of claim 11 , wherein the gesture comprises one or more of pointing to a physical or virtual object within the field of view, mimicking grasping the physical or virtual object within the field of view, or mimicking dragging the physical object or virtual object within the field of view.

13 . The spatially sensitive augmented reality system of claim 1 , wherein the at least one processor analyzes the three-dimensional representation to identify the boundaries of one or more physical objects by one or more of color matching to identify portions of the three-dimensional representation having a similar color and shape matching to identify portions of the three dimensional representation having a shape corresponding to a physical object commonly found at the rescue scene.

14 . The spatially sensitive augmented reality system of claim 13 , wherein the physical object commonly found at a rescue scene comprises at least one of: a patient, an acute care provider, a medical device, a disposable medical instrument, an endotracheal tube, a laparoscope, a vehicle or patient transportation device, or an environmental hazard.

15 . The spatially sensitive augmented reality system of claim 1 , further comprising an image sensor configured to acquire two-dimensional images of the rescue scene, and wherein the at least one processor is configured to identify boundaries of physical objects in the three-dimensional representation of the rescue scene based on analysis of the acquired two-dimensional images.

16 . The spatially sensitive augmented reality system of claim 15 , wherein the at least one processor is configured to refine boundaries of the identified physical objects within the three-dimensional representation of the rescue scene based on analysis of the acquired two-dimensional images.

17 . The spatially sensitive augmented reality system of claim 16 , wherein refining boundaries of the identified physical objects comprises, based on analysis of the two-dimensional images, identifying the physical object in the two-dimensional image and extracting an outline of the physical object by a color matching technique.

18 . A spatially sensitive augmented reality system for guiding an acute care provider in positioning of an endotracheal tube, the system comprising:
a wearable augmented reality device comprising at least one three-dimensional sensor configured to capture images corresponding to a field of view of the acute care provider, a visual display for viewing by the acute care provider, and at least one processor configured to:
receive and process from the at least one three-dimensional sensor three-dimensional information of a scene comprising

Assignees

• Zoll Medical Corp

Inventors

• Freeman Gary A
• Johnson Guy R
• Carroll Justin R
• Bonner Brett B

Classifications

• A61B2034/254

  being adapted depending on the stage of the surgical procedure · CPC title

• A61B2034/256

  having a database of accessory information, e.g. including context sensitive help or scientific articles · CPC title

• A61H2201/5012

  using the internet · CPC title

• A61N1/39044

  in combination with cardiopulmonary resuscitation [CPR] therapy · CPC title

• A61N1/3993

  User interfaces for automatic external defibrillators · CPC title