Method and system for displaying an augmented reality to an operator of a medical imaging apparatus

Patent metadata

Field	Value
Publication number	US-10055870-B2
Application number	US-201615255191-A
Country	US
Kind code	B2
Filing date	Sep 2, 2016
Priority date	Sep 3, 2015
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

In a method for automatically supporting an operator in working through an execution sequence protocol with a number of sequence steps, with which a person under examination is being examined in a medical imaging facility, the execution sequence protocol to be carried out for the person under examination is determined, and image data are acquired that show a field of vision of at least a part of the environment of the medical imaging facility and the person under examination, as seen by the operator. A next sequence step of the execution sequence protocol is determined that is to be carried out by the operator. Visual information is created that will inform the operator about the operating step to be carried out, and the visual information is project at a viewing facility for displaying an augmented reality to the operator, in which the field of vision is presented augmented by the visual information.

First claim

We claim as our invention:

1. A method for automatically supporting an operator in proceeding through an execution sequence protocol, comprising multiple sequence steps, with which a patient is being examined in a medical imaging apparatus, said method comprising:
in a computer, determining the execution sequence protocol to be implemented for said patient;
with an image data detector, acquiring image data that show a field of vision, as seen by the operator, comprising at least a part of the medical imaging apparatus and the patient;
in said computer, determining a next sequence step of said execution sequence, which is to be implemented by said operator;
in said computer, receiving sensor data from a sensor selected from the group consisting of an acceleration sensor, a magnetic field sensor, and ultrasound sensor, a photo detector, and an RF coil;
in said computer, using said sensor data to determine at least one of a position of the medical imaging apparatus in the image data, an orientation of the medical imaging apparatus in the image data, and a sight position of the operator;
in said computer, generating visual information that informs said operator about said next sequence step; and
from said computer, causing said visual information to be projected at a viewing facility, seen by said operator, by displaying augmented reality to the operator in which said field of vision is shown augmented by said visual information, with said visual information being shown at a location in said field of vision that is selected by said computer dependent on said at least one of said position of the medical imaging apparatus in the image data, said orientation of the medical imaging apparatus in the image data, and said sight position of the operator.

2. A method as claimed in claim 1 comprising, in said computer, determining a position of the patient relative to the medical imaging apparatus automatically from said execution sequence protocol, and projecting said position of the patient relative to the medical imaging apparatus in said visual information in said field of vision in said augmented reality.

3. A method as claimed in claim 1 comprising, in said computer, providing personal information about said patient together with said visual information projected into said field of vision in said augmented reality.

4. A method as claimed in claim 1 comprising:
in said computer, automatically determining, from said execution sequence protocol, additional devices that are needed for implementing said execution sequence protocol, and projecting device information about the additionally needed devices into said field of vision in said augmented reality.

5. A method as claimed in claim 4 comprising:
in said computer, determining, from said image data, which of said additionally needed devices is already situated in said field of vision of said operator;
in said computer, determining any of said additionally needed devices that are not present in said field of vision of said operator; and
generating said device information to inform said operator about which of said additionally needed devices are not in said field of vision of said operator.

6. A method as claimed in claim 1 comprising, in said computer, checking said image data to determine whether said image data represent an operating error by detecting a situation for said medical imaging apparatus or said patient that does not match said execution sequence protocol, and projecting error information, describing said operating error, into said field of vision in said augmented reality.

7. A method as claimed in claim 1 comprising converting a description of said next operating step into audio information, and playing said audio information to said operator.

8. A method as claimed in claim 1 comprising employing a magnetic resonance apparatus as said medical imaging apparatus.

9. A method as claimed in claim 8 comprising determining, as said next sequence step to be implemented, positioning of a radio-frequency coil on said patient, and including, in said visual information, a designation of a position to be occupied by said RF coil relative to the magnetic resonance apparatus and the patient.

10. A method as claimed in claim 1 comprising, in said computer, determining a current status of said medical imaging apparatus and generating status information designating said current status, and projecting said status information into said field of vision with said visual information in said augmented reality.

11. A method as claimed in claim 1 comprising, in said computer, determining a location in the image data, relative to at least one other object in said image data, at which the visual information is arranged in the image data, and projecting said visual information at said location into said field of vision in said augmented reality.

12. A system for automatically supporting an operator in proceeding through an execution sequence protocol, comprising multiple sequence steps, with which a patient is being examined in a medical imaging apparatus, said comprising:
a computer configured to determine the execution sequence protocol to be implemented for said patient;
an image data detector that acquires image data that show a field of vision, as seen by the operator comprising at least a part of the medical imaging apparatus and of the patient;
said computer being configured to determine a next sequence step of said execution sequence, which is to be implemented by said operator;
a sensor in communication with said computer, said sensor being selected from the group consisting of an acceleration sensor, a magnetic field sensor, and ultrasound sensor, a photo detector, and an RF coil;
said computer being configured to receive said sensor data and us said sensor data to determine at least one of the position of the medical imaging apparatus in the image data, the orientation of the medical imaging apparatus in the image data, and the sight position of the operator;
said computer being configured to generate visual information that informs said operator about said next sequence step; and
a viewing facility in communication with said computer, said computer being configured to cause said visual information to be projected at said viewing facility, seen by said operator, by displaying augmented reality to the operator in which said field of vision is shown augmented by said visual information, with said visual information being shown at a location in said field of vision that is selected by said computer dependent on said at least one of said position of the medical imaging apparatus in the image data, said orientation of the medical imaging apparatus in the image data, and said sight position of the operator.

13. A system as claimed in claim 12 comprising wherein said computer is configured to determine a position of the patient relative to the medical imaging apparatus automatically from said execution sequence protocol, and projecting said position of the patient relative to the medical imaging apparatus in said visual information in said field of vision in said augmented reality.

14. A system as claimed in claim 12 wherein said computer being configured to provide personal information about said patient together with said visual information projected into said field of vision in said augmented reality.

15. A system as claimed in claim 12 comprising:
said computer is configured to automatically determine, from said execution sequence protocol, additional devices that are needed for implementing said execution sequence protocol, and to project device information

Assignees

• Siemens Healthcare Gmbh

Inventors

• Benner Thomas
• Potthast Andreas

Classifications

• A61B5/743

  Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots · CPC title

• A61B5/704

  Tables · CPC title

• G16H40/63

  for local operation · CPC title

• A61B8/00

  Diagnosis using ultrasonic, sonic or infrasonic waves · CPC title

• A61B5/0035

  adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography · CPC title