Wearable heads-up display (“HUD”) for a pharmacy workflow management system

Patent metadata

Field	Value
Publication number	US-12417830-B2
Application number	US-202318123682-A
Country	US
Kind code	B2
Filing date	Mar 20, 2023
Priority date	Mar 18, 2022
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

A wearable heads-up display (“HUD”) for a pharmacy workflow management system is disclosed herein. An example HUD includes smart-glasses that are communicatively coupled to a client device. The smart-glasses include at least one camera and/or barcode scanner to record information needed for the verification of a medication dose during medication formulation preparation. The smart-glasses include at least one microphone to record voice commands. Further, the smart-glasses include at least one embedded display screen that shows sequential steps of a preparation protocol for guiding a pharmacy technician to prepare a medication dose. An application on the client device and/or the smart-glasses is configured to recognize and use voice commands to provide navigation for the embedded display screen. The voice commands may also be used to provide data entry for medication dose preparation verification. Gestures may be detected by the camera and translated by the application into navigation or data entry commands.

First claim

The invention is claimed as follows:

1. A heads-up display (“HUD”) system comprising:
an eyewear device including:
a frame including a right lens, a left lens, a bridge connecting the right lens and the left lens, a first arm connected to the right lens, and a second arm connected to the left lens,
a camera connected to or integrated with the frame,
a microphone and speaker connected to or integrated with the first arm or the second arm, and
a display module including a screen that is positioned in front of or behind the right lens or the left lens with respect to a viewpoint of a wearer; and

a client device communicatively coupled to the eyewear device, the client device including a processor and a memory device storing instructions defining an application, execution of the instructions causing the processor of the client device to:
cause the application to communicatively couple to a pharmacy server storing a plurality of dose orders for medication doses ordered for preparation,
cause the display module of the eyewear device to display a user interface that includes at least a portion of the plurality of dose orders,
receive an input via the microphone or the camera indicative of a dose order for selection,
receive a dose order file from the pharmacy server, the dose order file corresponding to the selected dose order,
receive or retrieve a preparation protocol based on a medication type of the dose order file, the preparation protocol including a sequence of steps for preparing a medication dose for the selected dose order, each step in the sequence of steps associated with graphical step instructions for the wearer, at least one of the steps of the sequence of steps requiring a digital image to be recorded,
for each step of the preparation protocol, display corresponding graphical step instructions on the screen of the eyewear device,
when a step of the preparation protocol requires the recording of a digital image, activate the camera for recording the digital image,
when a step of the preparation protocol requires user entry of verification information, (i) display, in conjunction with graphical step instructions of the step, options for voice entry via the microphone or digital image entry via the camera, and (ii) progress to a next step of the preparation protocol after the verification information is received, and
store to a verification file, an indication of completion progress of the preparation protocol, recorded digital images, and the verification information.

2. The system of claim 1 , wherein the graphical step instructions for each step are displayed adjacent to a graphical indication of required ingredients that specifies a quantity of each ingredient needed and a quantity of each ingredient already obtained.

3. The system of claim 1 , wherein the application is configured to authenticate the wearer with the pharmacy server before having access to the plurality of dose orders.

4. The system of claim 3 , wherein the plurality of dose orders is selected by the pharmacy server among all received dose orders based on the authentication of the wearer.

5. The system of claim 1 , wherein the at least one step of the sequence of steps requires the recording of the digital image of at least one of:
a label or a barcode of an ingredient container;
a preparation container;
a display of a weight scale or balance; or
an administration container.

6. The system of claim 1 , wherein the camera is a first camera module that is located on the bridge, the eyewear device including a second camera module located on the first arm or the second arm.

7. The system of claim 6 , wherein the first camera module includes a high-resolution camera with at least five mega-pixels and the second camera module includes an optical zoom, a barcode scanner, and a laser pointer.

8. The system of claim 6 , wherein a touchpad is included with the second camera module or a housing of the microphone and speaker.

9. The system of claim 6 , wherein the processor of the client device in conjunction with the second camera module causes the second camera module to zoom and crop a digital image around a detected label or a barcode.

10. The system of claim 1 , wherein the graphical step instructions are first graphical step instructions for display at the eyewear device, the preparation protocol including second graphical step instructions that are configured for display on the client device, the second graphical step instructions including additional preparation content that is omitted from the first graphical step instructions.

11. The system of claim 1 , wherein the processor is further configured to:
receive, from the microphone of the eyewear device, an audio signal;
convert the audio signal into text;
determine the text corresponds to a voice command to start a video call with a pharmacist;
use a video conference feature of the client device to connect to a device of the pharmacist; and
activate the camera to transmit recorded video to the device of the pharmacist during a video conference with the pharmacist.

12. A heads-up display (“HUD”) system comprising:
an eyewear device including:
a frame including a right lens, a left lens, a bridge connecting the right lens and the left lens, a first arm connected to the right lens, and a second arm connected to the left lens,
a barcode scanner connected to or integrated with the frame,
a microphone and speaker connected to or integrated with the first arm or the second arm, and
a display module including a screen that is positioned in front of or behind the right lens or the left lens with respect to a viewpoint of a wearer; and

a client device communicatively coupled to the eyewear device, the client device including a processor and a memory device storing instructions defining an application, execution of the instructions causing the processor of the client device to:
cause the application to communicatively couple to a pharmacy server storing a plurality of dose orders for medication doses ordered for preparation,
cause the display module of the eyewear device to display a user interface that includes at least a portion of the plurality of dose orders,
receive an input via the microphone indicative of a dose order for selection,
receive a dose order file from the pharmacy server, the dose order file corresponding to the selected dose order,
receive or retrieve a preparation protocol based on a medication type of the dose order file, the preparation protocol including a sequence of steps for preparing a medication dose for the selected dose order, each step in the sequence of steps associated with graphical step instructions for the wearer, at least one of the steps of the sequence of steps requiring a barcode of an ingredient container to be scanned,
for each step of the preparation protocol, display corresponding graphical step instructions on the screen of the eyewear device,
when a step of the preparation protocol requires the scanning of a barcode, activate the barcode scanner,
when a step of the preparation protocol requires user entry of verification information, (i) display, in conjunction with graphical step instructions of the step, a text box for voice entry via the microphone, and (ii) progress to a next step of the preparation protocol after the verification information is received, and
store to a verification file, an indication of completion progress of the preparation protocol, data from scanned barcodes, and the verification information.

13. The system of claim 12 , wherein when the step of the preparation protocol requires user entry of the verification information

Assignees

• Baxter Int
• Baxter Healthcare Sa

Inventors

• Jacobs Amanda Gloria
• Kramer Gregory Dean
• Brittain Jeffrey Robert
• Olson John
• Kappil Christina

Classifications

• H04R2499/15

  Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops · CPC title

• H04R1/028

  associated with devices performing functions other than acoustics, e.g. electric candles · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title