Automated image acquisition for assisting a user to operate an ultrasound device

Patent metadata

Field	Value
Publication number	US-10993697-B2
Application number	US-201715626423-A
Country	US
Kind code	B2
Filing date	Jun 19, 2017
Priority date	Jun 20, 2016
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

Aspects of the technology described herein relate to techniques for guiding an operator to use an ultrasound device. Thereby, operators with little or no experience operating ultrasound devices may capture medically relevant ultrasound images and/or interpret the contents of the obtained ultrasound images. For example, some of the techniques disclosed herein may be used to identify a particular anatomical view of a subject to image with an ultrasound device, guide an operator of the ultrasound device to capture an ultrasound image of the subject that contains the particular anatomical view, and/or analyze the captured ultrasound image to identify medical information about the subject.

First claim

What is claimed is:

1. An ultrasound guidance apparatus, comprising:
at least one processor configured to guide an operator of an ultrasound device to capture, with the ultrasound device, a target ultrasound image containing a target anatomical view of a subject, the guiding comprising:
generating, using an ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view, at least one instruction for how to position the ultrasound device; and
providing the operator with the at least one instruction for how to position the ultrasound device through an augmented reality (AR) interface in which the at least one instruction is overlaid on an image of the subject.

2. The ultrasound guidance apparatus of claim 1 , wherein the at least one processor is configured to generate a guidance plan for how to guide the operator to capture the target ultrasound image containing the target anatomical view.

3. The ultrasound guidance apparatus of claim 2 , wherein the at least one processor is configured to provide the at least one instruction to the operator based on the guidance plan.

4. The ultrasound guidance apparatus of claim 3 , wherein the at least one processor is configured to guide the operator to capture the target ultrasound image at least in part by identifying an anatomical view contained in the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view using a deep learning technique.

5. The ultrasound guidance apparatus of claim 4 , wherein the at least one processor is configured to guide the operator to capture the target ultrasound image at least in part by identifying, using the anatomical view, a direction in which to move the ultrasound device.

6. The ultrasound guidance apparatus of claim 3 , wherein the at least one processor is configured to guide the operator to capture the target ultrasound image at least in part by determining whether the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view contains an anatomical view of the subject within a target region of the subject.

7. The ultrasound guidance apparatus of claim 6 , wherein the at least one processor is configured to provide the at least one instruction to the operator at least in part by providing an instruction to the operator to move the ultrasound device toward a position at which the ultrasound device can obtain images of views within the target region of the subject responsive to a determination that the anatomical view contained in the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view is outside the target region.

8. The ultrasound guidance apparatus of claim 6 , wherein the at least one processor is configured to provide the at least one instruction to the operator at least in part by providing an instruction to the operator to move the ultrasound device toward a position at which the ultrasound device can obtain an image of the target anatomical view responsive to a determination that the anatomical view contained in the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view is within the target region.

9. The ultrasound guidance apparatus of claim 1 , further comprising a display coupled to the at least one processor and configured to display the at least one instruction to the operator.

10. The ultrasound guidance apparatus of claim 9 , wherein the display and the at least one processor are integrated into a computing device.

11. A system, comprising:
an ultrasound device; and
at least one processor configured to guide an operator of the ultrasound device to capture, with the ultrasound device, a target ultrasound image containing a target anatomical view of a subject, the guiding comprising:
generating, using an ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view, at least one instruction for how to position the ultrasound device; and
providing the operator with the at least one instruction for how to position the ultrasound device through an augmented reality (AR) interface in which the at least one instruction is overlaid on an image of the subject.

12. The system of claim 11 , wherein the ultrasound device comprises an ultrasonic transducer selected from the group consisting of: a capacitive micromachined ultrasonic transducer (CMUT), a CMOS ultrasonic transducer (CUT), and a piezoelectric micromachined ultrasonic transducer (PMUT).

13. The system of claim 11 , wherein the at least one processor is integrated into a mobile smartphone or a tablet.

14. The system of claim 11 , wherein the at least one processor is configured to guide the operator to capture the target ultrasound image at least in part by:
determining whether the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view contains the target anatomical view;
responsive to determining that the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view does not contain the target anatomical view,
generating, using the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view, a guidance plan for how to guide the operator of the ultrasound device; and
providing the at least one instruction to the operator based on the guidance plan.

15. The system of claim 14 , wherein the guidance plan comprises a sequence of instructions to guide the operator of the ultrasound device to move the ultrasound device to a target location.

16. The system of claim 15 , wherein each instruction in the sequence of instructions is an instruction to move the ultrasound device in a translational or rotational direction.

17. The system of claim 14 , wherein the at least one processor is configured to generate the guidance plan at least in part by determining whether the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view contains an anatomical view of the subject within a target region of the subject.

18. The system of claim 17 , wherein the at least one processor is configured to provide the at least one instruction to the operator at least in part by providing an instruction to the operator to move the ultrasound device toward a position at which the ultrasound device can obtain images of views within the target region of the subject responsive to a determination that the anatomical view contained in the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view is not within the target region.

19. The system of claim 17 , wherein the at least one processor is configured to provide the at least one instruction to the operator at least in part by providing an instruction to the operator to move the ultrasound device toward a position at which the ultrasound device can obtain an image of the target anatomical view responsive to a determination that the anatomical view contained in the ultrasound image currently obtained with the ultrasound device and not containing the target anatomical view is within the target region.

20. A method, comprising:
using at least one computing device comprising at least one processor to perform:
obtaining an ultrasound image of a subject currently obtained with an ultrasound device and not containing a tar

Assignees

• Butterfly Network Inc

Inventors

• Nouri Daniel
• Rothberg Alex
• Rothberg Jonathan M
• De Jonge Matthew
• Jia Jimmy
• Sofka Michal
• Neben Abraham

Classifications

• A61B8/46 Primary

  Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06F18/24133 Primary

  Distances to prototypes · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title