Convolutional neural network for identification of anatomical landmark

Patent metadata

Field	Value
Publication number	US-12573192-B2
Application number	US-202318514522-A
Country	US
Kind code	B2
Filing date	Nov 20, 2023
Priority date	Jun 20, 2016
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A method includes: obtaining an ultrasound image of an anatomical area from an ultrasound imaging device; inputting the ultrasound image into a first stage of a convolutional neural network, the first stage configured to determine key-point locations of the anatomical area; generating, for each of the key-point locations, a cropped region of the ultrasound image; inputting each of the cropped regions into a second stage of the convolutional neural network, the second stage configured to locate an anatomical landmark of the anatomical area; and outputting a location of the anatomical landmark.

First claim

What is claimed is:

1 . A method, comprising:
obtaining an ultrasound image of an anatomical area from an ultrasound imaging device; inputting the ultrasound image into a first stage of a convolutional neural network, the first stage configured to determine key-point locations of the anatomical area; generating, for each of the key-point locations, a cropped region of the ultrasound image; inputting each of the cropped regions into a second stage of the convolutional neural network, the second stage configured to locate an anatomical landmark of the anatomical area; and outputting a location of the anatomical landmark.

2 . The method of claim 1 ,
wherein the first stage of the neural network includes a layer that reduces an original resolution of the ultrasound image, and wherein the cropped regions are generated with the original resolution of the ultrasound image.

3 . The method of claim 1 ,
wherein a kernel size of the first stage is smaller than a kernel size of the second stage.

4 . The method of claim 1 ,
wherein a weight vector of the second stage is combined with a weight vector of the first stage to determine the location of the anatomical landmark.

5 . The method of claim 1 , further comprising:
comparing the location of the anatomical landmark to a target anatomical view; and generating a guidance plan to reach the target anatomical view based on the location of the anatomical landmark in the ultrasound image.

6 . The method of claim 1 , further comprising:
locating a second anatomical landmark by using the convolutional neural network; and determining an anatomical measurement based on the anatomical landmark and the second anatomical landmark.

7 . The method of claim 6 , further comprising:
identifying a medical parameter based on the anatomical measurement; and displaying the ultrasound image with an overlay including the medical parameter.

8 . The method of claim 7 ,
wherein the medical parameter is an ejection fraction, and wherein the key-points locations are based on a parasternal long axis view of a left ventricle.

9 . An apparatus, comprising:
a processor communicatively coupled to an ultrasound imaging device, wherein the processor is configured to: obtain an ultrasound image of an anatomical area from the ultrasound imaging device; input the ultrasound image into a first stage of a convolutional neural network, the first stage configured to determine key-point locations of the anatomical area; generate, for each of the key-point locations, a cropped region of the ultrasound image; input each of the cropped regions into a second stage of the convolutional neural network, the second stage configured to locate an anatomical landmark of the anatomical area; and output a location of the anatomical landmark.

10 . The apparatus of claim 9 ,
wherein the first stage of the neural network includes a layer that reduces an original resolution of the ultrasound image, and wherein the processor is configured to generate the cropped regions with the original resolution of the ultrasound image.

11 . The apparatus of claim 9 ,
wherein the processor is further configured to:
compare the location of the anatomical landmark to a target anatomical view; and
generate a guidance plan to reach the target anatomical view based on the location of the anatomical landmark in the ultrasound image.

12 . The apparatus of claim 9 ,
wherein the processor is further configured to:
locate a second anatomical landmark by using the convolutional neural network; and
determine an anatomical measurement based on the anatomical landmark and the second anatomical landmark.

13 . The apparatus of claim 12 ,
wherein the processor is further configured to:
identify a medical parameter based on the anatomical measurement; and
output the ultrasound image with an overlay including the medical parameter.

14 . A system, comprising:
an ultrasound imaging device configured to capture an ultrasound image; and a computing device communicatively coupled to the ultrasound imaging device, wherein the computing device is configured to: obtain an ultrasound image of an anatomical area from the ultrasound imaging device; input the ultrasound image into a first stage of a convolutional neural network, the first stage configured to determine key-point locations of the anatomical area; generate, for each of the key-point locations, a cropped region of the ultrasound image; input each of the cropped regions into a second stage of the convolutional neural network, the second stage configured to locate an anatomical landmark of the anatomical area; and output a location of the anatomical landmark.

15 . The system of claim 14 ,
wherein the first stage of the neural network includes a layer that reduces an original resolution of the ultrasound image, and wherein the computing device is configured to generate the cropped regions with the original resolution of the ultrasound image.

16 . The system of claim 14 ,
wherein the computing device is further configured to:
compare the location of the anatomical landmark to a target anatomical view; and
generating a guidance plan to reach the target anatomical view based on the location of the anatomical landmark in the ultrasound image.

17 . The system of claim 14 ,
wherein the computing device is further configured to:
locate a second anatomical landmark by using the convolutional neural network; and
determine an anatomical measurement based on the anatomical landmark and the second anatomical landmark.

18 . The system of claim 17 ,
wherein the computing device is further configured to:
identify a medical parameter based on the anatomical measurement; and
output the ultrasound image with an overlay including the medical parameter.

19 . A non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by a processor, cause the processor to:
obtain an ultrasound image of an anatomical area from an ultrasound imaging device; input the ultrasound image into a first stage of a convolutional neural network, the first stage configured to determine key-point locations of the anatomical area; generate, for each of the key-point locations, a cropped region of the ultrasound image; input each of the cropped regions into a second stage of the convolutional neural network, the second stage configured to locate an anatomical landmark of the anatomical area; and output a location of the anatomical landmark.

Assignees

• Bfly Operations Inc

Inventors

• De Jonge Matthew
• Schneider Robert
• Elgena David
• Rothberg Alex
• Rothberg Jonathan M
• Sofka Michal
• Gafner Tomer
• Thiele Karl
• Neben Abraham

Classifications

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T2210/41

  Medical · CPC title

• G06T2207/30061

  Lung · CPC title