System and methods for tracking anatomical features in ultrasound images

Patent metadata

Field	Value
Publication number	US-11583244-B2
Application number	US-201916593854-A
Country	US
Kind code	B2
Filing date	Oct 4, 2019
Priority date	Oct 4, 2019
Publication date	Feb 21, 2023
Grant date	Feb 21, 2023

Abstract

Methods and systems are provided for tracking anatomical features across multiple images. One example method includes outputting, for display on a display device, an annotation indicative of a first location of an identified anatomical feature of a first ultrasound image, the annotation generated based on a first output of a model and outputting, for display on the display device, an adjusted annotation based on a second output of the model, the second output of the model generated based on a second ultrasound image and further based on the first output of the model, the adjusted annotation indicative of a second location of the identified anatomical feature in the second ultrasound image.

First claim

The invention claimed is:

1. A method, comprising:
outputting, for display on a display device, an annotation indicative of a first location of an identified anatomical feature of a first ultrasound image, the annotation generated based on a first output of a model; and
outputting, for display on the display device, an adjusted annotation based on a second output of the model, the second output of the model generated by the model based on input including a second ultrasound image and the first output of the model, the adjusted annotation indicative of a second location of the identified anatomical feature in the second ultrasound image, wherein the second ultrasound image and the first output are entered at different layers of the model.

2. The method of claim 1 , wherein the adjusted annotation is adjusted relative to the annotation in one or more of location, size, shape, and visual appearance.

3. The method of claim 1 , wherein the adjusted annotation is adjusted relative to the annotation in a location of the annotation to reflect the identified anatomical feature moving from the first location to the second location, and further comprising if the second location is at least partially out of a field of view of an ultrasound probe used to acquire the second ultrasound image, also adjusting a visual appearance of the annotation.

4. The method of claim 1 , wherein the first output of the model is generated using the first ultrasound image as input to the model, wherein the second ultrasound image is acquired after the first ultrasound image, and wherein the first output includes a probability, for each pixel of the second ultrasound image, that the pixel is part of the identified anatomical feature.

5. The method of claim 1 , wherein the second ultrasound image and the first output are entered at a common input layer of the model and at a common resolution.

6. The method of claim 1 , wherein the first output is entered at a lower resolution than the second ultrasound image.

7. The method of claim 1 , wherein the second ultrasound image is entered at an input layer of the model and the first output is entered at a bottom layer of the model.

8. A method, comprising:
inputting a first ultrasound image into a segmentation and tracking model;
receiving, as a first output from the segmentation and tracking model, a first location of an identified anatomical feature in the first ultrasound image;
displaying, on a display device, a first annotation at the first location on a second ultrasound image;
inputting the second ultrasound image and the first output into the segmentation and tracking model;
receiving, as a second output from the segmentation and tracking model, a second location of the identified anatomical feature in the second ultrasound image; and
displaying, on the display device, a second annotation at the second location on a third ultrasound image, wherein the segmentation and tracking model comprises a convolutional neural network having an input layer and an output layer, wherein the first output is output from the output layer, and wherein the second ultrasound image and the first output are entered into the convolutional neural network at the input layer.

9. The method of claim 8 , wherein the second ultrasound image is acquired after the first ultrasound image and the third ultrasound image is acquired after the second ultrasound image.

10. The method of claim 9 , wherein one or more additional ultrasound images are acquired between acquisition of the second ultrasound image and acquisition of the third ultrasound image, and wherein the first annotation is displayed on the display device with each of the one or more additional ultrasound images.

11. The method of claim 10 , further comprising determining, using a motion tracker, that motion of the identified anatomical feature is present across the second ultrasound image and the one or more additional ultrasound images, and in response, adjusting the first annotation based on the determined motion.

12. The method of claim 8 , further comprising adjusting one or more ultrasound imaging parameters of an ultrasound probe used to acquire the first ultrasound image, the second ultrasound image, and the third ultrasound image based at least in part on the first output and/or the second output.

13. The method of claim 12 , wherein the one or more ultrasound imaging parameters comprise one or more of frequency, depth, and gain.

14. The method of claim 8 , wherein displaying the first annotation at the first location on the second ultrasound image comprises displaying the first annotation as an overlay on the identified anatomical feature in the second ultrasound image.

15. A system, comprising:
an ultrasound probe;
a memory storing instructions; and
a processor communicably coupled to the memory and when executing the instructions, configured to:
acquire, via the ultrasound probe, a first ultrasound image;
identify, via output from a model, one or more anatomical features present in the first ultrasound image;
acquire, via the ultrasound probe, a second ultrasound image; and
determine a respective location of each of the identified one or more anatomical features in the second ultrasound image via the model using the second ultrasound image as input to the model and using the output as feedback to the model, wherein the model is a convolutional neural network (CNN), wherein the output is an output layer of the CNN, and wherein the output is entered to the CNN at an input layer of the CNN with the second ultrasound image.

16. The system of claim 15 , further comprising a display device, and wherein, when executing the instructions, the processor is configured to output, to the display device, one or more annotations each representing a respective location.

17. The system of claim 15 , wherein when executing the instructions, the processor is configured to output, to the display device, a third ultrasound image acquired after the second ultrasound image, the one or more annotations displayed on the third ultrasound image.

18. The system of claim 15 , wherein the output and the second ultrasound image are formed as a vector that is entered at the input layer of the CNN.

Assignees

• Ge Prec Healthcare Llc

Inventors

• Schein Sagi
• Shaked Doron
• Bergman Ruth
• Sato Naomi
• Halmann Menachem

Classifications

• A61B8/463

  characterised by displaying multiple images or images and diagnostic data on one display · CPC title

• A61B8/468 Primary

  allowing annotation or message recording · CPC title

• A61B8/085 Primary

  for locating body or organic structures, e.g. tumours, calculi, blood vessels, nodules · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/30101

  Blood vessel; Artery; Vein; Vascular · CPC title