Segmentation, landmark detection and view classification using multi-task learning

Patent metadata

Field	Value
Publication number	US-10910099-B2
Application number	US-201916272169-A
Country	US
Kind code	B2
Filing date	Feb 11, 2019
Priority date	Feb 20, 2018
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

Medical image data may be applied to a machine-learned network learned on training image data and associated image segmentations, landmarks, and view classifications to classify a view of the medical image data, detect a location of one or more landmarks in the medical image data, and segment a region in the medical image data based on the application of the medical image data to the machine-learned network. The classified view, the segmented region, or the location of the one or more landmarks may be output.

First claim

We claim:

1. A method for performing multiple diagnostic tasks on medical image data, the method comprising:
receiving, by a processor, first medical image data;
applying, by the processor, the first medical image data to a machine-learned network learned on second medical image data and associated image segmentations, landmarks, and view classifications;
classifying, by the processor, a view of the first medical image data based on the application of the first medical image data to the machine-learned network;
detecting, by the processor, a location of one or more landmarks in the first medical image data based on the application of the first medical image data to the machine-learned network;
segmenting, by the processor, a region in the first medical image data based on the application of the first medical image data to the machine-learned network; and
outputting, by the processor, the classified view, the segmented region, or the location of the one or more landmarks.

2. The method of claim 1 , further comprising:
rescaling, by a processor, the first medical image data to match a resolution of the second medical image data.

3. The method of claim 1 , wherein classifying the view further comprises:
generating an anatomic label and an orientation of the first medical image data.

4. The method of claim 1 , wherein detecting the location of the one or more landmarks is based on the view classification.

5. The method of claim 1 , wherein the first medical image data is generated by an ultrasound, magnetic resonance tomography, or computed tomography imaging system.

6. The method of claim 5 , wherein the second medical image data is generated by an ultrasound, magnetic resonance tomography, or computed tomography imaging system, and wherein the first medical image data is generated by a different imaging modality than at least a portion of the second medical image data.

7. The method of claim 1 , wherein the processor is part of a medical imaging system.

8. A medical imaging system for performing multiple diagnostic tasks on medical image data, the system comprising:
a memory storing a machine-learned network learned on second medical image data and ground truth including segmentation, landmark, and view classification for each of a plurality of second images of the second medical image data; and
an image processor configured to apply the medical image data to the machine-learned network and, based thereon, detect a location of one or more landmarks in the first medical image data, classify a view of the first medical image data, segment anatomy in the first medical image, or combinations thereof.

9. The system of claim 8 , further comprising:
an ultrasound, magnetic resonance tomography, or computed tomography medical imaging scanner configured to generate the first medical image data.

10. The system of claim 9 , wherein the machine-learned network was trained on second medical image data having been generated by a further medical imaging scanner of a modality different from the medical imaging scanner configured to generate the first medical image data.

11. The method of claim 1 , wherein the view of the first medical image data is an orientation of the first medical image data in reference to a viewing point or to a side of a body.

12. The method of claim 3 , wherein the orientation of the first medical image data is referenced to a viewing point or to a side of a body.

13. The system of claim 8 , wherein the view of the first medical image data is an orientation of the first medical image data in reference to a viewing point or to a side of a body.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Xu Zhoubing
• Huo Yuankai
• Park Jin-Hyeong
• Grbic Sasa
• Zhou Shaohua Kevin

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G16H30/40 Primary

  for processing medical images, e.g. editing · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title