Method and system for artificial intelligence based medical image segmentation

Patent metadata

Field	Value
Publication number	US-10878219-B2
Application number	US-201716094900-A
Country	US
Kind code	B2
Filing date	Jul 19, 2017
Priority date	Jul 21, 2016
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

Methods and systems for artificial intelligence based medical image segmentation are disclosed. In a method for autonomous artificial intelligence based medical image segmentation, a medical image of a patient is received. A current segmentation context is automatically determined based on the medical image and at least one segmentation algorithm is automatically selected from a plurality of segmentation algorithms based on the current segmentation context. A target anatomical structure is segmented in the medical image using the selected at least one segmentation algorithm.

First claim

The invention claimed is:

1. A method for autonomous artificial intelligence based medical image segmentation, comprising:
receiving a medical image of a patient;
automatically determining a current segmentation context based on the medical image by inputting the medical image to a trained deep neural network that automatically extracts learned image characteristics from the medical image;
automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms including a plurality of different deep-learning based segmentation algorithms based on the current segmentation context using the trained deep neural network, wherein the trained deep neural network predicts a best segmentation algorithm or combination of segmentation algorithms from the plurality of segmentation algorithms for performing segmentation of a target anatomical structure in the medical image based on the extracted learned image characteristics; and
segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm.

2. The method of claim 1 , wherein the steps of automatically determining the current segmentation context based on the medical image and selecting the at least one segmentation algorithm from the plurality of segmentation algorithms based on the current segmentation context are performed by a software-based trained master segmentation artificial agent running on a computer system.

3. The method of claim 1 , wherein automatically determining a current segmentation context based on the medical image further comprises:
automatically identifying the target anatomical structure based on a view of the medical image currently being visualized on a display device by automatically detecting what anatomical structure is currently being visualized on the display device using a trained machine learning based classifier.

4. The method of claim 3 , wherein automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises:
automatically selecting, by the trained deep neural network, the at least one segmentation algorithm from multiple segmentation algorithms specialized for segmentation of the automatically identified target anatomical structure based on the extracted learned image characteristics.

5. The method of claim 1 , wherein the trained deep neural network is trained based on medical image training samples and synthetic training samples generated at a plurality if noise levels from the medical image training samples.

6. The method of claim 1 , wherein automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms comprises selecting multiple segmentation algorithms from the plurality of segmentation algorithms based on the current segmentation context, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:
segmenting the target anatomical structure in the medical image using each of the selected multiple segmentation algorithms; and
fusing segmentation results from the selected multiple segmentation algorithms to generate final segmentation result for the target anatomical structure in the medical image.

7. The method of claim 1 , wherein the plurality of different deep-learning based segmentation algorithms each have a respective deep network architecture trained for a specific segmentation task.

8. The method of claim 1 , wherein segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:
segmenting the target anatomical structure in the medical image using the trained deep neural network with one or more integrated priors.

9. The method of claim 1 , wherein the selected at least one segmentation algorithm comprises a deep reinforcement learning based segmentation algorithm, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:
iteratively adjusting a statistical shape model representing a shape of the target anatomical structure in the medical image by selecting, at each iteration, an action corresponding to an adjustment of a parameter of the statistical shape model based on action values calculated using deep neural network trained using deep reinforcement learning.

10. The method of claim 1 , wherein segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:
generating a plurality of Gaussian distributed probability maps at different scales from the medical image using a trained deep image-to-image network;
combining the plurality of Gaussian probability maps at the different scales into a combined probability map; and
extracting a segmented boundary of the target anatomical structure from the medical image based on the combined probability map.

11. The method of claim 1 , wherein the selected at least one segmentation algorithm comprises a deep learning partial inference based segmentation algorithm, and segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:
generating an initial segmentation mask for the target anatomical structure from the medical image using a trained initial convolutional encoder decoder (CED);
constructing a unified tensor combining a current segmentation mask and the medical image; and
generating an updated segmentation mask for the target anatomical structure using a trained partial inference CED.

12. The method of claim 1 , wherein segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm comprises:
fitting an active shape model to a shape of the target anatomical structure in the medical image by using a recurrent neural network (RNN) to adjust locations of points on the active shape model.

13. An apparatus for autonomous artificial intelligence based medical image segmentation, comprising:
means for receiving a medical image of a patient;
means for automatically determining a current segmentation context based on the medical image by inputting the medical image to a trained deep neural network that automatically extracts learned image characteristics from the medical image;
means for automatically selecting at least one segmentation algorithm from a plurality of segmentation algorithms including a plurality of different deep-learning based segmentation algorithms based on the current segmentation context using the trained deep neural network, wherein the trained deep neural network predicts a best segmentation algorithm or combination of segmentation algorithms from the plurality of segmentation algorithms for performing segmentation of a target anatomical structure in the medical image based on the extracted learned image characteristics; and
means for segmenting the target anatomical structure in the medical image using the selected at least one segmentation algorithm.

14. The apparatus of claim 13 , wherein the means for automatically determining a current segmentation context based on the medical image further comprises:
means for automatically identifying the target anatomical structure based on a view of the medical image currently being visualized on a display device by automatically detecting what anatomical structure is currently being visualized on the display device using a trained machine learning based classifier.

15. The apparatus of claim 14 , wherein the means for automat

Assignees

• Siemens Healthcare Gmbh

Inventors

• Zhou Shaohua Kevin
• Chen Mingqing
• Ding Hui
• Georgescu Bogdan
• Gulsun Mehmet Akif
• Kim Tae Soo
• Kiraly Atilla Peter
• Lu Xiaoguang
• Park Jin-Hyeong
• Sharma Puneet
• Sun Shanhui
• Xu Daguang
• Xu Zhoubing
• Zheng Yefeng

Classifications

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06V20/695

  Preprocessing, e.g. image segmentation · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06V10/7553

  based on shape, e.g. active shape models [ASM] · CPC title