Deep reinforcement learning for recursive segmentation

Patent metadata

Field	Value
Publication number	US-10733788-B2
Application number	US-201916251242-A
Country	US
Kind code	B2
Filing date	Jan 18, 2019
Priority date	Mar 15, 2018
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

Systems and methods are provided for generating segmented output from input regardless of the resolution of the input. A single trained network is used to provide segmentation for an input regardless of a resolution of the input. The network is recursively trained to learn over large variations in the input data including variations in resolution. During training, the network refines its prediction iteratively in order to produce a fast and accurate segmentation that is robust across resolution differences that are produced by MR protocol variations.

First claim

The invention claimed is:

1. A method for generating segmented magnetic resonance volumes in a magnetic resonance imaging system, the method comprising:
scanning a patient by the magnetic resonance imaging system; magnetic resonance volume data resulting from the scanning;
inputting the magnetic resonance volume data to a machine trained network that is recursively trained using machine learning techniques to generate segmented volumes from input magnetic resonance volume data regardless of a resolution of the input magnetic resonance volume data, wherein recursively training the network comprises generating probability maps from a SoftMax activation layer at each iteration of the recursive training, up-sampling or pooling the input volume to different resolutions for each iteration, and after each iteration multiplying the probability maps with the up-sampled or pooled input volume and inputting the multiplied probability maps back to the network for subsequent iteration;
generating, by the machine trained network, a segmented magnetic resonance volume from the input magnetic resonance volume data; and
displaying the segmented magnetic resonance volume.

2. The method of claim 1 , wherein the machine trained network is a three-dimensional image-to-image network.

3. The method of claim 1 , wherein the magnetic resonance imaging system is configured to acquire magnetic resonance volume data of a brain of a patient.

4. The method of claim 3 , wherein the machine trained network is trained to perform a segmentation task comprising skull stripping of the magnetic resonance volume data.

5. The method of claim 4 , further comprising:
performing a subsequent segmentation of the segmented magnetic resonance volume to identify tissue types.

6. A method for training a network using machine learning to generate segmented magnetic resonance images regardless of input resolution, the method comprising:
inputting a magnetic resonance image of a plurality of magnetic resonance images of a set of training data into a network using a quantity of input channels, wherein the quantity is equal to a quantity of classifications provided by the network;
generating, by the network, a segmented image including a quantity of probability maps equal to the quantity of classifications, wherein each probability map of the quantity of probability maps comprises a probability map for a different class of the quantity of classifications;
comparing the segmented image to a ground truth segmented image for the magnetic resonance image;
adjusting the network based on the comparison;
selecting, by a reinforcement agent, a resolution action as a function of the comparison;
multiplying each of the quantity of input channels of the magnetic resonance image by a respective probability map of the quantity of probability maps;
performing the resolution action on the magnetic resonance images for each of the quantity of input channels;
inputting the altered magnetic resonance images of the quantity of input channels into the network;
repeating generating, comparing, adjusting, selecting, multiplying, performing, and inputting for at least five iterations; and
outputting a machine trained network.

7. The method of claim 6 , wherein the network is a dense image to image network.

8. The method of claim 6 , wherein the resolution action comprises one of up sampling, pooling, or no action.

9. The method of claim 6 , wherein the network is configured to input a magnetic resonance brain image and output a segmented magnetic resonance brain image.

10. The method of claim 6 , wherein the reinforcement agent is trained using a reinforcement mechanism to select the resolution action.

11. The method of claim 6 , wherein the segmented image is used as the input for a subsequent iteration.

12. The method of claim 6 , wherein generating, comparing, adjusting, selecting, multiplying, performing, and inputting is repeated for at least ten iterations.

13. The method of claim 6 , wherein each magnetic resonance image of the plurality of magnetic resonance images of the set of training data is input into the network prior to outputting the machine trained network.

14. A system for generating a machine trained network configured to use inputs of different resolutions, the system comprising:
a magnetic resonance imaging system configured to acquire magnetic resonance data at different resolutions;
a memory configured to store the magnetic resonance data, associated labeled magnetic resonance data, and a network; and
an image processor configured to recursively train the network using an input volume from the magnetic resonance data and an associated labeled volume using back propagation;
wherein the network generates probability maps from a SoftMax activation layer at each iteration of the recursive training;
wherein the input volume is up sampled or pooled to different resolutions for each iteration;
wherein after each iteration, the probability maps are multiplied with the input volume and input back to the network;

wherein the recursive training is repeated for each volume in the magnetic resonance data.

15. The system of claim 14 , wherein the network is configured to input a magnetic resonance brain volume and output a segmented magnetic resonance brain volume regardless of the resolution of the input magnetic resonance brain volume.

16. The system of claim 14 , wherein the image processer is further configured to select whether the input volume is up sampled or pooled to different resolutions for each iteration as a function of a selection by a reinforcement agent stored in the memory.

17. The system of claim 16 , wherein the reinforcement agent selects whether the input volume is up sampled or pooled to different resolutions for each iteration as a function of a comparison between an output of the network and an associated labeled volume at each iteration.

18. The system of claim 14 , wherein the network is a three dimensional dense image to image network.

19. The system of claim 14 , wherein the image processor is configured to perform at least five iterations of the recursive training for each volume in the magnetic resonance data.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Ceccaldi Pascal
• Chen Xiao
• Mailhe Boris
• Odry Benjamin L
• Nadar Mariappan S

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title