Automatic condition diagnosis using a segmentation-guided framework

Patent metadata

Field	Value
Publication number	US-11830187-B2
Application number	US-202117331263-A
Country	US
Kind code	B2
Filing date	May 26, 2021
Priority date	May 26, 2021
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

Methods and systems for training computer-aided condition detection systems. One method includes receiving a plurality of images for a plurality of patients, some of the images including an annotation associated with a condition; iteratively applying a first deep learning network to each of the images to produce a segmentation map, a feature map, and an image-level probability of the condition for each of the images; iteratively applying a second deep learning network to each feature map produced by the first network to produce a plurality of outputs; training the first network based on the segmentation map produced for each image; and training the second network based on the output produced for each of the patients. The second network includes a plurality of convolution layers and a plurality of convolutional long short-term memory (LSTM) layers. Each of the outputs includes a patient-level probability of the condition for one of the patients.

First claim

What is claimed is:

1. A method of training a computer-aided condition detection system, the method comprising:
receiving a plurality of medical images for a plurality of patients, a portion of the plurality of medical images including at least one annotation of a condition;
iteratively applying a first deep learning network to each of the plurality of medical images to produce a segmentation map, a feature map, and an image-level probability of the condition for each of the plurality of medical images; and
iteratively applying a second deep learning network to the feature map produced by the first deep learning network for each of the plurality of medical images to produce a plurality of patient outputs;
training the first deep learning network based on the segmentation map produced by the first deep learning network for each image included in the portion of the plurality of medical images; and
training the second deep learning network based on the patient output produced by the second deep learning network for each of the plurality of patients,
wherein the second deep learning network includes a plurality of convolution layers and a plurality of convolutional long short-term memory (LSTM) layers; and
wherein each of the plurality of patient outputs includes a patient-level probability of the condition for one of the plurality of patients.

2. The method of claim 1 , wherein the segmentation map includes a disease lesion segmentation map.

3. The method of claim 1 , wherein the plurality of medical images include a plurality of two-dimensional (2-D) images, each of a plurality of subsets of the plurality of 2-D images constituting a series of images corresponding to a three-dimensional (3-D) image of one of the plurality of patients, wherein each 3-D image has a label indicating whether one of the plurality of patients has the condition.

4. The method of claim 3 , further comprising training the first deep learning network using back propagation to reduce a loss function.

5. The method of claim 4 , wherein training the first deep learning network includes:
for each 2-D image included in the portion of the plurality of 2-D medical images including an annotation of the condition:
comparing the annotation of the condition to the segmentation map produced by first deep learning network for the 2-D image to determine a segmentation loss, and
comparing the image-level probability produced by the first deep learning network for the 2-D image with a label of the 2-D image to produce a classification loss; and
updating a first set of parameters of the first deep learning network using one or both of the classification loss and the segmentation loss.

6. The method of claim 5 , further comprising:
freezing the first set of parameters;
updating a second set of parameters of the second deep learning network based on comparing the patient-level probability of the condition of each patient output of the plurality of patient outputs to the label of the associated 3-D image.

7. The method of claim 5 , further comprising:
updating the first set of parameters of the first deep learning network and a second set of parameters of the second deep learning network based on comparing the patient-level probability of the condition of each patient output of the plurality of patient outputs to the label of the associated 3-D image.

8. The method of claim 1 , wherein the condition is pulmonary embolism.

9. The method of claim 1 , wherein the first deep learning network is one selected from a group consisting of U-Net, V-Net, and Convolutional Autoencoders.

10. The method of claim 1 , wherein the first deep learning network includes one or more encoding convolutional layers, one or more bottleneck convolutional layers, and one or more decoding convolutional layers.

11. The method of claim 10 , wherein the first deep learning network includes one or more dense layers attached to an output of a last of the one or more bottleneck convolutional layers, wherein the image level probability is an output of the one or more dense layers and the segmentation map is an output of a last of the one or more decoding convolutional layers.

12. The method of claim 1 , wherein iteratively applying the second deep learning network to the feature map produced for each of the plurality of medical images to produce the plurality of patient outputs includes iteratively applying the second deep learning network to the feature map produced for each of the plurality of medical images to produce an output for each of the plurality of medical images and pooling, for each of the plurality of patients, the outputs for a subset of the plurality of medical images to produce the patient-level probability for each of the plurality of patients.

13. The method of claim 12 , wherein pooling the outputs for a subset of the plurality of medical images includes pooling the outputs using an aggregation function.

14. The method of claim 13 , wherein the aggregation function is one selected from a group consisting of a mean function, a max function, a mode function, and a self-attention function.

15. The method of claim 14 , wherein the aggregation function is the self-attention function, and the self-attention function is a third deep learning network including a plurality of fully connected dense layers and a plurality of non-linear activation function layers.

16. The method of claim 1 , wherein each of the plurality of convolutional long short-term memory (LSTM) layers is one selected from a group consisting of a unidirectional LSTM layer and a bidirectional LSTM layer.

17. The method of claim 1 , wherein training the second deep learning network includes training the second deep learning network to minimize a classification loss using at least one selected from a group consisting of a binary cross-entropy loss and a focal loss as an objective function.

18. The method of claim 1 , wherein the method further comprises, after training the first deep learning network and after training the second deep learning network:
receiving a three-dimensional (3-D) medical image of a patient, the 3-D image including a plurality of two-dimensional (2-D) medical images of the patient;
iteratively applying the first deep learning network to each of the plurality of 2-D medical images of the patient to produce a plurality of feature maps associated with the patient, wherein each of the plurality of feature maps associated with the patient is associated with one of the plurality of 2-D medical images of the patient;
iteratively applying the second deep learning network to each of the plurality of feature maps associated with the patient to produce a plurality of classification outputs associated with the patient;
pooling the plurality of classification outputs associated with the patient to generate a condition probability of the patient, the condition probability of the patient including a probability of the patient having the condition; and
outputting the condition probability of the patient.

19. A system for training a computer-aided condition detection program, the system comprising:
a computing device including an electronic processor configured to:
receive a plurality of medical images for a plurality of patients, a portion of the plurality of medical images including at least one annotation of a condition;
iteratively apply a first deep learning network to each of the plurality of medical images to produce a segmentation map, a feature map, and an image-level probability of the condition for each o

Assignees

• IBM

Inventors

• Shi Luyao
• Beymer David James
• Marvast Ehsan Dehghan
• Rajan Deepta

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title