Medical image classification based on a generative adversarial network trained discriminator

Patent metadata

Field	Value
Publication number	US-10937540-B2
Application number	US-201715850116-A
Country	US
Kind code	B2
Filing date	Dec 21, 2017
Priority date	Dec 21, 2017
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

Mechanisms are provided to implement a generative adversarial network (GAN). A discriminator of the GAN is configured to discriminate input medical images into a plurality of classes including a first class indicating a medical image representing a normal medical condition, a second class indicating an abnormal medical condition, and a third class indicating a generated medical image. A generator of the GAN generates medical images and a training medical image set is input to the discriminator that includes labeled medical images, unlabeled medical images, and generated medical images. The discriminator is trained to classify training medical images in the training medical image set into corresponding ones of the first, second, and third classes. The trained discriminator is applied to a new medical image to classify the new medical image into a corresponding one of the first class or second class. The new medical image is either labeled or unlabeled.

First claim

What is claimed is:

1. A method, in a data processing system comprising a processor and a memory, for training a classification engine, the memory comprising instructions that are executed by the processor to configure the processor to implement a generative adversarial network (GAN), the method comprising:
configuring a discriminator of the GAN to discriminate input medical images into a plurality of classes comprising a first class indicating a medical image representing a normal medical condition, one or more second classes indicating one or more abnormal medical conditions, and a third class indicating a generated medical image;
generating, by a generator of the GAN, one or more generated medical images;
inputting, to the discriminator of the GAN, a training medical image set comprising a first subset of labeled medical images, a second subset of unlabeled medical images, and a third subset comprising the one or more generated medical images;
training the discriminator to classify training medical images in the training medical image set into corresponding ones of the first class, the one or more second classes, and the third class; and
applying the trained discriminator to a new medical image to classify the new medical image into a corresponding one of the first class or one or more second classes, wherein the new medical image is either labeled or unlabeled.

2. The method of claim 1 , wherein the third class is used to train the generator to generate medical images that approximate actual medical images from noise input data by applying an adversarial training technique between the generator and the discriminator.

3. The method of claim 1 , wherein training the discriminator of the GAN to classify the training medical images in the training medical image set further comprises applying an adversarial training technique, based on the training medical image set, to train the discriminator to discriminate between medical images showing anomalies and medical images showing normal medical conditions.

4. The method of claim 1 , wherein the one or more generated medical images are unlabeled medical images generated by the generator based on training of the generator to generate medical images approximating actual medical images.

5. The method of claim 1 , wherein the one or more abnormal medical conditions comprise one or more medical diseases of a specific domain of medical images.

6. The method of claim 1 , further comprising training the generator to generate medical images approximating actual medical images at least by training the image generator to generate medical images that cause the discriminator to fail to correctly discriminate generated medical images from actual medical images for a predetermined ratio of instances indicating convergence of the training of the GAN.

7. The method of claim 1 , wherein training the discriminator to classify the training medical images in the training medical image set into corresponding ones of the first class, the one or more second classes, and the third class comprises:
for unlabeled medical images in the training medical image set, training the discriminator to correctly identify each of the unlabeled medical images as either a generated medical image or an actual medical image regardless of whether or not the unlabeled medical image shows an abnormality or a normal medical condition;
for labeled medical images in the training medical image set, training the discriminator to correctly identify each of the labeled medical images as either a medical image showing a normal medical condition or a medical image showing an abnormality, and further correctly identifying each of the labeled medical images as either being a generated medical image or an actual medical image; and
for generated medical images in the training medical image set, training the discriminator to correctly identify each generated medical image as being a generated medical image and not an actual medical image.

8. The method of claim 1 , wherein the discriminator is a convolutional neural network with K+1 output nodes, where K is a number of classes in the first class and the one or more second classes, and the K+1 class is the third class, and wherein the discriminator outputs, for each input medical image, a vector output having K+1 values indicating into which of the first class, one or more second classes, or third class the discriminator classifies the input medical image.

9. The method of claim 1 , wherein the discriminator is a convolutional neural network with 2K output nodes, where K is a number of classes in the one or more second classes, there is a separate output node indicating whether the input medical image is a generated medical image or an actual medical image for each of the K number of classes, and wherein the discriminator outputs, for each input medical image, a vector output having 2K values indicating into which class of the one or more second classes the discriminator classifies the input image and whether or not the input medical image is determined by the discriminator to be an actual medical image or a generated medical image.

10. The method of claim 1 , wherein configuring the discriminator further comprises:
configuring a loss function of the discriminator to include error components for each of actual labeled medical images, actual unlabeled medical images, and generated medical images, wherein training the discriminator further comprises training the generator to generate medical images approximating actual medical images at least by minimizing the loss function of the discriminator.

11. A computer program product comprising a non-transitory computer readable medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a generative adversarial network (GAN) and to:
configure a discriminator of the GAN to discriminate input medical images into a plurality of classes comprising a first class indicating a medical image representing a normal medical condition, one or more second classes indicating one or more abnormal medical conditions, and a third class indicating a generated medical image;
generate, by a generator of the GAN, one or more generated medical images;
input, to the discriminator of the GAN, a training medical image set comprising a first subset of labeled medical images, a second subset of unlabeled medical images, and a third subset comprising the one or more generated medical images;
train the discriminator to classify training medical images in the training medical image set into corresponding ones of the first class, the one or more second classes, and the third class; and
apply the trained discriminator to a new medical image to classify the new medical image into a corresponding one of the first class or one or more second classes, wherein the new medical image is either labeled or unlabeled.

12. The computer program product of claim 11 , wherein the third class is used to train the generator to generate medical images that approximate actual medical images from noise input data by applying an adversarial training technique between the generator and the discriminator.

13. The computer program product of claim 11 , wherein the computer readable program further causes the data processing system to train the discriminator of the GAN to classify the training medical images in the training medical image set further at least by applying an adversarial training technique, based on the training medical image set, to train the discriminator to discriminate between medical images sh

Assignees

• IBM
• IBM

Inventors

• Madani Ali
• Moradi Mehdi
• Syeda-Mahmood Tanveer F

Classifications

• G06V10/7753

  Incorporation of unlabelled data, e.g. multiple instance learning [MIL] · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G16H30/40 Primary

  for processing medical images, e.g. editing · CPC title

• G06N3/048

  Activation functions · CPC title