Training set enrichment with insignificantly-abnormal medical images

Patent metadata

Field	Value
Publication number	US-11301720-B2
Application number	US-202016860086-A
Country	US
Kind code	B2
Filing date	Apr 28, 2020
Priority date	Apr 28, 2020
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

A method including: automatically detecting, using at least one machine learning algorithm, one or more abnormalities depicted in a medical image of a patient; automatically determining whether the one or more abnormalities have remained temporally and unchanged, based on an older medical image of the patient; and upon determining that the one or more abnormalities have remained temporally and spatially unchanged: automatically inpainting the one or more abnormalities in the medical image, and automatically enrich a new training set with the inpainted medical image.

First claim

What is claimed is:

1. A method comprising operating at least one hardware processor to:
automatically detect, using at least one machine learning algorithm, one or more abnormalities depicted in a medical image of a patient;
automatically determine whether the one or more abnormalities have remained temporally unchanged, based on an older medical image of the patient; and
upon determining that the one or more abnormalities have remained temporally unchanged:
automatically inpaint the one or more abnormalities in the medical image, and

add the inpainted medical image, with a label denoting that the inpainted medical image is normal, to a new training set for a new machine learning classifier.

2. The method of claim 1 , wherein the detection of the one or more abnormalities by the at least one machine learning algorithm comprises:
applying a machine learning classifier to the medical image of a patient, wherein the machine learning classifier is configured to classify medical images as normal or abnormal, and wherein the application of the machine learning classifier to the medical image results in classification of the medical image as abnormal; and
segmenting the one or more abnormalities in the medical image, using an artificial neural network (ANN) configured for segmentation.

3. The method of claim 2 , further comprising:
prior to applying the machine learning classifier: training the machine learning classifier to detect abnormalities, based on a training set which comprises medical images that are each manually labeled as normal or abnormal; and
prior to segmenting the one or more abnormalities: training the ANN to segment abnormalities, based on a training set which comprises medical images in which abnormalities are manually segmented.

4. The method of claim 2 , wherein:
the determination of whether the one or more abnormalities have remained temporally unchanged comprises comparing the segmented one or more abnormalities with a corresponding area in the older medical image.

5. The method of claim 1 , wherein the detection of the one or more abnormalities by the at least one machine learning algorithm comprises:
segmenting the one or more abnormalities in the medical image, using an artificial neural network (ANN) configured for segmentation of abnormalities.

6. The method of claim 5 , further comprising:
prior to segmenting the one or more abnormalities: training the ANN to segment abnormalities, based on a training set which comprises medical images in which abnormalities are manually segmented.

7. The method of claim 5 , wherein:
the determination of whether the one or more abnormalities have remained temporally unchanged comprises comparing the segmented one or more abnormalities with a corresponding area in the older medical image.

8. The method of claim 1 , wherein:
the determination of whether the one or more abnormalities have remained temporally unchanged is based on detecting at least one of: a size change, a color change, and a texture change.

9. The method of claim 1 , wherein:
the at least one machine learning algorithm is configured to detect abnormalities that are selected from the group consisting of: benign lesions, artificial implants, injuries, and interventional tissue modifications.

10. The method of claim 1 , further comprising:
repeating the steps of claim 1 for multiple medical images of multiple patients, wherein the inpainted medical images are used to enrich a single new training set;
manually labeling each of the inpainted medical images as normal;
adding, to the single new training set, additional medical images that are labeled as abnormal; and
training a new machine learning classifier based on the single new training set.

11. The method of claim 10 , wherein the new machine learning classifier is configured, following its training, to detect abnormalities that are selected from the group consisting of: malignant lesions, and premalignant lesions.

12. A system comprising:
(a) at least one hardware processor; and
(b) a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by said at least one hardware processor to:
automatically detect, using at least one machine learning algorithm, one or more abnormalities depicted in a medical image of a patient,
automatically determine whether the one or more abnormalities have remained temporally unchanged, based on an older medical image of the patient, and
upon determining that the one or more abnormalities have remained temporally unchanged:
automatically inpaint the one or more abnormalities in the medical image, and
add the inpainted medical image, with a label denoting the inpainted medical image is normal, to a new training set for a new machine learning classifier.

13. The system of claim 12 , wherein the detection of the one or more abnormalities by the at least one machine learning algorithm comprises:
applying a machine learning classifier to the medical image of a patient, wherein the machine learning classifier is configured to classify medical images as normal or abnormal, and wherein the application of the machine learning classifier to the medical image results in classification of the medical image as abnormal; and
segmenting the one or more abnormalities in the medical image, using an artificial neural network (ANN) configured for segmentation,
wherein the determination of whether the one or more abnormalities have remained temporally unchanged comprises comparing the segmented one or more abnormalities with a corresponding area in the older medical image.

14. The system of claim 12 , wherein the detection of the one or more abnormalities by the at least one machine learning algorithm comprises:
segmenting the one or more abnormalities in the medical image, using an artificial neural network (ANN) configured for segmentation of abnormalities,
wherein the determination of whether the one or more abnormalities have remained temporally unchanged comprises comparing the segmented one or more abnormalities with a corresponding area in the older medical image.

15. The system of claim 12 , wherein:
the determination of whether the one or more abnormalities have remained temporally unchanged is based on detecting at least one of: a size change, a color change, and a texture change.

16. The system of claim 12 , wherein the program code if further executable by said at least one hardware processor to:
repeat the execution of the program code of claim 12 for multiple medical images of multiple patients, wherein the inpainted medical images are used to enrich a single new training set;
manually label each of the inpainted medical images as normal;
add, to the single new training set, additional medical images that are labeled as abnormal; and
train a new machine learning classifier based on the single new training set.

17. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:
automatically detect, using at least one machine learning algorithm, one or more abnormalities depicted in a medical image of a patient,
automatically determine whether the one or more abnormalities have remained temporally unchanged, based on an older medical image of the patient, and
upon determining that the one or more abnormalities have remained temporally unchanged:
automatically inpaint the one or more abnormalities in the medical image, and
add th

Assignees

• IBM

Inventors

• Levanony Dana
• Perek Shaked
• Hexter Efrat

Classifications

• G06V10/759

  Region-based matching · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T7/0016 Primary

  involving temporal comparison · CPC title