Machine learning data augmentation using diffusion-based generative models

Patent metadata

Field	Value
Publication number	US-12406339-B2
Application number	US-202318314901-A
Country	US
Kind code	B2
Filing date	May 10, 2023
Priority date	May 10, 2023
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

Systems and methods for generating augmented images are provided. One or more input medical images are received. At least one of noise and one or more transformations are applied to the one or more input medical images to generate one or more noisy augmented images. The one or more noisy augmented images are denoised using a diffusion-based denoising system to generate one or more denoised augmented images. The applying and the denoising are repeated for one or more iterations using the one or more denoised augmented images as the one or more input medical images to generate one or more final augmented images. The one or more final augmented images are output.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
receiving one or more input medical images;
applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images, wherein applying the at least one of the noise and the one or more transformations comprises generating an uncertainty map of uncertainty introduced by the at least one of the noise and the one or more transformations;
denoising the one or more noisy augmented images using a diffusion-based denoising system based on the uncertainty map to generate one or more denoised augmented images;
repeating the applying and the denoising for one or more iterations using the one or more denoised augmented images as the one or more input medical images to generate one or more final augmented images; and
outputting the one or more final augmented images.

2. The computer-implemented method of claim 1 , wherein applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images comprises:
applying the one or more transformations to the one or more input medical images to generate one or more transformed input medical images; and
adding the noise to the one or more transformed input medical images to generate the one or more noisy augmented images.

3. The computer-implemented method of claim 2 , wherein generating an uncertainty map of uncertainty introduced by the at least one of the noise and the one or more transformations comprises:
generating the uncertainty map based on a difference between 1) the one or more transformed input medical images and 2) the one or more noisy augmented images.

4. The computer-implemented method of claim 1 , wherein denoising the one or more noisy augmented images using a diffusion-based denoising system based on the uncertainty map to generate one or more denoised augmented images comprises:
imputing regions masked by the noise during a first pass of the denoising; and
capturing local texture features during a second pass of the denoising.

5. The computer-implemented method of claim 1 , wherein applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images comprises:
randomly applying the one or more transformations to the one or more input medical images within a user-defined parameter.

6. The computer-implemented method of claim 1 , further comprising:
training a machine learning based model for performing a medical imaging analysis task based on the one or more final augmented images.

7. The computer-implemented method of claim 1 , wherein the diffusion- based denoising system comprises at least one of a diffusion probabilistic model, a DDPM (denoising diffusion probabilistic model), and a DDIM (denoising diffusion implicit model).

8. An apparatus comprising:
means for receiving one or more input medical images;
means for applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images, wherein the means for applying the at least one of the noise and the one or more transformations comprises means for generating an uncertainty map of uncertainty introduced by the at least one of the noise and the one or more transformations;
means for denoising the one or more noisy augmented images using a diffusion-based denoising system based on the uncertainty map to generate one or more denoised augmented images;
means for repeating the applying and the denoising for one or more iterations using the one or more denoised augmented images as the one or more input medical images to generate one or more final augmented images; and
means for outputting the one or more final augmented images.

9. The apparatus of claim 8 , wherein the means for applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images comprises:
means for applying the one or more transformations to the one or more input medical images to generate one or more transformed input medical images; and
means for adding the noise to the one or more transformed input medical images to generate the one or more noisy augmented images.

10. The apparatus of claim 9 , wherein the means for generating an uncertainty map of uncertainty introduced by the at least one of the noise and the one or more transformations comprises:
means for generating the uncertainty map based on a difference between 1) the one or more transformed input medical images and 2) the one or more noisy augmented images.

11. The apparatus of claim 8 , wherein the means for denoising the one or more noisy augmented images using a diffusion-based denoising system based on the uncertainty map to generate one or more denoised augmented images comprises:
means for imputing regions masked by the noise during a first pass of the denoising; and
means for capturing local texture features during a second pass of the denoising.

12. The apparatus of claim 8 , wherein the means for applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images comprises:
means for randomly applying the one or more transformations to the one or more input medical images within a user-defined parameter.

13. A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving one or more input medical images;
applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images, wherein applying the at least one of the noise and the one or more transformations comprises generating an uncertainty map of uncertainty introduced by the at least one of the noise and the one or more transformations;
denoising the one or more noisy augmented images using a diffusion-based denoising system based on the uncertainty map to generate one or more denoised augmented images;
repeating the applying and the denoising for one or more iterations using the one or more denoised augmented images as the one or more input medical images to generate one or more final augmented images; and
outputting the one or more final augmented images.

14. The non-transitory computer readable medium of claim 13 , wherein applying at least one of noise and one or more transformations to the one or more input medical images to generate one or more noisy augmented images comprises:
applying the one or more transformations to the one or more input medical images to generate one or more transformed input medical images; and
adding the noise to the one or more transformed input medical images to generate the one or more noisy augmented images.

15. The non-transitory computer readable medium of claim 14 , wherein generating an uncertainty map of uncertainty introduced by the at least one of the noise and the one or more transformations comprises:
generating the uncertainty map based on a difference between 1) the one or more transformed input medical images and 2) the one or more noisy augmented images.

16. The non-transitory computer readable medium of claim 13 , the operations further comprising:
training a machine learning based model for performing a medical imaging analysis task based on the one or more

Assignees

• Siemens Healthineers Ag

Inventors

• Gibson Eli
• Mailhe Boris

Classifications

• G06T12/30

  Image post-processing, e.g. metal artefact correction · CPC title

• G06T2207/30004

  Biomedical image processing · CPC title

• G06T2207/20092

  Interactive image processing based on input by user · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20048

  Transform domain processing · CPC title