Patch-based medical image generation for complex input datasets

Patent metadata

Field	Value
Publication number	US-2023342995-A1
Application number	US-202318304974-A
Country	US
Kind code	A1
Filing date	Apr 21, 2023
Priority date	Apr 21, 2022
Publication date	Oct 26, 2023
Grant date	—

Abstract

Systems, methods, and media for patch-based medical image generation for complex input datasets. Patch-based medical image generation can include creating a training dataset with an image patch and corresponding sensor data patch and training a neural network using the training dataset. Then, sensor data acquired from a patient using a medical imaging system can be applied as input to the neural network, and a medical image of the patient can be generated based on an output of the neural network.

First claim

1 . A method for medical imaging, comprising:
collecting a first medical image of a first patient from a database; splitting the first medical image into a first image patch and a second image patch; applying a Fourier transform to the first image patch to transform the first image patch into a first sensor data patch; creating a training dataset comprising the first image patch and the first sensor data patch; training a neural network using the training dataset; after training the neural network using the training dataset, applying sensor data acquired from a second patient using a medical imaging system as an input to the neural network; generating a second medical image of the second patient based on an output of the neural network; and displaying the second medical image of the second patient for clinical analysis.

2 . The method of claim 1 , further comprising adding synthetic phase to the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch.

3 . The method of claim 1 , further comprising resizing the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch.

4 . The method of claim 1 , further comprising adding random noise to the first sensor data patch before creating the training dataset.

5 . The method of claim 1 , wherein training the neural network using the training dataset comprises providing the first sensor data patch as the input to the neural network and associating the first sensor data patch with the first image patch as the output of the neural network.

6 . The method of claim 2 , wherein the first sensor data patch comprises complex-valued magnetic resonance k-space data.

7 . The method of claim 1 , wherein the neural network comprises a data-driven, manifold learning neural network.

8 . The method of claim 1 , further comprising:
applying the Fourier transform to the second image patch to transform the second image patch into a second sensor data patch; and adding the second image patch and the second sensor data patch to the training dataset before training the neural network using the training dataset.

9 . The method of claim 1 , further comprising:
before applying the sensor data acquired from the second patient as the input to the neural network, splitting the sensor data acquired from the second patient into a third sensor data patch and a fourth sensor data patch; wherein applying the sensor data acquired from the second patient as the input to the neural network comprises first applying the third sensor data patch as the input to the neural network and subsequently applying the fourth sensor data patch as the input to the neural network.

10 . A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by at least one processor, cause the at least one processor to implement operations comprising:
collecting a first medical image of a first patient from a database; splitting the first medical image into a first image patch and a second image patch; applying a Fourier transform to the first image patch to transform the first image patch into a first sensor data patch; creating a training dataset comprising the first image patch and the first sensor data patch; training a neural network using the training dataset; after training the neural network using the training dataset, applying sensor data acquired from a second patient using a medical imaging modality as an input to the neural network; generating a second medical image of the second patient based on an output of the neural network; and displaying the second medical image of the second patient for clinical analysis.

11 . The computer-readable medium of claim 9 , the operations further comprising:
adding synthetic phase to the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch; and resizing the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch; and adding random noise to the first sensor data patch before creating the training dataset; wherein the first sensor data patch comprises complex-valued magnetic resonance k-space data and the neural network comprises a data-driven, manifold learning neural network.

12 . A system comprising:
a display; one or more sensors; one or more processors; and one or more non-transitory computer readable storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to implement operations comprising:
collecting a first medical image of a first patient from a database;
splitting the first medical image into a first image patch and a second image patch;
applying a Fourier transform to the first image patch to transform the first image patch into a first sensor data patch;
creating a training dataset comprising the first image patch and the first sensor data patch;
training a neural network using the training dataset;
after training the neural network using the training dataset, applying sensor data acquired from a second patient as an input to the neural network;
generating a second medical image of the second patient based on an output of the neural network; and
causing the display to display the second medical image of the second patient for clinical analysis.

13 . The system of claim 12 , the operations further comprising:
adding synthetic phase to the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch; and adding random noise to the first sensor data patch before creating the training dataset.

14 . The system of claim 12 , the operations further comprising resizing the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch.

15 . The system of claim 12 , wherein:
the first sensor data patch comprises complex-valued magnetic resonance k-space data; and the neural network comprises a data-driven, manifold learning neural network.

16 . A method for training a neural network for medical imaging, comprising:
collecting a medical image of a patient from a database; splitting the medical image into at least a first image patch and a second image patch; applying a Fourier transform to the first image patch to transform the first image patch into a first sensor data patch; applying a Fourier transform to the second image patch to transform the second image patch into a second sensor data patch; creating a training dataset comprising the first image patch and the first sensor data patch, and the second image patch and the second sensor data patch; and training a neural network using the training dataset.

17 . The method of claim 16 , further comprising:
adding synthetic phase to the first image patch before applying the Fourier transform to the first image patch to transform the first image patch into the first sensor data patch; and adding synthetic phase to the second image patch before applying the Fourier transform to the second image patch to transform the second image patch into the second sensor data patch.

18 . The method of claim 16 , further comprising:
resizing the fir

Assignees

• Massachusetts Gen Hospital
• Univ Boston

Inventors

• Koonjoo Neha
• Rosen Matthew S
• Zhu Bo
• Bhutto Danyal Fareed

Classifications

• G06T12/10 Primary

  Image preprocessing, e.g. calibration, positioning of sources or scatter correction · CPC title

• G06T11/005 Primary

  Physics · mapped topic

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06N3/08

  Learning methods · CPC title

• G06T2210/41

  Medical · CPC title