Systems and methods for processing medical images with invertible neural networks

Patent metadata

Field	Value
Publication number	US-2024062438-A1
Application number	US-202217891668-A
Country	US
Kind code	A1
Filing date	Aug 19, 2022
Priority date	Aug 19, 2022
Publication date	Feb 22, 2024
Grant date	—

Abstract

Described herein are systems, methods, and instrumentalities associated with using an invertible neural network to complete various medical imaging tasks. Unlike traditional neural networks that may learn to map input data (e.g., a blurry reconstructed MRI image) to ground truth (e.g., a fully-sampled MRI image), the invertible neural network may be trained to learn a mapping from the ground truth to the input data, and may subsequently apply an inverse of the mapping (e.g., at an inference time) to complete a medical imaging task. The medical imaging task may include, for example, MRI image reconstruction (e.g., to increase the sharpness of a reconstructed MRI image), image denoising, image super-resolution, and/or the like.

First claim

What is claimed is:

1 . An apparatus, comprising:
one or more processors configured to:
obtain a first medical image of an anatomical structure;
process the first medical image of the anatomical structure through an invertible neural network (INN) to obtain a second medical image of the anatomical structure, wherein the second medical image includes an improvement to the first medical image with respect to at least one of a sharpness of the second medical image, a resolution of the second medical image, or an amount of noise in the second medical image, and wherein, during the processing of the first medical image, the INN is used to map the first medical image to the second medical image based on an inverse of a mapping function learned by the INN through training; and
store or transmit the second medical image.

2 . The apparatus of claim 1 , wherein the INN includes an invertible residual network.

3 . The apparatus of claim 2 , wherein the invertible residual network includes multiple convolution layers and wherein a constraint is imposed upon the multiple convolution layers to limit an amount of change that occurs at an output of the invertible residual network in response to an amount of change that occurs at an input of the invertible residual network.

4 . The apparatus of claim 3 , wherein the constraint is imposed through a Lipschitz constant.

5 . The apparatus of claim 1 , wherein the first medical image includes a magnetic resonance imaging (MRI) image reconstructed based on under-sampled MRI data, wherein the mapping function learned by the INN through training is for mapping a fully-sampled MRI image to a latent space representation of a known probability distribution based on a reconstructed MRI image, and wherein, during the processing of the first medical image, the inverse of the mapping function is applied to the first medical image such that the second medical image obtained through the processing approximately follows a probability distribution of fully-sampled MRI images.

6 . The apparatus of claim 5 , wherein the known probability distribution is a Gaussian distribution.

7 . The apparatus of claim 5 , wherein the mapping function is learned by the INN through a training process and, during the training process, the INN is configured to:
obtain a fully-sampled MRI training image and a reconstructed MRI training image, wherein the fully-sampled MRI training image belongs to a training dataset of fully-sampled MRI images and the reconstructed MRI training image is generated based on under-sampled MRI training data to approximate the fully-sampled MRI training image; determine, based on the fully-sampled MRI training image and the reconstructed MRI training image, an estimated latent space representing the known probability distribution; determine a loss based on the estimated latent space and a ground truth; and adjust one or more parameters of the INN based on the determined loss.

8 . The apparatus of claim 1 , wherein the first medical image includes an magnetic resonance imaging (MRI) image reconstructed based on under-sampled MRI data, wherein the mapping function learned by the INN through training is for mapping a fully-sampled MRI image to a reconstructed MRI image, and wherein, during the processing of the first medical image, the inverse of the mapping function is applied to the first medical image such that the second medical image obtained through the processing has an increased sharpness compared to the first medical image.

9 . The apparatus of claim 8 , wherein the mapping function is learned by the INN through a training process and, during the training process, the INN is configured to:
obtain a fully-sampled MRI training image; predict, based on the fully-sampled MRI training image, an MRI image that corresponds to a reconstruction of under-sampled MRI training data; determine a loss between the predicted MRI image and a ground truth image reconstructed from under-sampled MRI training data; and adjust one or more parameters of the INN based on the determined loss.

10 . The apparatus of claim 1 , wherein the first medical image is generated based on under-sampled magnetic resonance imaging (MRI) data, and wherein the one or more processors being configured to process the first medical image through the INN to obtain the second medical image comprises the one or more processors being configured to predict, using the INN, a preliminary MRI image based on the first medical image, determine MRI data that corresponds to the preliminary MRI image, update at least a part of the MRI data that corresponds to the preliminary MRI image based on the under-sampled MRI data, and generate the second medical image based on the updated MRI data.

11 . The apparatus of claim 10 , wherein the MRI data corresponding to the preliminary MRI image is determined by applying a Fourier transform to the preliminary MRI image, and wherein the second medical image is generated by applying an inverse Fourier transform to the updated MRI data.

12 . A method, comprising:
obtaining a first medical image of an anatomical structure; processing the first medical image of the anatomical structure through an invertible neural network (INN) to obtain a second medical image of the anatomical structure, wherein the second medical image includes an improvement to the first medical image with respect to at least one of a sharpness of the second medical image, a resolution of the second medical image, or an amount of noise in the second medical image, and wherein, during the processing of the first medical image, the INN is used to map the first medical image to the second medical image based on an inverse of a mapping function learned by the INN through training; and storing or transmitting the second medical image.

13 . The method of claim 12 , wherein the INN includes an invertible residual network.

14 . The method of claim 13 , wherein the invertible residual network includes multiple convolution layers and wherein a constraint is imposed upon the multiple convolution layers to limit an amount of change that occurs at an output of the invertible residual network in response to an amount of change that occurs at an input of the invertible residual network.

15 . The method of claim 14 , wherein the constraint is imposed through a Lipschitz constant.

16 . The method of claim 12 , wherein the first medical image includes a magnetic resonance imaging (MRI) image reconstructed based on under-sampled MRI data, wherein the mapping function learned by the INN through training is for mapping a fully-sampled MRI image to a latent space representation of a known probability distribution, and wherein, during the processing of the first medical image, the inverse of the mapping function is applied to the first medical image such that the second medical image obtained through the processing approximately follows a probability distribution of fully-sampled MRI images.

17 . The method of claim 16 , wherein the mapping function is learned by the INN through a training process and, during the training process, the INN is configured to:
obtain a fully-sampled MRI training image and a reconstructed MRI training image, wherein the fully-sampled MRI training image belongs to a training dataset of fully-sampled MRI images and the reconstructed MRI training image is generated based on under-sampled MRI training data to approximate the fully-sampled MRI training image; determine, based on the fully-sampled MRI training image and the reconstructed MRI training image, an

Assignees

• Shanghai United Imaging Intelligence Co Ltd

Inventors

• Chen Zhang
• Dong Siyuan
• Sun Shanhui
• Chen Xiao
• Liu Yikang
• Chen Terrence

Classifications

• G06T12/30 Primary

  Image post-processing, e.g. metal artefact correction · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/10088

  Magnetic resonance imaging [MRI] · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T5/60

  using machine learning, e.g. neural networks · CPC title