System and method for automated transform by manifold approximation

Patent metadata

Field	Value
Publication number	US-2025209805-A1
Application number	US-202519079065-A
Country	US
Kind code	A1
Filing date	Mar 13, 2025
Priority date	Sep 1, 2016
Publication date	Jun 26, 2025
Grant date	—

Abstract

A system may transform sensor data from a sensor domain to an image domain using data-driven manifold learning techniques which may, for example, be implemented using neural networks. The sensor data may be generated by an image sensor, which may be part of an imaging system. Fully connected layers of a neural network in the system may be applied to the sensor data to apply an activation function to the sensor data. The activation function may be a hyperbolic tangent activation function. Convolutional layers may then be applied that convolve the output of the fully connected layers for high level feature extraction. An output layer may be applied to the output of the convolutional layers to deconvolve the output and produce image data in the image domain.

First claim

What is claimed is:

1 . A medical imaging system comprising:
a computer processor configured to receive image data acquired by an imaging system from a patient, wherein the image data is in a first domain; a neural network trained to receive the image data and transform the image data from the first domain into a second domain that is different from the first domain to create an image of the patient; and a display configured to display the image of the patient.

2 . The medical imaging system of claim 1 , wherein the neural network is a data-driven, manifold-learning, neural network configured to receive the image data and to transform the image data from the first domain to an image domain to produce the image of the patient;

3 . The medical imaging system of claim 1 , wherein the imaging system includes at least one of:
a radio frequency (RF) system of a magnetic resonance imaging (MRI) system and wherein the image data comprises magnetic resonance data; an x-ray detector of a computed tomography (CT) system and wherein the image data comprises x-ray attenuation data; a gamma ray detector of an emission tomography system and wherein the image data comprises emission tomography data; an ultrasound transducer of a ultrasound system and wherein the image data comprises ultrasound data; and an optical sensor of an optical imaging system and wherein the image data comprises optical imaging data.

4 . A system comprising:
an input configured to receive sensor data in a sensor domain from an image sensor configured to generate the sensor data, wherein the sensor data is acquired from a patient during a medical imaging process; a neural network trained to receive the sensor data from the image sensor and to transform the sensor data from the sensor domain to an image domain to produce a medical image of the patient; and one of a display to display the medical image of the patient or a communications system to transmit the medical image of the patient.

5 . The system of claim 4 , wherein the processor is configured to transform the sensor data from the sensor domain to the image domain using the neural network by applying the sensor data to a plurality of fully connected layers of the neural network to produce the medical image of the patient.

6 . The system of claim 5 , wherein the plurality of fully connected layers comprises:
a first hidden layer configured to operate on sensor data using matrix multiplication followed by an activation function; and a second hidden layer configured to produce a matrix from the first hidden layer, wherein the matrix has dimensions corresponding to dimensions of the sensor data.

7 . The system of claim 6 , wherein the plurality of fully connected layers further comprises:
an input layer configured to separate real components of the sensor data from imaginary components of the sensor data and to concatenate the real components and the imaginary components to produce an input vector; and wherein the first hidden layer is applied to the input vector.

8 . The system of claim 6 , wherein the activation function is a hyperbolic tangent activation function.

9 . The system of claim 4 , wherein the processor is further configured to transform the sensor data from the sensor domain to the image domain using the neural network by applying the sensor data to a plurality of convolutional layers of the neural network.

10 . The system of claim 9 , wherein the processor is further configured to transform the sensor data from the sensor domain to the image domain using the neural network by applying, with a deconvolutional layer of the neural network, a predetermined number of deconvolutional filters to a convolutional layer of the plurality of convolutional layers to produce the medical image of the patient.

11 . The system of claim 4 , wherein the image sensor includes at least one of:
a radio frequency (RF) system of a magnetic resonance imaging (MRI) system and wherein the sensor data comprises magnetic resonance data; an x-ray detector of a computed tomography (CT) system and wherein the sensor data comprises x-ray attenuation data; a gamma ray detector of an emission tomography system and wherein the sensor data comprises emission tomography data; an ultrasound transducer of a ultrasound system and wherein the sensor data comprises ultrasound data; and an optical sensor of an optical imaging system and wherein the sensor data comprises optical imaging data.

12 . A method comprising:
accessing, with a processor, sensor data acquired with an image sensor from a patient, wherein the sensor data is in a sensor domain; delivering, with the processor, the sensor data to a neural network trained for transforming the sensor data from the sensor domain to an image domain using the neural network to produce an image of the patient; and receiving, with the processor, the image of the patient from the neural network.

13 . The method of claim 12 , wherein executing instructions for transforming the sensor data from the sensor domain to the image domain using the neural network comprises executing instructions for applying a plurality of fully connected layers of the neural network to the sensor data to produce a matrix.

14 . The method of claim 13 , wherein applying the plurality of fully connected layers of the neural network to the sensor data to produce the matrix comprises:
separating, at an input layer of the plurality of fully connected layers, real components of the sensor data from imaginary components of the sensor data; concatenating, at the input layer, the real components and the imaginary components to produce an input vector; applying a first hidden layer of a plurality of fully connected layers to the input vector; and producing, with a second hidden layer of the plurality of fully connected layers, the matrix from the first hidden layer, wherein the matrix has dimensions corresponding to dimensions of the sensor data.

15 . The method of claim 13 , wherein applying the first hidden layer of the plurality of fully connected layers to the input vector comprises performing matrix multiplication on the input vector before applying a hyperbolic tangent activation function.

16 . The method of claim 13 , wherein executing instructions for transforming the sensor data from the sensor domain to the image domain using the neural network further comprises executing instructions for applying, with a plurality of convolutional layers of the neural network, a predetermined number of convolutional filters to the matrix.

17 . The method of claim 16 , wherein executing instructions for transforming the sensor data from the sensor domain to the image domain using the neural network further comprises executing instructions for applying, with a deconvolutional layer of the neural network, a predetermined number of deconvolutional filters to a convolutional layer of the plurality of convolutional layers to produce image data in the image domain that corresponds to the image of the patient.

18 . The method of claim 12 , wherein generating the sensor data comprises:
applying, with a magnetic resonance imaging system, a magnetic resonance pulse sequence to a sample; detecting, with the magnetic resonance imaging system, responsive magnetic resonance signals generated by the sample in response to the magnetic resonance pulse sequence; and sampling the responsive magnetic resonance signals to generate the sensor data.

19 . The method of claim 12 , wherein the sensor dat

Assignees

• Massachusetts Gen Hospital

Inventors

• Rosen Matthew S
• Zhu Bo
• Rosen Bruce R

Classifications

• G06T12/20

  Inverse problem, i.e. transformations from projection space into object space · CPC title

• A61B5/055

  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T2210/41

  Medical · CPC title