Method and system for cross-domain synthesis of medical images using contextual deep network

Patent metadata

Field	Value
Publication number	US-9892361-B2
Application number	US-201615003119-A
Country	US
Kind code	B2
Filing date	Jan 21, 2016
Priority date	Jan 21, 2015
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

A method and apparatus for cross-domain medical image synthesis is disclosed. A source domain medical image is received. A synthesized target domain medical image is generated using a trained contextual deep network (CtDN) to predict intensities of voxels of the target domain medical image based on intensities and contextual information of voxels in the source domain medical image. The contextual deep network is a multi-layer network in which hidden nodes of at least one layer of the contextual deep network are modeled as products of intensity responses and contextual response.

First claim

The invention claimed is:

1. A method for synthesizing a target domain medical image from a source domain medical image, comprising:
receiving the source domain medical image; and
generating a synthesized target domain medical image using a trained contextual deep network to predict intensities of voxels of the target domain medical image based on intensities and contextual information of voxels in the source domain medical image,
wherein the contextual deep network is a multi-layer network in which hidden nodes of at least one layer of the contextual deep network are modeled as products of intensity responses and contextual response, a response of each hidden node of the at least one layer of the contextual deep network is calculated as a product of a first function of the intensities of the voxels in the source domain medical image and a second function of the contextual information of the voxels in the source domain medical image, and the second function maps the contextual information of the voxels in the source domain medical image to a value of zero or one for each hidden node of the at least one layer of the contextual deep network.

2. The method of claim 1 , wherein the contextual information of the voxels in the source domain medical image is spatial locations of the voxels in the source domain medical image.

3. The method of claim 2 , wherein generating the synthesized target domain medical image using the trained contextual deep network to predict the intensities of the voxels of the target domain medical image based on the intensities and the contextual information of the voxels in the source domain medical image comprises:
for each of a plurality of image patches in the source domain medical image,
extracting intensities of voxels in the image patch and spatial coordinates of a center voxel in the image patch; and
calculating an intensity of a corresponding voxel in the target domain medical image based on the intensities of the voxels in the image patch of the source domain medical image and the spatial coordinates of the center voxel in the image patch of the source domain medical image using the trained contextual deep network, wherein the corresponding voxel in the target domain medical image is a voxel having the same spatial coordinates in the target domain medical image as the center voxel in the image patch of the source domain medical image.

4. The method of claim 1 , wherein the at least one layer of the contextual deep network comprises a second layer of the contextual deep network.

5. The method of claim 4 , wherein the contextual information of the voxels in the source domain medical image is spatial coordinates of the voxels in the source domain medical image.

6. The method of claim 5 , wherein the second function is

ς
⁡

(

x
;
θ

)

=

2
×

sigm
(

-

[



x
-

x
^

1



2

σ
2

,
…
⁢

,



x
-

x
^

p
2



2

σ
2

]

T

)

,

where x denotes the spatial coordinates of the voxels in the source domain medical image, σ is a predetermined constant, p 2 is a number of hidden nodes in the second layer of the contextual deep network, and {circumflex over (x)} 1 , . . . , {circumflex over (x)} p 2 are augmented variables associated with the hidden nodes of the second layer that are learned in training of the trained contextual deep network.

7. The method of claim 1 , wherein the target domain medical image is a segmentation mask image showing a segmented anatomical structure.

8. The method of claim 7 , wherein the source image is a computed tomography image, and generating the synthesized target domain medical image using the trained contextual deep network to predict the intensities of the voxels of the target domain medical image based on the intensities and the contextual information of the voxels in the source domain medical image comprises:
generating a segmentation mask image showing a segmented prostate using the trained contextual deep network to predict intensities of voxels of the segmentation mask image based on intensities and spatial locations of voxels in the source computed tomography image.

9. An apparatus for synthesizing a target domain medical image from a source domain medical image, comprising:
means for receiving the source domain medical image; and
means for generating a synthesized target domain medical image using a trained contextual deep network to predict intensities of voxels of the target domain medical image based on intensities and contextual information of voxels in the source domain medical image,
wherein the contextual deep network is a multi-layer network in which hidden nodes of at least one layer of the contextual deep network are modeled as products of intensity responses and contextual response, a response of each hidden node of the at least one layer of the contextual deep network

Assignees

• Siemens Healthcare Gmbh

Inventors

• Nguyen Hien
• Zhou Shaohua Kevin

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• A61B5/00

  Measuring for diagnostic purposes (radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00); Identification of persons · CPC title