Method and System for Anatomical Object Detection Using Marginal Space Deep Neural Networks

Patent metadata

Field	Value
Publication number	US-2016174902-A1
Application number	US-201615055161-A
Country	US
Kind code	A1
Filing date	Feb 26, 2016
Priority date	Oct 17, 2013
Publication date	Jun 23, 2016
Grant date	—

Abstract

A method and system for anatomical object detection using marginal space deep neural networks is disclosed. The pose parameter space for an anatomical object is divided into a series of marginal search spaces with increasing dimensionality. A respective sparse deep neural network is trained for each of the marginal search spaces, resulting in a series of trained sparse deep neural networks. Each of the trained sparse deep neural networks is trained by injecting sparsity into a deep neural network by removing filter weights of the deep neural network.

First claim

1 . A method for anatomical object detection in a 3D medical image of a patient, comprising:
receiving a 3D medical image of a patient including a target anatomical object; and detecting a 3D pose of the target anatomical object in the 3D medical image in a series of marginal parameter spaces of increasing dimensionality using a respective trained sparse deep neural network for each of the marginal search spaces.

2 . The method of claim 1 , wherein each respective trained sparse deep neural network inputs hypothesis image patches in the respective marginal parameter space, samples voxels in each hypothesis image patch in a sparse sampling pattern adaptively learned during training of the respective trained sparse deep neural network, and calculates, for each hypothesis image patch, a probability that the hypothesis image patch is an image patch of the target anatomical object in the respective marginal parameter space based on the voxels sampled in the hypothesis image patch in the sparse sampling pattern.

3 . The method of claim 1 , wherein at least one of the respective trained sparse deep neural networks is trained by:
training a deep neural network by learning parameters of a respective filter for each of a plurality of neurons in each of a plurality of layers of the deep neural network, wherein the parameters of each respective filter include a bias and a plurality of weights that weight node outputs of the nodes of a previous one of the plurality of layers; and sparsifying the weights of the filters for at least one layer of the trained deep neural network.

4 . The method of claim 3 , wherein sparsifying the weights of the filters for at least one layer of the trained deep neural network comprises:
reducing a number of non-zero weights in each filter for the at least one layer in the trained deep neural network by setting a predetermined percentage of non-zero weights with lowest magnitudes in each filter equal to zero; and retraining the deep neural network with only the remaining non-zero weights in each filter for the at least one layer to refine the bias and the remaining non-zero weights in each filter for the at least one layer to recover from an effect of reducing the number of non-zero weights.

5 . The method of claim 4 , wherein sparsifying the weights of the filters for at least one layer of the trained deep neural network further comprises:
repeating the reducing and retraining steps in each of a plurality of iterations.

6 . The method of claim 5 , wherein sparsifying the weights of the filters for at least one layer of the trained deep neural network further comprises:
performing, in each iteration, a normalization to preserve an L 1 -norm for each filter for the at least one layer.

7 . The method of claim 5 , wherein repeating the reducing and retraining steps in each of a plurality of iterations comprises:
repeating the reducing and retraining steps until a target sparsity degree is reached.

8 . The method of claim 3 , wherein the at least one layer is a first hidden layer of the deep neural network.

9 . The method of claim 3 , wherein sparsifying the weights of the filters for at least one layer of the trained deep neural network comprises:
performing re-weighted L1-norm regularization on the weights of the filters for the at least one layer of the trained deep neural network, wherein the re-weighted L1-norm regularization drives a plurality of non-zero weights of the filters for the at least one layer to zero; and training a sparse deep neural with only the remaining non-zero weights in the filters for the at least one layer.

10 . The method of claim 1 , wherein detecting a 3D pose of the target anatomical object in the 3D medical image in a series of marginal parameter spaces of increasing dimensionality using a respective trained sparse deep neural network for each of the marginal search spaces comprises, for each of the marginal parameter spaces:
filtering hypothesis image patches in that marginal parameter space using a respective cascade of trained shallow sparse neural networks to eliminate a plurality of negative hypothesis image patches; and evaluating each of the remaining hypothesis image patches in that marginal parameter space using the respective trained sparse deep neural network to determine a classification score for each of the remaining hypothesis image patches in that marginal parameter space.

11 . The method of claim 10 , wherein the respective cascade of trained shallow sparse neural networks for each marginal parameter space is trained by sequentially adding shallow sparse neural networks to the cascaded until a balanced set of positive and negative training samples is achieved for training the respective sparse deep neural network.

12 . The method of claim 1 , further comprising:
rigidly registering a mean model of the target anatomical object to the 3D medical mage using the detected 3D pose of the target anatomical object in the 3D medical image; refining the model of the target anatomical structure in the 3D medical image using a trained sparse deep neural network boundary detector applied to image patches corresponding to sampling points along a line normal to a surface of the model at each point on the surface of the model; and projecting the refined model of the target anatomical object in the 3D medical image to a learned shape space of the target anatomical object.

13 . The method of claim 1 , wherein the series of marginal parameter spaces comprises a position parameter space, a position-orientation parameter space, and a position-orientation-scale parameter space.

14 . The method of claim 13 , wherein detecting a 3D pose of the target anatomical object in the 3D medical image in a series of marginal parameter spaces of increasing dimensionality using a respective trained sparse deep neural network for each of the marginal search spaces comprises:
detecting position candidates in the 3D medical image using a first cascade of trained sparse shallow neural networks to filter image patches centered at each of a plurality of voxels in the medical image and a first trained sparse deep neural network to evaluate image patches not eliminated by the first cascade; generating position-orientation hypotheses based on the detected position candidates; detecting position-orientation candidates in the 3D medical image using a second cascade of trained sparse shallow neural networks to filter image patches corresponding to the position-orientation hypotheses and a second trained sparse deep neural network to evaluate image patches corresponding position-orientation hypotheses not eliminated by the second cascade; generating position-orientation-scale hypotheses based on the detected position-orientation candidates; and detecting the pose of the target anatomical object in the 3D medical image using a third cascade of trained sparse shallow neural networks to filter image patches corresponding to the position-orientation-scale hypotheses and a third trained sparse deep neural network to evaluate image patches corresponding position-orientation-scale hypotheses not eliminated by the third cascade.

15 . A method of automated anatomical landmark detection in a 3D medical image, comprising:
detecting a plurality of landmark candidates for a target anatomical image in the 3D medical image using an initial shallow neural network detector; calculating deeply learned features for each of the plurality of landmark candidates using a trained deep neural network; and detecting the target anatomical landmark in the 3D medical image from the plurality

Assignees

• Siemens Ag

Inventors

• Georgescu Bogdan
• Zheng Yefeng
• Hguyen Hien
• Singh Vivek Kumar
• Comaniciu Dorin
• Liu David

Classifications

• G06T2207/20024

  Filtering details · CPC title

• A61B8/0883

  for diagnosis of the heart · CPC title

• G06T2207/10072

  Tomographic images · CPC title

• A61B8/565

  involving data transmission via a network · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title