Spatially consistent multi-scale anatomical landmark detection in incomplete 3D-CT data

Patent metadata

Field	Value
Publication number	US-10373313-B2
Application number	US-201715695112-A
Country	US
Kind code	B2
Filing date	Sep 5, 2017
Priority date	Mar 2, 2017
Publication date	Aug 6, 2019
Grant date	Aug 6, 2019

Abstract

A method and system for automated spatially-consistent multi-scale detection of anatomical landmarks in medical images is disclosed. A discrete scale-space representation of a medical image of a patient is generated. A plurality of anatomical landmarks are detected at a coarsest scale-level of the discrete scale-space representation of the medical image using a respective trained search model trained at the coarsest scale-level for each of the plurality of anatomical landmarks. Spatial coherence of the detected anatomical landmarks is enforced by fitting a learned robust shape model of the plurality of anatomical landmarks to the detected anatomical landmarks at the coarsest scale-level to robustly determine a set of the anatomical landmarks within a field-of-view of the medical image. The detected landmark location for each of the landmarks in the set of anatomical landmarks is refined at each remaining scale-level of the discrete scale-space representation of the medical image using, for each landmark, a respective trained search model trained at each remaining scale-level and constrained based on the predicted landmark location at a previous scale-level.

First claim

The invention claimed is:

1. A method for automated spatially-consistent detection of anatomical landmarks in a medical image of a patient, comprising:
generating a discrete scale-space representation of a medical image of a patient, wherein the discrete scale-space representation of the medical image includes a plurality of scale-levels;
detecting a plurality of anatomical landmarks at a coarsest scale-level of the discrete scale-space representation of the medical image using a respective trained search model trained to predict a trajectory from a starting location to a predicted landmark location at the coarsest scale-level for each of the plurality of anatomical landmarks;
enforcing spatial coherence of the detected anatomical landmarks by fitting a learned shape model of the plurality of anatomical landmarks to the detected anatomical landmarks at the coarsest scale-level to robustly determine a set of the anatomical landmarks within a field-of-view of the medical image; and
refining the predicted landmark location for each of the landmarks in the set of anatomical landmarks at each remaining scale-level of the discrete scale-space representation of the medical image using, for each landmark in the set of anatomical landmarks, a respective trained search model trained to predict a trajectory to the predicted landmark location at each remaining scale-level, wherein the trained search model for each remaining scale-level for each landmark is constrained based on a range surrounding the predicted landmark location for that landmark at a previous scale-level.

2. The method of claim 1 , wherein for each anatomical landmark, the trained search model for each scale-level is an intelligent artificial agent that predicts the trajectory to the predicted landmark location at the corresponding scale-level by iteratively using a trained deep neural network (DNN) that inputs a region of interest surrounding a current location in the corresponding scale-level of the discrete scale-space representation of the medical image to calculate action-values corresponding to actions that move the current location in different direction and selecting an action having the highest action-value to be applied to the current location.

3. The method of claim 2 , wherein the discrete scale-space representation of the medical image includes M scale-levels from 0 to M−1, where scale-level 0 of the discrete scale-space representation of the medical image is the medical image at its original resolution and scale-level M−1 is the coarsest scale level, and the trained search model for the coarsest scale level M−1 for each landmark is a global search model that starts the search for the trajectory to the predicted landmark location from the center of the coarsest scale-level of the discrete scale-space representation of the medical image.

4. The method of claim 3 , wherein the trained search model for the coarsest scale level M−1 for each landmark is trained to reward trajectories that leave an image space of a training image through a correct image border when the corresponding landmark is missing from the training image.

5. The method of claim 4 , wherein detecting a plurality of anatomical landmarks at a coarsest scale-level of the discrete scale-space representation of the medical image using a respective trained search model trained to predict a trajectory from a starting location to a predicted landmark location at the coarsest scale-level for each of the plurality of anatomical landmarks comprises:
for each of the plurality of anatomical landmarks, predicting a trajectory to a predicted landmark location in the coarsest scale-level of the discrete scale-space representation of the medical image or a trajectory that leaves the image space of the discrete scale-space representation of the medical image.

6. The method of claim 3 , wherein for each remaining scale-level t from t=M−2, . . . , 0, the trained search model for scale-level t for each anatomical landmark in the set of anatomical landmarks starts the search for the trajectory to the predicted landmark location from a convergence point of the trained search model for the previous scale level t+1 for that anatomical landmark.

7. The method of claim 1 , wherein enforcing spatial coherence of the detected anatomical landmarks by fitting a learned shape model of the plurality of anatomical landmarks to the detected anatomical landmarks at the coarsest scale-level to robustly determine a set of the anatomical landmarks within a field-of-view of the medical image comprises:
fitting the learned shape model of the plurality of anatomical landmarks to the detected anatomical landmarks at the coarsest scale-level using an M-estimator sampling consensus based on random 3-point samples from the detected anatomical landmarks at the coarsest scale-level.

8. The method of claim 1 , further comprising:
automatically determining a scan range of the medical image based on the set of anatomical landmarks at a final scale-level of the discrete scale-space representation of the medical image based on the learned shape model of the plurality of anatomical landmarks.

9. The method of claim 8 , wherein automatically determining a scan range of the medical image based on the set of anatomical landmarks at a final scale-level of the discrete scale-space representation of the medical image based on the learned shape model of the plurality of anatomical landmarks comprises:
interpolating scan range values between detected landmark locations in the in the final scale-level of the discrete scale-space representation of the medical image of a minimum anatomical landmark along a z-axis in the learned shape model and a maximum anatomical landmark along the z-axis in the learned shape model, wherein the detected landmark location of the minimum anatomical landmark defines a 0% point and the detected landmark location of the maximum anatomical landmark defines a 100% point; and
extrapolating scan range values above the detected landmark location of the maximum anatomical landmark or below the detected landmark location of the minimum anatomical landmark using an estimated scale parameter of the fitted shape model.

10. An apparatus for automated spatially-consistent detection of anatomical landmarks in a medical image of a patient, comprising:
means for generating a discrete scale-space representation of a medical image of a patient, wherein the discrete scale-space representation of the medical image includes a plurality of scale-levels;
means for detecting a plurality of anatomical landmarks at a coarsest scale-level of the discrete scale-space representation of the medical image using a respective trained search model trained to predict a trajectory from a starting location to a predicted landmark location at the coarsest scale-level for each of the plurality of anatomical landmarks;
means for enforcing spatial coherence of the detected anatomical landmarks by fitting a learned shape model of the plurality of anatomical landmarks to the detected anatomical landmarks at the coarsest scale-level to robustly determine a set of the anatomical landmarks within a field-of-view of the medical image; and
means for refining the predicted landmark location for each of the landmarks in the set of anatomical landmarks at each remaining scale-level of the discrete scale-space representation of the medical image using, for each landmark in the set of anatomical landmarks, a respective trained search model trained to predict a trajectory to the predicted landmark location at each remaining scale-level, wherein the trained search model for each remaining scale-level for each landmark is constrained based on a range surrounding the predicted

Assignees

• Siemens Healthcare Gmbh

Inventors

• Ghesu Florin Cristian
• Georgescu Bogdan
• Grbic Sasa
• Comaniciu Dorin

Classifications

• G06T7/75 Primary

  involving models · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06T2207/30004

  Biomedical image processing · CPC title

• G06K2209/051

  Physics · mapped topic

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title