Adaptive nonlinear optimization of shape parameters for object localization in 3D medical images

Patent metadata

Field	Value
Publication number	US-10846875-B2
Application number	US-201916270918-A
Country	US
Kind code	B2
Filing date	Feb 8, 2019
Priority date	Jun 7, 2018
Publication date	Nov 24, 2020
Grant date	Nov 24, 2020

Abstract

System and methods are provided for localizing a target object in a medical image. The medical image is discretized into a plurality of images having different resolutions. For each respective image of the plurality of images, starting from a first image and progressing to a last image with the progression increasing in resolution, a sequence of actions is performed for modifying parameters of a target object in the respective image. The parameters of the target object comprise nonlinear parameters of the target object. The sequence of actions is determined by an artificial intelligence agent trained for a resolution of the respective image to optimize a reward function. The target object is localized in the medical image based on the modified parameters of the target object in the last image.

First claim

The invention claimed is:

1. A method for localizing a target object in a medical image, comprising:
discretizing the medical image into a plurality of images having different resolutions;
for each respective image of the plurality of images, starting from a first image and progressing to a last image with the progression increasing in resolution, performing a sequence of actions for modifying parameters of the target object in the respective image, the parameters of the target object comprising nonlinear parameters of the target object, wherein each of the sequences of actions is determined to optimize a reward function by an artificial intelligence (AI) agent of a plurality of AI agents each separately trained for a corresponding one of the resolutions of the plurality of images; and
localizing the target object in the medical image based on the modified parameters of the target object in the last image.

2. The method of claim 1 , wherein the parameters of the target object comprise translation, rotation, and scaling parameters defining a nine dimensional space.

3. The method of claim 1 , wherein the AI agent is trained using deep reinforcement learning.

4. The method of claim 1 , wherein the sequence of actions comprise a stop action in which the parameters of the target object are unchanged.

5. The method of claim 1 , wherein the modified parameters of the target object in the respective image are used as initial parameters for the target object in a next image in the plurality of images.

6. The method of claim 1 , wherein performing a sequence of actions for modifying parameters of the target object in the respective image comprises:
repeatedly performing an action for modifying the parameters of the target object for a current state in the respective image that optimizes the reward function learned by the AI agent trained for the resolution of the respective image until a stopping condition is satisfied.

7. The method of claim 6 , wherein the stopping condition comprises one of a stop action determined by the AI agent, a predetermined number of steps, and consecutive complementary actions.

8. The method of claim 1 , wherein the target object is an anatomical landmark.

9. An apparatus for localizing a target object in a medical image, comprising:
means for discretizing the medical image into a plurality of images having different resolutions;
means for, for each respective image of the plurality of images, starting from a first image and progressing to a last image with the progression increasing in resolution, performing a sequence of actions for modifying parameters of the target object in the respective image, the parameters of the target object comprising nonlinear parameters of the target object, wherein each of the sequences of actions is determined to optimize a reward function by an artificial intelligence (AI) agent of a plurality of AI agents each separately trained for a corresponding one of the resolutions of the plurality of images; and
means for localizing the target object in the medical image based on the modified parameters of the target object in the last image.

10. The apparatus of claim 9 , wherein the parameters of the target object comprise translation, rotation, and scaling parameters defining a nine dimensional space.

11. The apparatus of claim 9 , wherein the AI agent is trained using deep reinforcement learning.

12. The apparatus of claim 9 , wherein the modified parameters of the target object in the respective image are used as initial parameters for the target object in a next image in the plurality of images.

13. The apparatus of claim 9 , wherein the means for performing a sequence of actions for modifying parameters of the target object in the respective image comprises:
means for repeatedly performing an action for modifying the parameters of the target object for a current state in the respective image that optimizes the reward function learned by the AI agent trained for the resolution of the respective image until a stopping condition is satisfied.

14. The apparatus of claim 13 , wherein the stopping condition comprises one of a stop action determined by the AI agent, a predetermined number of steps, and consecutive complementary actions.

15. A non-transitory computer readable medium storing computer program instructions for localizing a target object in a medical image, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
discretizing the medical image into a plurality of images having different resolutions;
for each respective image of the plurality of images, starting from a first image and progressing to a last image with the progression increasing in resolution, performing a sequence of actions for modifying parameters of the target object in the respective image, the parameters of the target object comprising nonlinear parameters of the target object, wherein each of the sequences of actions is determined to optimize a reward function by an artificial intelligence (AI) agent of a plurality of AI agents each separately trained for a corresponding one of the resolutions of the plurality of images; and
localizing the target object in the medical image based on the modified parameters of the target object in the last image.

16. The non-transitory computer readable medium of claim 15 , wherein the parameters of the target object comprise translation, rotation, and scaling parameters defining a nine dimensional space.

17. The non-transitory computer readable medium of claim 15 , wherein the AI agent is trained using deep reinforcement learning.

18. The non-transitory computer readable medium of claim 15 , wherein the sequence of actions comprise a stop action in which the parameters of the target object are unchanged.

19. The non-transitory computer readable medium of claim 15 , wherein the modified parameters of the target object in the respective image are used as initial parameters for the target object in a next image in the plurality of images.

20. The non-transitory computer readable medium of claim 15 , wherein the target object is an anatomical landmark.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Etcheverry Mayalen Irene Catherine
• Georgescu Bogdan
• Grbic Sasa
• Comaniciu Dorin
• Odry Benjamin L
• Re Thomas
• Kaushik Shivam
• Geiger Bernhard
• Nadar Mariappan S

Classifications

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T3/20

  Linear translation of whole images or parts thereof, e.g. panning · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title