Determining a transformation between coordinate frames of sets of image data

Patent metadata

Field	Value
Publication number	US-10929989-B2
Application number	US-201816173853-A
Country	US
Kind code	B2
Filing date	Oct 29, 2018
Priority date	Nov 1, 2017
Publication date	Feb 23, 2021
Grant date	Feb 23, 2021

Abstract

The disclosure relates to a method of determining a transformation between coordinate frames of sets of image data. The method includes receiving a model of a structure extracted from first source image data, the first source image data being generated according to a first imaging modality and having a first data format, wherein the model has a second data format, different from the first data format. The method also includes determining, using an intelligent agent, a transformation between coordinate frames of the model and first target image data, the first target image data being generated according to a second imaging modality different to the first imaging modality.

First claim

The invention claimed is:

1. A method of determining a transformation between coordinate frames of sets of image data, the method comprising:
receiving a model of a structure extracted from first source image data, the first source image data being generated according to a first imaging modality and having a first data format, wherein the model has a second data format, different from the first data format;
determining, using an intelligent agent, a transformation between coordinate frames of the model and first target image data, the first target image data being generated according to a second imaging modality different from the first imaging modality;
generating projection image data based on a two-dimensional projection of the model;
determining, by the intelligent agent, a reward for each of a plurality of actions applicable to the projection image data;
selecting an action based on the determined rewards; and
transforming the projection image data according to the selected action.

2. The method of claim 1 , further comprising:
receiving a further model of a structure extracted from second source image data, the second source image data being generated according to a third imaging modality, different from the first modality, and having a third data format different from the first data format, wherein the further model has the second data format; and
determining, using the intelligent agent, a transformation between coordinate frames of the further model and second target image data, the second target image data being generated according to a fourth imaging modality different from the first imaging modality.

3. The method of claim 1 , further comprising:
segmenting the first source image data to extract the model.

4. The method of claim 3 , further comprising:
generating a polygon mesh based on the segmented first source image data.

5. The method of claim 1 , wherein the first source image data comprises three-dimensional image data and the first target image data comprises two-dimensional image data.

6. The method of claim 1 , wherein the transforming of the projection image data comprises:
applying the selected action to the model to generate a transformed model; and
generating further projection image data based on a two-dimensional projection of the transformed model.

7. The method of claim 1 , wherein the reward for each action of the plurality of actions is determined based on a translation of the projection image data, a rotation of the projection image data, or a combination thereof.

8. The method of claim 1 , further comprising:
specifying imaging geometry associated with the first target image data; and
generating the projection image data in accordance with the specified imaging geometry.

9. The method of claim 1 , wherein the first source image data comprises one of: magnetic resonance image data, computed tomography image data, and ultrasound image data, or
wherein the first target image data comprises X-ray image data.

10. The method of claim 1 , further comprising:
performing a training process to train the intelligent agent, the training process comprising:
providing training data to the intelligent agent, the training data comprising three-dimensional image data;
processing the training data to extract a three-dimensional model of a structure from the training data and to generate digitally reconstructed two-dimensional image data;
generating a two-dimensional projection of the extracted three-dimensional model;
determining, by the intelligent agent, a reward for each of a plurality of actions applicable to the two-dimensional projection;
selecting an action based on the determined rewards; and

transforming the extracted three-dimensional model according to the selected action.

11. The method of claim 10 , wherein the two-dimensional projection has a field of view that is smaller than a field of view of the digitally reconstructed two-dimensional image data.

12. The method of claim 10 , wherein the training data comprises computed tomography image data.

13. A medical imaging device comprising:
a processor configured to:
receive a model of a structure extracted from first source image data, the first source image data being generated according to a first imaging modality and having a first data format, wherein the model has a second data format, different from the first data format;
determine, using an intelligent agent, a transformation between coordinate frames of the model and first target image data, the first target image data being generated according to a second imaging modality different from the first imaging modality;
generate projection image data based on a two-dimensional projection of the model;
determine a reward for each of a plurality of actions applicable to the projection image data;
select an action based on the determined rewards; and
transform the projection image data according to the selected action.

14. The medical imaging device of claim 13 , further comprising:
a display unit,
wherein the processor is configured to:
generate an overlay based on the transformation between coordinate frames of the model and the first target image data;
apply the overlay to the target image data to generate an annotated image; and
display the annotated image on the display unit.

15. A non-transitory computer program product comprising a computer program, the computer program being loadable into a memory unit of a data processing system, the computer program comprising program code sections, that when executed by a processor of the data processing system, cause the data processing system to:
receive a model of a structure extracted from first source image data, the first source image data being generated according to a first imaging modality and having a first data format, wherein the model has a second data format, different from the first data format;
determine a transformation between coordinate frames of the model and first target image data, the first target image data being generated according to a second imaging modality different from the first imaging modality;
generate projection image data based on a two-dimensional projection of the model;
determine a reward for each of a plurality of actions applicable to the projection image data;
select an action based on the determined rewards; and
transform the projection image data according to the selected action.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Kurzendorfer Tanja
• Liao Rui
• Mansi Tommaso
• Miao Shun
• Mountney Peter
• Toth Daniel

Classifications

• G06T2207/30004

  Biomedical image processing · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/344 Primary

  involving models · CPC title

• G06T7/30

  Determination of transform parameters for the alignment of images, i.e. image registration · CPC title