Registering a two-dimensional image with a three-dimensional image

Patent metadata

Field	Value
Publication number	US-11423554-B2
Application number	US-202016994943-A
Country	US
Kind code	B2
Filing date	Aug 17, 2020
Priority date	Aug 28, 2019
Publication date	Aug 23, 2022
Grant date	Aug 23, 2022

Abstract

The disclosure relates to a computer-implemented method for the provision of a transformation instruction for registering a 2D image with a 3D image. The method includes: receiving the 2D image and the 3D image; generating input data based on the 2D image having contour pixels and the 3D image having contour voxels; applying a trained function to the input data for identification of contour pixels of the 2D image and contour voxels of the 3D image, wherein at least one parameter of the trained function is adjusted based on a comparison of training contour pixels with comparison contour pixels and a comparison of training contour voxels corresponding thereto with comparison contour voxels; determining the transformation instruction based on the identified contour pixels of the 2D image and the contour voxels corresponding thereto of the 3D image for registering the 2D image with the 3D image; and providing the transformation instruction.

First claim

The invention claimed is:

1. A computer-implemented method for provision of a transformation instruction for registering a two-dimensional (2D) image with a three-dimensional (3D) image, the method comprising:
receiving the 2D image and the 3D image;
generating input data based on the 2D image comprising contour pixels and the 3D image comprising contour voxels, wherein the contour pixels are dependent on contour features of the 2D image, wherein the contour voxels are dependent on contour features of the 3D image, and wherein the contour voxels of the 3D image have a substantially perpendicular contour surface normal in respect of a specified projection direction;
applying a trained function to the input data for identification of contour pixels of the 2D image and contour voxels of the 3D image, which correspond with each other, wherein at least one parameter of the trained function is adjusted based on a comparison of training contour pixels with comparison contour pixels and a comparison of training contour voxels corresponding thereto with comparison contour voxels; and
determining the transformation instruction based on the identified contour pixels of the 2D image and the contour voxels corresponding thereto of the 3D image for registering the 2D image with the 3D image; and
providing the transformation instruction.

2. The computer-implemented method of claim 1 , wherein the 2D image comprises a projection image of an examination region of an examination object depicted in the 3D image, and
wherein the specified projection direction is specified as a function of the projection image.

3. The computer-implemented method of claim 1 , further comprising:
adjusting the specified projection direction as a function of the transformation instruction; and
repeating the generating of the input data, the applying of the trained function, the determining of the transformation instruction, and the adjusting of the specified projection direction until an occurrence of a termination condition.

4. The computer-implemented method of claim 1 , wherein the generating of the input data further comprises creating a gradient image from the 3D image by applying a projection mapping, and
wherein the input data is also based on the gradient image.

5. The computer-implemented method of claim 4 , wherein the generating of the input data further comprises creating a second gradient image from the 2D image, and
wherein the input data is also based on the second gradient image.

6. The computer-implemented method of claim 1 , further comprising:
receiving a movement parameter,
wherein the contour features of the 2D image are adjusted in the generating of the input data using the movement parameter.

7. The computer-implemented method of claim 1 , further comprising:
registering the 2D image with the 3D image based on the transformation instruction; and
providing the registered 3D image.

8. A computer-implemented method for provision of a trained function, the method comprising:
receiving a two-dimensional (2D) training image and a three-dimensional (3D) training image;
identifying comparison contour voxels of the 3D training image and comparison contour pixels of the 2D training image, which correspond with each other, by: identification of contour features of the 3D training image; identification of contour features of the 2D training image; selection of the contour features of the 2D training image and the contour features of the 3D training image, which correspond with each other; and selection of the comparison contour pixels of the 2D training image and the comparison contour voxels of the 3D training image as a function of the selected contour features of the 2D training image and the selected contour features of the 3D training image;
generating input data based on the 2D training image comprising training contour pixels and the 3D training image comprising training contour voxels, wherein the training contour pixels are dependent on training contour features of the 2D training image, wherein the training contour voxels are dependent on training contour features of the 3D training image, which training contour voxels of the 3D training image have a substantially perpendicular contour surface normal in respect of a specified projection direction;
applying the trained function to the input data for identification of training contour pixels of the 2D training image and training contour voxels of the 3D training image, which correspond with each other;
adjusting at least one parameter of the trained function based on a comparison of the training contour pixel with the comparison contour pixels and a comparison of training contour voxels corresponding thereto with the comparison contour voxels; and
providing the trained function.

9. The computer-implemented method of claim 8 , wherein the 2D training image comprises a projection image of an examination region depicted in the 3D training image, wherein the specified projection direction is specified as a function of the projection image.

10. The computer-implemented method of claim 8 , wherein the generating of the input data further comprises generation of a training gradient image from the 3D training image by applying a projection mapping, and
wherein the input data is also based on the training gradient image.

11. A computer-implemented method of claim 10 , wherein the generating of the input data further comprises generation of a second training gradient image from the 2D training image, and
wherein the input data is also based on the second training gradient image.

12. The computer-implemented method of claim 8 , further comprising:
receiving a training movement parameter,
wherein the contour features of the 2D training image and the training contour features of the 2D training image are adjusted using the training movement parameter.

13. A device for provision of a transformation instruction for registering a two-dimensional ( 2 D) image with a three-dimensional ( 3 D) image, the device comprising:
a processor,
wherein the processor is configured to receive the 2 D image and the 3 D image,
wherein the processor is further configured to generate input data based on the 2 D image comprising contour pixels and the 3 D image comprising contour voxels, wherein the contour pixels are dependent on contour features of the 2 D image, wherein the contour voxels are dependent on contour features of the 3 D image, which contour voxels of the 3 D image have a substantially perpendicular contour surface normal in respect of a specified projection direction,
wherein the processor is further configured to apply a trained function to the input data for identification of contour pixels of the 2 D image and contour voxels of the 3 D image, which correspond with each other, wherein at least one parameter of the trained function is configured to be adjusted based on a comparison of training contour pixels with comparison contour pixels and a comparison of training contour voxels corresponding thereto with comparison contour voxels,
wherein the processor is further configured to determine a transformation instruction based on the identified contour pixels of the 2 D image and the contour voxels corresponding thereto of the 3 D image for registering the 2 D image with the 3 D image, and
wherein the processor is further configured to provide the transformation instruction.

14. The device of claim 13 , wherein the device is a medical imaging device, and wherein the 2D image and the 3D image are images of an examination object.

15. A training unit for provis

Assignees

• Siemens Healthcare Gmbh

Inventors

• Borsdorf Anja
• Schaffert Roman
• Wang Jian
• Weiss Markus

Classifications

• G06T2207/10116

  X-ray image · CPC title

• G06T7/33 Primary

  using feature-based methods · CPC title

• G06T2207/10081

  Computed x-ray tomography [CT] · CPC title

• G06T2207/10088

  Magnetic resonance imaging [MRI] · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title