Image fusion for interventional guidance

Patent metadata

Field	Value
Publication number	US-9681856-B2
Application number	US-201514970189-A
Country	US
Kind code	B2
Filing date	Dec 15, 2015
Priority date	Feb 23, 2012
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A method for real-time fusion of a 2D cardiac ultrasound image with a 2D cardiac fluoroscopic image includes acquiring real time synchronized US and fluoroscopic images, detecting a surface contour of an aortic valve in the 2D cardiac ultrasound (US) image relative to an US probe, detecting a pose of the US probe in the 2D cardiac fluoroscopic image, and using pose parameters of the US probe to transform the surface contour of the aortic valve from the 2D cardiac US image to the 2D cardiac fluoroscopic image.

First claim

What is claimed is:

1. A method of transforming target structure anatomies in a pre-operative image I 2 into an intra-operative image I 1 , comprising the steps of:
determining a transformation Φ aligns a target structure T 2 and an anchor structure A 2 in the pre-operative image I 2 into a corresponding target structure T 1 and anchor structure A 1 in the intra-operative image I 1 by finding a transformation {circumflex over (Φ)} that maximizes a functional log(P(Φ|I 1 ,A 2 )) using an expectation-maximization approach, wherein the target structure T 1 is not visible in the intra-operative image,
using the transformation {circumflex over (Φ)} to fuse the pre-operative image I 2 with the intra-operative image I 1 to create fused image wherein the target structure T 2 is visible in the fused image.

2. The method of claim 1 , wherein said transformation Φ is a rigid transformation, wherein an initial transformation Φ 0 is approximated as a translation, wherein Φ 0 represents a translation between a barycenter a 2 of the anchor anatomy A 2 in the pre-operative image I 2 and a detected barycenter a 1 of the anchor anatomy A 1 in the intra-operative image I 1 .

3. The method of claim 2 , wherein initial transformation Φ 0 is determined by a position detector trained by a probabilistic boosting tree classifier and Haar features on the barycenter a 1 of the anchor anatomy A 1 in the intra-operative image I 1 .

4. The method of claim 1 , wherein a pericardium is used as the anchor anatomy A 1 and A 2 and the aortic valve is used as the target anatomy T 1 and T 2 .

5. The method of claim 1 , wherein finding a transformation {circumflex over (Φ)} that maximizes a functional log(P(Φ|I 1 ,A 2 )) comprises:
generating K sample Φ i t point sets (x 1 , x 2 , x 3 , . . . , x K ) from the pre-operative anchor anatomy A 2 , wherein each point set comprises N points and each sample is represented as an isotropic 6D Gaussian distribution Φ i t =N 6 (μ i ,Σ i ), Σ i =σ i I, wherein I is an identity matrix and σ i is a one dimensional variable calculated as a kernel function from a probability map F(I) evaluated at the point locations y i,j , i=1, . . . , K, j=1, . . . , N;
transforming the point sets Φ i t into the intra-operative image I 1 locations y* i =Φ t (x i ), i=1, . . . , K, according to an appearance of the intra-operative image I 1 ,
assigning each point y* i,j , j=1, . . . , N from the point set x i to a new location y i,j , j=1, . . . , N, based on a local appearance of the intra-operative image I 1 ,
approximating final parameters of each sample Φ i t by an isotropic Gaussian distribution, wherein a mean μ is computed from a least squares solution between the point set Φ i t in the pre-operative I 2 and the updated point set (y 1 , y 2 , y 3 , . . . , y K ) in the intra-operative image I 1 by minimizing a mapping error function

e
i

=

1
N

⁢

∑

j
=
1

N

⁢

⁢



Φ
i
t

⁡

(

x

i
,
j

)

-

y

i
,
j



;

 and
determining an updated global transformation Φ t+1 from

Φ

t
+
1

=

arg
⁢

⁢
max

Φ

⁢

(

Φ
❘

Φ
t

)

based on an estimated mixture model

⊕

=

∑

i
=
1

K

⁢

⁢

Φ
i
t

 of the K transformation samples Φ i t , i=1, . . . , K.

6. The method of claim 5 , wherein

Φ

t
+
1

=

arg
⁢

⁢
max

Φ

⁢

(

Φ
❘

Φ
t

)

is estimated using a mean shift algorithm.

7. The method of claim 5 , further comprising deriving the probability map F(I 1 ) from the intra-operative image I 1 by evaluating a boosting classifier trained using Haar features and surface annotations of the anchor anatomy A 1 in the intra-operative image I 1 , wherein each vertex of a model of the intra-operative image I 1 is assigned as a positive sample and random points within a threshold distance are used as negative samples, and those vertices for which a feature response is low are rejected as positive examples.

Assignees

• Mountney Peter
• Grbic Sasa
• Ionasec Razvan Ioan
• John Matthias
• Comaniciu Dorin
• Siemens Ag

Inventors

• Mountney Peter
• Grbic Sasa
• Ionasec Razvan Ioan
• John Matthias
• Comaniciu Dorin

Classifications

• G06F18/24

  Classification techniques · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• A61B6/487

  involving fluoroscopy · CPC title

• G06T7/0034

  Physics · mapped topic

• G06T7/0028

  Physics · mapped topic