Method and system for real time stent enhancement on live 2D fluoroscopic scene

Patent metadata

Field	Value
Publication number	US-9082158-B2
Application number	US-201313759170-A
Country	US
Kind code	B2
Filing date	Feb 5, 2013
Priority date	Sep 19, 2012
Publication date	Jul 14, 2015
Grant date	Jul 14, 2015

Abstract

A method and system for real time stent enhancement on a live 2D fluoroscopic scene is disclosed. A motion compensated stent enhancement image is generated from a first set of frames in a fluoroscopic image sequence. A weighting field is generated based on the motion compensated stent enhancement image. For each new frame in the fluoroscopic image sequence that is received, the stent is enhanced in the new frame by compounding the new frame with the motion compensated stent enhancement image using the weighting field.

First claim

The invention claimed is:

1. A method of stent enhancement in a fluoroscopic image sequence, comprising:
generating a motion compensated stent enhancement image from a first set of frames in a fluoroscopic image sequence;
generating a weighting field based on the motion compensated stent enhancement image;
receiving a new frame in the fluoroscopic image sequence; and
enhancing a stent in the new frame by compounding the new frame with the motion compensated stent enhancement image using the weighting field.

2. The method of claim 1 , wherein the generating a motion compensated stent enhancement image from a first set of frames in a fluoroscopic image sequence comprises:
detecting a pair of balloon markers in each frame of the first set of frames in the fluoroscopic image sequence;
aligning the first set of frames based on the detected pair of balloon markers in each frame of the first set of frames; and
generating the motion compensated stent enhancement image as a composite image of the aligned first set of frames.

3. The method of claim 2 , wherein the detecting a pair of balloon markers in each frame of the first set of frames in the fluoroscopic image sequence comprises:
detecting individual balloon marker candidates in each frame of the first set of frames using a trained balloon marker detector;
determining a number of balloon marker pair candidates in each frame from the detected individual balloon marker candidates in each frame;
detecting, in a respective hypothesis space defined by each respective balloon marker pair candidate in each frame, a spline representing a guidewire using a trained guidewire classifier; and
selecting one of the balloon marker pair candidates in each frame based on probability scores from the trained balloon marker detector and the trained guidewire classifier.

4. The method of claim 2 , wherein the aligning the first set of frames based on the detected pair of balloon markers in each frame of the first set of frames comprises:
calculating a consistency score of each frame in the first set of frames based on the detected pair of balloon markers in each frame of the first set of frames;
selecting a reference frame of the first set of frames based on the consistency score and a detection probability of the detected pair of balloon markers in each frame of the first set of frames;
for each frame of the first set of frames other than the reference frame, calculating a distance measure between the detected pair of balloon markers in that frame and the detected pair of balloon markers in the reference frame; and
removing a frame from the first set of frames if the distance measure calculated for the frame is greater than a threshold; and
registering each remaining frame of the first set of frames to the reference frame based on the detected balloon markers in that frame and the detected balloon markers in the reference frame.

5. The method of claim 2 , wherein the generating a weighting field based on the motion compensated stent enhancement image comprises:
detecting a stent region of interest in the motion compensated stent enhancement image;
detecting a guidewire in the motion compensated stent enhancement image; and
generating the weighting field by applying a Gaussian kernel to points along the detected guidewire in the detected stent region of interest in the motion compensated stent enhancement image.

6. The method of claim 5 , wherein the detecting a stent region of interest in the motion compensated stent enhancement image comprises:
detecting a bounding box of a stent in the motion compensated stent enhancement image using marginal space learning based detection.

7. The method of claim 5 , wherein the detecting a guidewire in the motion compensated stent enhancement image comprises:
detecting a guidewire in a hypothesis space defined by the pair of balloon markers in the motion compensated stent enhancement image using a trained guidewire classifier.

8. The method of claim 5 , wherein the enhancing a stent in the new frame by compounding the new frame with the motion compensated stent enhancement image using the weighting field comprises:
detecting a pair of balloon markers in the new frame;
aligning the new frame and the motion compensated stent enhancement image based on the detected pair of balloon markers in the new frame and the pair of balloon markers in the motion compensated stent enhancement image; and
generating a compound image that is a weighted combination of the aligned new frame and the motion compensated stent enhancement image weighted based on the weighting field.

9. The method of claim 8 , wherein the generating a compound image that is a weighted combination of the aligned new frame and the motion compensated stent enhancement image weighted based on the weighting field comprises:
generating the compound image O as O=I s ×W+I×(1−W), where I s is the motion compensated stent enhancement image, I is the aligned new frame, and W is the weighting field.

10. The method of claim 1 , further comprising:
repeating the steps of receiving a new frame and enhancing a stent in the new frame for each of a plurality of new frames in the fluoroscopic image sequence.

11. The method of claim 10 , wherein the step of enhancing a stent in the new frame is performed in real time for each of the plurality of new frames as each of the plurality of new frames is received.

12. The method of claim 1 , wherein the first set of frames comprises a first number of frames in the fluoroscopic image sequence.

13. The method of claim 12 , wherein the first set of frames comprises a first 30 frames in the fluoroscopic image sequence.

14. An apparatus for stent enhancement in a fluoroscopic image sequence, comprising:
means for generating a motion compensated stent enhancement image from a first set of frames in a fluoroscopic image sequence;
means for generating a weighting field based on the motion compensated stent enhancement image; and
means for compounding a new frame of the fluoroscopic image sequence with the motion compensated stent enhancement image using the weighting field.

15. The apparatus of claim 14 , wherein the means for generating a motion compensated stent enhancement image from a first set of frames in a fluoroscopic image sequence comprises:
means for detecting a pair of balloon markers in each frame of the first set of frames in the fluoroscopic image sequence;
means for aligning the first set of frames based on the detected pair of balloon markers in each frame of the first set of frames; and
means for generating the motion compensated stent enhancement image as a composite image of the aligned first set of frames.

16. The apparatus of claim 15 , wherein the means for generating a weighting field based on the motion compensated stent enhancement image comprises:
means for detecting a stent region of interest in the motion compensated stent enhancement image;
means for detecting a guidewire in the motion compensated stent enhancement image; and
means for generating the weighting field from points along the detected guidewire in the detected stent region of interest in the motion compensated stent enhancement image.

17. The apparatus of claim 15 , wherein the means for compounding a new frame of the fluoroscopic image sequence with the motion compensated stent enhancement image using the weighting field comprises:
means for detecting a pair of balloon markers in the new frame;
means for aligning the new frame and the motion compensated stent enhancement image based on the det

Assignees

• Chen Terrence
• Durlak Peter
• Rossmeier Markus
• Comaniciu Dorin
• Siemens Ag

Inventors

• Chen Terrence
• Durlak Peter
• Rossmeier Markus
• Comaniciu Dorin

Classifications

• G06T5/60

  using machine learning, e.g. neural networks · CPC title

• G06T5/73 Primary

  Deblurring; Sharpening · CPC title

• G06T2207/30204

  Marker · CPC title

• G06T2207/30048

  Heart; Cardiac · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title