Three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging

Patent metadata

Field	Value
Publication number	US-11534136-B2
Application number	US-201816130320-A
Country	US
Kind code	B2
Filing date	Sep 13, 2018
Priority date	Feb 26, 2018
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

For three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging, the three-dimension segmentation is output by a machine-learnt multi-task generator. The machine-learnt multi-task generator is trained from 3D information, such as a sparse ICE volume assembled from the 2D ICE images. The machine-learnt multi-task generator is trained to output both the 3D segmentation and a complete volume. The 3D segmentation may be used to project to 2D as an input with an ICE image to another network trained to output a 2D segmentation for the ICE image. Display of the 3D segmentation and/or 2D segmentation may guide ablation of tissue in the patient.

First claim

What is claimed is:

1. A method for three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging in a medical imaging system, the method comprising:
sensing positions of scan planes within a cardiac system of a patient for the intracardiac echocardiography (ICE) imaging;
forming an ICE volume from ultrasound data for the scan planes from the positions;
generating the three-dimensional segmentation from input of the ICE volume to a single machine-learned multi-task generator having been trained adversarialy with first and second discriminators, the first discriminator configured to judge a volume completion output from the single machine-learned multi-task generator and the second discriminator configured to judge a three-dimensional segmentation output from the single machine-learned multi-task generator; and
displaying an image of the three-dimensional segmentation.

2. The method of claim 1 wherein forming the ICE volume comprises mapping the ultrasound data to three dimensions using the positions of the scan planes.

3. The method of claim 1 wherein generating the three-dimensional segmentation comprises generating boundaries for two or more heart structures.

4. The method of claim 1 wherein generating comprises generating with the single machine-learned multi-task generator generating volume data less sparse than the ICE volume.

5. The method of claim 4 wherein displaying comprises displaying the image of the three-dimensional segmentation and the volume data.

6. The method of claim 4 wherein generating comprises generating with the single machine-learned multi-task generator having been trained with an adversarial loss with ground truth volumes from a computed tomography or magnetic resonance scan.

7. The method of claim 6 wherein generating comprises generating with the single machine-learned multi-task generator having been trained with the adversarial loss and a reconstruction loss of error per location for the volume data and the three-dimensional segmentation.

8. The method of claim 1 wherein generating comprises generating with the single machine-learned multi-task generator including downsampling and upsampling blocks with skip connections.

9. The method of claim 1 further comprising:
projecting the three-dimensional segmentation to a plane;
generating a two-dimensional segmentation for an ICE image by input of the projection of the three-dimensional segmentation to the plane and input of the ICE image to a machine-learned network; and
displaying the ICE image with the two-dimensional segmentation.

10. The method of claim 9 wherein the machine-learned network comprises a generator of a conditional generative adversarial network.

11. The method of claim 1 wherein displaying comprises displaying the three-dimensional segmentation as a segmentation of a left atrium or ventricle.

12. The method of claim 11 further comprising performing ablation for atrial fibrillation with the image of the three-dimensional segmentation as a guide.

13. A method for machine training of three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging, the method comprising:
defining a multi-task generative adversarial network with output layers for volume completion and three-dimensional segmentation, the multi-task generative adversarial network including a single generator;
assigning computed tomography or magnetic resonance volumes from a first group of patients as ground truth volumes for ultrasound imaging of a second group of patients;
machine training, using the assigned ground truth volumes, the multi-task generative adversarial network to generate first volumes and three-dimensional segments from input ultrasound volumes assembled from planar images, wherein the machine training comprises machine training with the multi-task generative adversarial network comprising the single generator and first and second discriminators, the first discriminator receiving a volume completion output and the second discriminator receiving a three-dimensional segmentation output; and
storing the single generator of the machine-trained multi-task generative adversarial network.

14. The method of claim 13 wherein assigning comprises pairing one of the computed tomography or magnetic resonance volumes to each sample intracardiac echocardiography volume of the second group of patients based on similarity.

15. The method of claim 13 wherein machine training comprising machine training with a combination of adversarial and reconstruction losses.

16. The method of claim 13 further comprising machine training another network based on input of the planar images and projections from the three-dimensional segments to output two-dimensional segments for the planar images.

17. A medical imaging system for three-dimensional segmentation from two-dimensional intracardiac echocardiography imaging in ablation, the medical imaging system comprising:
an intracardiac echocardiography (ICE) transducer;
an ultrasound imager configured to generate two-dimensional images ICE images of a part of a cardiac system of a patient with the ICE transducer;
an image processor configured to generate the three-dimensional segmentation from the two-dimensional ICE images with a machine-learned multi-task generative adversarial network comprising of a multi-task architecture with a first task being volume completion and a second task being three-dimensional segmenting; and
a display configured to display ablation guidance relative to an image of the three-dimensional segmentation.

18. The medical imaging system of claim 17 wherein the ICE transducer is in a catheter and further comprising a position sensor on the catheter, wherein an input to the machine-learned generative network is an ultrasound volume formed from the two-dimensional ICE images with position information from the position sensor.

19. The medical imaging system of claim 17 wherein ground truth for the volume completion is computed tomography or magnetic resonance volumes from patients and is different than ground truth for the three-dimensional segmenting.

Assignees

• Siemens Medical Solutions Usa Inc

Inventors

• Funka-Lea Gareth
• Liao Haofu
• Zhou Shaohua Kevin
• Zheng Yefeng
• Tang Yucheng

Classifications

• A61B8/13

  Tomography (A61B8/10, A61B8/12 take precedence) · CPC title

• A61B8/4254

  using sensors mounted on the probe · CPC title

• A61B5/055

  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging · CPC title

• A61B8/5261

  combining images from different diagnostic modalities, e.g. ultrasound and X-ray · CPC title

• A61B90/37

  Surgical systems with images on a monitor during operation · CPC title