Shape-based generative adversarial network for segmentation in medical imaging

Patent metadata

Field	Value
Publication number	US-11464491-B2
Application number	US-202016885517-A
Country	US
Kind code	B2
Filing date	May 28, 2020
Priority date	Aug 7, 2019
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

For segmentation in medical imaging, a shape generative adversarial network (shape GAN) is used in training. By including shape information in a lower dimensional space than the pixels or voxels of the image space, the network may be trained with a shape loss or optimization. The adversarial loss and the shape loss are used to train the network, so the resulting generator may segment complex shapes in 2D or 3D. Other optimization may be used, such as using a loss in image space.

First claim

We claim:

1. A method for segmentation in a medical imaging system, the method comprising:
scanning, by a medical scanner, a patient, the scanning providing first imaging data representing the patient;
segmenting, by an image processor, an object represented in the first imaging data, the segmenting using a machine-learned generative network applied to the first imaging data, the machine-learned generative network having been trained using at least an adversarial loss generated by a machine-learned discriminator network and a shape loss based on eigenvalues from a linearization of a segmentation output by the machine-learned generative network; and
outputting, on a display, an image of the object based on the segmenting.

2. The method of claim 1 wherein scanning comprises scanning with the medical scanner being an ultrasound system.

3. The method of claim 1 wherein segmenting comprises applying the machine-learned generative network as an image-to-image convolutional deep learned network.

4. The method of claim 1 wherein segmenting comprises segmenting with the machine-learned generative network having been trained where a shape space for the shape loss comprises a lower dimensional space from manifold or unsupervised learning.

5. The method of claim 4 wherein segmenting comprises segmenting with the machine-learned generative network having been trained where the shape space comprised an eigenmatrix from a principal component analysis of a signed distance function.

6. The method of claim 1 wherein segmenting comprises segmenting with the machine-learned generative network having been trained where the optimization further comprised a first loss in an image space.

7. The method of claim 1 wherein segmenting comprises segmenting with the machine-learned generative network trained to output a signed distance per pixel or voxel from a boundary of the object.

8. The method of claim 1 further comprising:
determining shape parameters in a shape space from a distance function output by the machine-learned generative network; and
classifying the object based on a comparison of the shape parameters to a distribution of parameters.

9. The method of claim 1 further comprising:
determining shape parameters in a shape space from a distance function output by the machine-learned generative network; and
indicating a confidence in the segmenting based on a comparison of the shape parameters to a distribution of parameters.

10. A system for object localization, the system comprising:
a medical scanner configured to scan a patient, the scan resulting in image data representing the patient;
an image processor configured to locate, with a machine-learned image-to-image network, an object represented in the image data, the machine-learned image-to-image network having been trained using at least an adversarial loss generated by a machine-learned discriminator network and a shape loss based on eigenvalues from a linearization of a segmentation output by the machine-learned image-to-image network; and
a display configured to display an image as a function of the located object.

11. The system of claim 10 wherein the machine-learned image-to-image network is configured to output a distance by pixel or voxel from a boundary of the object and was trained with the adversarial loss on the distance.

12. The system of claim 10 wherein the machine-learned image-to-image network was trained with a smooth L 1 loss for the eigenvalues.

13. The system of claim 10 wherein the image processor is configured to determine shape parameters from the image data and detect a characteristic of the object from the shape parameters.

14. The system of claim 10 wherein the image processor is configured to determine shape parameters from the image data and identify a similar case from the shape parameters.

15. The system of claim 10 wherein the image processor is configured to determine shape parameters from the image data and identify a confidence in the location of the object from the shape parameters.

16. A method for machine training a model for segmentation, the method comprising:
machine training a generator to output the segmentation based on an adversarial loss generated by a machine-learned discriminator network and a shape loss based on eigenvalues from a linearization of a segmentation output by the generator a; and
storing the trained generator.

17. The method of claim 16 further comprising generating ground truths as distances per voxel or pixel from a boundary of an object, wherein the shape loss is a smooth L 1 loss based on shape in the distances and the adversarial loss is of the distances in an image space.

18. The method of claim 16 wherein machine training comprises machine training the generator as a fully convolutional neural network based on the shape loss, the adversarial loss, and an image loss.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Jacob Athira
• Passerini Tiziano

Classifications

• A61B8/5215 Primary

  involving processing of medical diagnostic data · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06F18/2135

  based on approximation criteria, e.g. principal component analysis · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N3/045

  Combinations of networks · CPC title