Semantic segmentation for cancer detection in digital breast tomosynthesis

Patent metadata

Field	Value
Publication number	US-10779785-B2
Application number	US-201816033332-A
Country	US
Kind code	B2
Filing date	Jul 12, 2018
Priority date	Jul 17, 2017
Publication date	Sep 22, 2020
Grant date	Sep 22, 2020

Abstract

A method, apparatus and non-transitory computer readable medium are for segmenting different types of structures, including cancerous lesions and regular structures like vessels and skin, in a digital breast tomosynthesis (DBT) volume. In an embodiment, the method includes: pre-classification of the DBT volume in dense and fatty tissue and based on the result; localizing a set of structures in the DBT volume by using a multi-stream deep convolutional neural network; and segmenting the localized structures by calculating a probability for belonging to a specific type of structure for each voxel in the DBT volume by using a deep convolutional neural network for providing a three-dimensional probabilistic map.

First claim

What is claimed is:

1. A method for segmenting a set of structures, including pathologic breast structures and regular breast structures, in a digital breast tomosynthesis (DBT) volume, comprising:
pre-classifying the DBT volume to provide a pre-classification result, the pre-classification result distinguishing between dense tissue and fatty tissue;
localizing a set of structures in the DBT volume by using at least one multi-stream deep image-to-image network, taking as input at least one image channel; and
segmenting the set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using at least one multi-stream deep image-to-image network, taking as input at least one image channel for providing a three-dimensional probabilistic map, wherein at least one of
the localizing is based on the pre-classification result and is executed with a specific model for dense tissue and with a specific model for fatty tissue and
the segmenting is based on the pre-classification result and is executed with a specific model for dense tissue and with a specific model for fatty tissue.

2. The method of claim 1 , wherein the localizing is executed by using an image-to-image multi-scale multi-streams deep learning network, taking as input at least one image channel to generate inverse distance maps to a center of each structure of interest.

3. The method of claim 2 , wherein the inverse distance maps are at least one of Euclidean, geodesic, are computed according to the center of each structure of interest, and are signed distance maps according to a boundary of the structure of interest.

4. The method of claim 1 , wherein for training the network, a Tversky loss function is used.

5. The method of claim 4 , wherein the Tversky loss function is a DICE loss function.

6. The method of claim 1 , wherein the DBT volume is forwarded as input to multiple resolution-specific streams and as an output of each of the resolution-specific streams a set of deconvolution layers is followed by activation layers and wherein different outputs of the resolution-specific streams are concatenated using a concatenation layer.

7. The method of claim 1 , wherein the localizing is based on generating a bounding box or an ellipsoid for the set of structures.

8. The method of claim 7 , wherein the segmenting is based on procedure, comprising:
pre-segmenting different types of structures, represented in the DBT volume, including regular structures and suspicious structures by way of filters and multi-stream deep image-to-image network;
structure-specific segmenting by generating, for each lesion bounding box, a candidate structure-specific prediction channel, encoding a probability for each voxel to belong to a specific type of structure, including a regular structure or a suspicious structure;
structure-specific refining of the structure-specific prediction channels by modeling neighborhood dependencies; and
aggregating the structure-specific prediction channels, once refined, by fusing the structure-specific prediction channels and probabilities and class probabilities by applying:
Pred=Σ c∈classes P ( c )·Pred c .

9. The method of claim 8 , wherein the segmenting further comprises:
calculating a class probability by classifying the candidate structure into a specific structure class by using a multi-stream deep image-to-image network.

10. The method of claim 1 , further comprising:
providing result data for each voxel of the DBT volume, and encoding a probability of belonging to a specific type of structure for breast cancer diagnosis.

11. A memory storing software code portions for executing the method of claim 1 , when the software code portions are run on a computer.

12. A non-transitory computer readable medium including program code for carrying out the method of claim 1 when the program code is run on a computer.

13. The method of claim 1 , wherein the localizing is based on a procedure, comprising:
extracting regions of interest;
generating bounding boxes;
adjusting the bounding boxes;
rejecting false positive structures; and
pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.

14. The method of claim 1 , wherein the localizing is executed by using an image-to-image multi-scale multi-streams deep learning network, taking as input at least one image channel to generate inverse distance maps to a center of each structure of interest.

15. The method of claim 14 , wherein the inverse distance maps are at least one of Euclidean, geodesic, are computed according to the center of each structure of interest, and are signed distance maps according to a boundary of the structure of interest.

16. A method for segmenting a set of structures, including pathologic breast structures and regular breast structures, in a digital breast tomosynthesis (DBT) volume, comprising:
localizing a set of structures in the DBT volume by using at least one multi-stream deep image-to-image network, taking as input at least one image channel; and
segmenting the set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using at least one multi-stream deep image-to-image network, taking as input at least one image channel for providing a three-dimensional probabilistic map, wherein the localizing is based on a procedure, comprising:
extracting regions of interest;
generating bounding boxes;
adjusting the bounding boxes;
rejecting false positive structures; and
pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.

17. A non-transitory computer readable medium including program code for carrying out the method of claim 16 when the program code is run on a computer.

18. A memory storing software code portions for executing the method of claim 16 , when the software code portions are run on a computer.

19. An apparatus for segmenting different types of structures, including cancerous lesions and regular structures, in a digital breast tomosynthesis (DBT) volume, comprising:
a localizer to localize a set of structures in the DBT volume by using a multi-stream deep convolutional neural network; and
a segmenter to segment the localized set of structures by calculating a probability for belonging to a specific type of structure, for each voxel in the DBT volume, by using a deep convolutional neural network for providing a three-dimensional probabilistic map, wherein localizer is configured to localize the set of structures based on a procedure, comprising:
extracting regions of interest;
generating bounding boxes;
adjusting the bounding boxes;
rejecting false positive structures; and
pre-classifying different regions of interest in the DBT volume by providing a class probability to belong to a specific type of structure, including, round mass, spiculated or benign lesions by using a deep convolutional neural network.

20. The apparatus of claim 19 , further comprising:
a pre-classifier, configured to provide a pre-classification result, the pre-classification result differentiating in the DBT volume between dense tissue and fatty tissue, whe

Assignees

• Siemens Healthcare Gmbh

Inventors

• Itu Lucian Mihai
• Lazar Laszlo
• Liu Siqi
• Pauly Olivier
• Seegerer Philipp
• Stroia Iulian Ionut
• Turcea Alexandru
• Vizitiu Anamaria
• Xu Daguang
• Zhou Shaohua Kevin

Classifications

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• A61B6/502 Primary

  for diagnosis of breast, i.e. mammography · CPC title

• G06T12/20

  Inverse problem, i.e. transformations from projection space into object space · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• G06T2207/20012

  Locally adaptive · CPC title