Systems and methods for fast mammography data handling

Patent metadata

Field	Value
Publication number	US-11954853-B2
Application number	US-202117382159-A
Country	US
Kind code	B2
Filing date	Jul 21, 2021
Priority date	Jul 21, 2021
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

This disclosure proposes to speed up computation time of a convolutional neural network (CNN) by leveraging information specific to a pre-defined region, such as a breast in mammography and tomosynthesis data. In an exemplary embodiment, a method for an image processing system is provided, comprising, generating an output of a trained convolutional neural network (CNN) of the image processing system based on an input image, including a pre-defined region of the input image as an additional input into at least one of a convolutional layer and a fully connected layer of the CNN to limit computations to input image data inside the pre-defined region; and storing the output and/or displaying the output on a display device.

First claim

The invention claimed is:

1. A method for an image processing system, comprising:
generating an output of a trained convolutional neural network (CNN) of the image processing system based on an input image, including a pre-defined region of the input image as an additional input into at least one of a convolutional layer and a fully connected layer of the CNN to limit computations to input image data inside the pre-defined region; and
storing the output and/or displaying the output on a display device;
wherein including the pre-defined region of the input image as an additional input into the at least one of a convolutional layer and a fully connected layer of the CNN further includes:
associating a mask with at least one convolutional layer of the CNN;
mapping the mask to an input of the at least one convolutional layer;
performing convolutions on input data inside the pre-defined region, and
not performing convolutions on input image data outside the pre-defined region.

2. The method of claim 1 , wherein mapping the mask to a feature input of the at least one convolutional layer further comprises downsampling and/or resizing the mask associated with a preceding layer of the CNN.

3. The method of claim 1 , wherein the mask is an array of values of a same set of dimensions as an input into the at least one convolutional layer, each value of the array of values corresponding to a respective pixel or feature of the input, and where a first value is assigned to the array if the respective pixel or feature is inside the pre-defined region, and a second value is assigned to the array if the respective pixel or feature is outside the pre-defined region.

4. The method of claim 3 , further comprising:
multiplying an input and/or an output of the at least one convolutional layer with a value at a corresponding spatial position of the mask.

5. The method of claim 3 , further comprising:
during a training stage of the CNN, applying the mask at only an input layer of the CNN to assign the second value to pixels of the input image outside the pre-defined region;
during an inference stage of the CNN:
inputting a background input image into the CNN, all pixel intensity values of the background input image equal to the second value;
obtaining a set of background features as an output of a last convolutional layer of the CNN;
inputting a new input image into the CNN;
replacing features obtained as an output of the last convolutional layer outside the pre-defined region with corresponding features of the set of background features; and
generating an output of the CNN using the replaced weights.

6. The method of claim 1 , further comprising:
during a training stage of the CNN, at least one of:
including the mask as an additional input into at least one convolutional layer of the CNN to perform convolutions on input image data inside the pre-defined region, and not perform convolutions on input image data outside the pre-defined region; and
including the mask as an additional input into at least one fully connected layer of the CNN to activate nodes based on input image data inside the pre-defined region, and not activate nodes based on input image data outside the pre-defined region.

7. The method of claim 6 , further comprising, during the training stage, backpropagating a result of a loss function through nodes of the CNN, using the mask at the at least one convolutional layer of the CNN to adjust weights of the CNN based on loss backpropagation inside the pre-defined region and not outside the pre-defined region.

8. The method of claim 1 , wherein the pre-defined region is composed of one or more areas of a breast, including normal and abnormal areas of the breast.

9. The method of claim 1 , wherein the mask is based on a pre-defined region delimited by one of a shape of a compression paddle or a shape of a biopsy window.

10. A method for an image processing system, comprising:
generating an output of a trained convolutional neural network (CNN) of the image processing system based on an input image, including a pre-defined region of the input image as an additional input into at least one of a convolutional layer and a fully connected layer of the CNN to limit computations to input image data inside the pre-defined region; and
storing the output and/or displaying the output on a display device;
wherein including the pre-defined region of the input image as an additional input into the at least one of a convolutional layer and a fully connected layer of the CNN further includes:
associating a mask with at least one fully connected layer of the CNN;
mapping the mask to an input of the at least one fully connected layer; and
calculating an output of the at least one fully connected layer based on input data inside the pre-defined region and not based on input data outside the pre-defined region.

11. An image processing system comprising:
a convolutional neural network (CNN);
a training dataset of images, the training dataset including a plurality of training pairs, each training pair including an input image of a breast and a ground truth data of the breast;
a processor communicably coupled to a non-transitory memory storing the CNN and including instructions that when executed cause the processor to:
define a region of the breast of each input image of each training pair, where image data in the region includes breast information and where image data not in the region does not include the breast information;
during training of the CNN:
during propagation, at each layer of the CNN, apply a mask to perform convolutions on input data inside the pre-defined region and not perform convolutions on input image data outside the pre-defined region; and
during backpropagation, at each layer of the CNN, apply the mask when using a gradient descent algorithm such that weights are adjusted at nodes of the CNN based on loss backpropagation inside the pre-defined region and not outside the pre-defined region; and

deploy the CNN to generate an output, and display the output on a display device and/or store the output in a database of the image processing system.

12. The system of claim 11 , wherein further includes further instructions are stored in the memory that when executed cause the processor to:
perform a downsampling operation to the mask at each pooling layer of the CNN, and
apply a downsampled mask at a layer subsequent to each pooling layer.

13. The system of claim 11 , wherein the output includes at least one of an indication of a presence of a lesion of the breast and location information of the lesion.

14. The system of claim 11 , wherein the CNN is trained using patch-based training, and the pre-defined region is a 2D or 3D patch used during the patch-based training.

15. A method for a convolutional neural network (CNN), comprising:
deploying the CNN to detect an abnormality in an input image during an inference stage; and
applying convolutional filters of the CNN to a first region of the input image, and not applying convolutional filters to a second region of the input image, wherein the first region and the second region are specified by a mask that is downsampled and propagated through layers of the CNN.

16. The method of claim 15 , wherein the first region does not intersect with the second region, and wherein a total area of the input image is equal to a total area of the first region added to a total area of the second region.

17. The method of claim 15 , wherein the mask is an array of binary values, wherein values of the array corre

Assignees

• Ge Prec Healthcare Llc

Inventors

• Bernard Sylvain
• Bismuth Vincent

Classifications

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G16H30/40

  for processing medical images, e.g. editing · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title