Method for training neural network and device thereof

Patent metadata

Field	Value
Publication number	US-11823056-B2
Application number	US-202016842435-A
Country	US
Kind code	B2
Filing date	Apr 7, 2020
Priority date	Oct 28, 2019
Publication date	Nov 21, 2023
Grant date	Nov 21, 2023

Abstract

Provided is a method for training a neural network and a device thereof. The method may train a neural network with three-dimensional (3D) training image data including a plurality of two-dimensional (2D) training image data. The method may include training, at a processor, a first convolutional neural network (CNN) with the plurality of 2D training image data, wherein the first convolutional neural network comprises 2D convolutional layers. The method may further include training, at the processor, a second convolutional neural network with the 3D training image data, wherein the second convolutional neural network comprises the 2D convolutional layers and 3D convolutional layers configured to receive an output of the 2D convolutional layers as an input.

First claim

What is claimed is:

1. A method for training a neural network with three-dimensional (3D) training image data comprising a plurality of two-dimensional (2D) training image data, the method comprising:
training, at a processor, a first convolutional neural network with the plurality of 2D training image data, wherein the first convolutional neural network comprises a plurality of 2D convolutional layers; and
training, at the processor, a second convolutional neural network with the 3D training image data,
wherein the second convolutional neural network comprises the 2D convolutional layers, and one or more 3D convolutional layers following an output of the 2D convolutional layers as an input,
wherein at least one of the training the first convolutional neural network or the training the second convolutional neural network comprises training the first convolutional neural network with the plurality of 2D training image data and additional 2D training image data of a different domain from the plurality of 2training image data,
wherein the 3D training image data comprises at least one of a digital breast tomosynthesis (DBT) image or a computed tomography (CT) image, and the additional 2D training image data comprises at least onf of a full-field digital mammography (FFDM) image or an X-ray image, and
wherein parameters of one or more 2D convolutional layers among the 2D convolutional layers are fixed during the training of the second convolutional neural network, and parameters of one or more remaining 2D convolution layers among the 2D convolutional layers are trained with the 3D training image data during the training of the second convolutional neural network.

2. The method of claim 1 , wherein parameters of the one or more 3D convolutional layers are trained with the 3D training image data during the training of the second convolutional neural network.

3. The method of claim 2 , wherein each of the plurality of 2D training image data comprises first patch image data, and
wherein the 2D convolutional layers comprise third convolutional layers and fourth convolutional layers configured to receive an output of the third convolutional layers as an input, and
wherein the training of the first convolutional neural network comprises:
training the first convolutional neural network with the first patch image data; and
training the first convolutional neural network with all of the plurality of 2D training image data, while fixing parameters of the third convolutional layers according to a result of training the first convolutional neural network with the first patch image data.

4. The method of claim 3 , wherein the first patch image data are all of the same size in the plurality of 2D training image data.

5. The method of claim 3 , wherein the location of the first patch image data is arbitrarily determined for each of the plurality of 2D training image data.

6. The method of claim 3 , wherein each of the plurality of 2D training image data comprises second patch image data,
wherein the size of the second patch image data is larger than the size of the first patch image data, and
wherein the fourth convolutional layers comprise a first set of convolutional layers and a second set of convolutional layers configured to receive an output of the first set of convolutional layers as an input,
wherein the method further comprises training the first convolutional neural network with the second patch image data, after training the first convolutional neural network with the first patch image data, and
wherein training the first convolutional neural network with all of the plurality of 2D training image data comprises:
fixing parameters of the first set of convolutional layers according to a result of training the first convolutional neural network with the second patch image data.

7. The method of claim 2 , wherein the training of the second convolutional neural network comprises:
concatenating respective 2D feature maps with one another extracted by the 2D convolutional layers from the plurality of 2D training image data to form a 3D feature map; and
using the 3D feature map for training the 3D convolutional layers.

8. The method of claim 7 , wherein the concatenating of the respective 2D feature maps with one another comprises compressing and concatenating the respective 2D feature maps with one another.

9. The method of claim 7 , wherein compressing the respective 2D feature maps comprises compressing the respective 2D feature maps through convolution by the first convolutional neural network.

10. The method of claim 1 , wherein when the 3D training image data comprises the DBT image, the additional 2D training image data comprises the FFDM image.

11. The method of claim 1 , wherein when the 3D training image data comprises the CT image, the additional 2D training image data comprises the X-ray image.

12. The method of claim 1 , wherein a number of the one or more 2D convolutional layers whose parameters are fixed during the training of the second convolutional neural network is determined based on a usage of a memory included in a device for training the neural network.

13. A non-transitory computer-readable recording medium comprising computer executable instructions, when executed, configured to cause a processor to perform a method of training a neural network with three-dimensional (3D) training image data comprising a plurality of two-dimensional (2D) training image data, the method comprising:
training, at the processor, a first convolutional neural network with first patch image data included in each of the plurality of 2D training image data, wherein the first convolutional neural network comprises a plurality of 2D convolutional layers, the 2D convolutional layers comprise first convolutional layers and second convolutional layers configured to receive an output of the first convolutional layers as an input, and the plurality of 2D training image data are included in the 3D training image data;
training, at the processor, the first convolutional neural network with all of the plurality of 2D training image data, wherein parameters of the first convolutional layers are fixed according to a result of training the first convolutional neural network with the first patch image data; and
training, at the processor, a second convolutional neural network with the 3D training image data, wherein the second convolutional neural network comprises the 2D convolutional layers, and one or more 3D convolutional layers following an output of the 2D convolutional layers as an input,
wherein at least one of the training the first convolutional neural network or the training the second convolutional neural network comprises training the first convolutional neural network with the plurality of 2D training image data and additional 2D training image data of a different domain from the plurality of 2D training image data,
wherein the 3D training image data comprises at least one of a digital breast tomosynthesis (DBT) image or a computed tomography (CT) image, and the additional 2D training image data comprises at least one of a full-field digital mammography (FFDM) image or an X-ray image, and
wherein parameters of one or more 2D convolutional layers among the 2D convolutional layers are fixed during the training of the second convolutional neural network, and parameters of one or more other 2D convolution layers among the 2D convolutional layers are trained with the 3D training image data during the training of the second convolutional neural network.

14. The recording medium of claim 13 , wherein parameters of the one or more 3D convolu

Assignees

• Lunit Inc

Inventors

• Lee Hyunjae
• Kim Hyo-Eun
• Lee Weonsuk

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N20/20

  Ensemble learning · CPC title