Large model support in deep learning

Patent metadata

Field	Value
Publication number	US-11526759-B2
Application number	US-201816180864-A
Country	US
Kind code	B2
Filing date	Nov 5, 2018
Priority date	Nov 5, 2018
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Techniques that facilitate model support in deep learning are provided. In one example, a system includes a graphics processing unit and a central processing unit memory. The graphics processing unit processes data to train a deep neural network. The central processing unit memory stores a portion of the data to train the deep neural network. The graphics processing unit provides, during a forward pass process of the deep neural network that traverses through a set of layers for the deep neural network from a first layer of the set of layers to a last layer of the set of layers that provides a set of outputs for the deep neural network, input data for a layer from the set of layers for the deep neural network to the central processing unit memory.

First claim

What is claimed is:

1. A system, comprising:
a graphics processing unit comprising a graphics processing unit cache memory, wherein the graphics processing unit trains, using the graphics processing unit cache memory, a deep neural network comprising a set of layers, and wherein the training comprises:
during a forward pass process of the deep neural network that employs training data and traverses through the set of layers from a first layer of the set of layers to a last layer of the set of layers that provides a set of outputs for the deep neural network, provide, to a central processing unit for storage in a central processing unit cache memory of the central processing unit, input data used as input for a layer from the set of layers during the forward pass process, wherein the layer is not the first layer, and at least a portion of the input data is employed, by the graphics processing unit, as input for the layer during a backward pass process of the deep neural network that traverses from the last layer to the first layer.

2. The system of claim 1 , wherein the central processing unit provides, during the backward pass process of the deep neural network, the input data to the graphics processing unit.

3. The system of claim 1 , wherein the graphics processing unit stores, during the forward pass process, output data from the layer of the deep neural network in the graphics processing unit cache memory.

4. The system of claim 1 , wherein the graphics processing unit provides, during the backward pass process of the deep neural network, gradient data from the layer of the deep neural network to the central processing unit for storage in the central processing unit cache memory.

5. The system of claim 1 , wherein the graphics processing unit receives, during the backward pass process of the deep neural network, parameter data for a layer of the deep neural network from the central processing unit cache memory.

6. The system of claim 1 , wherein the graphics processing unit provides the input data to the central processing unit via a compression scheme.

7. The system of claim 1 , wherein the graphics processing unit provides the input data to the central processing unit via a half-precision floating-point format.

8. The system of claim 1 , wherein the graphics processing unit is coupled to the central processing unit via a serial multi-lane communication link.

9. The system of claim 1 , wherein the central processing unit memory stores the portion of the data to facilitate improved processing performance for the graphics processing unit.

10. A computer-implemented method, comprising:
processing, by a graphics processing unit using a graphics processing unit cache memory of the graphics processing unit, training data to train a deep neural network that comprises a set of layers; and
providing, by the graphics processing unit to a central processing unit for storage in a central processing unit cache memory of the central processing unit, during a forward pass process of training the deep neural network that traverses through the set of layers from a first layer of the set of layers to a last layer of the set of layers that provides a set of outputs for the deep neural network, input data used as input for a layer from the set of layers during the forward pass process, wherein the layer is not the first layer, and at least a portion of the input data is employed, by the graphics processing unit, as input for the layer during a backward pass process of the deep neural network that traverses from the last layer to the first layer.

11. The computer-implemented method of claim 10 , further comprising:
receiving, by the graphics processing unit, the input data from the central processing unit during the backward pass process of the deep neural network.

12. The computer-implemented method of claim 10 , further comprising:
storing, by the graphics processing unit and during the forward pass process, output data from the layer of the deep neural network in the graphics processing unit cache memory.

13. The computer-implemented method of claim 10 , further comprising:
providing, by the graphics processing unit and during the backward pass process of the deep neural network, gradient data from the layer of the deep neural network to the central processing unit for storage in the central processing unit cache memory.

14. The computer-implemented method of claim 10 , further comprising:
receiving, by the graphics processing unit and during the backward pass process of the deep neural network, parameter data for the layer of the deep neural network from the central processing unit cache memory.

15. The computer-implemented method of claim 10 , wherein the providing comprises providing the input data to the central processing unit via a compression scheme.

16. The computer-implemented method of claim 10 , wherein the providing comprises providing the input data to the central processing unit via a half-precision floating-point format.

17. The computer-implemented method of claim 10 , wherein the providing comprises facilitating improved processing performance for the graphics processing unit.

18. A computer-implemented method, comprising:
receiving, by a central processing unit comprising a central processing unit cache memory, from a graphics processing unit, at least a portion of data employed by the graphics processing unit to train a deep neural network that comprises a set of layers, wherein the at least the portion of data comprises input data used as input during the training for a layer of the set of layers during a forward pass process that traverses through the set of layers from a first layer of the set of layers to a last layer of the set of layers that provides a set of outputs for the deep neural network, and the layer is not the first layer;
storing, by the central processing unit, at least the portion of the data in the central processing unit cache memory; and
providing, by the central processing unit, at least the portion of the data to the graphics processing unit executing a backward pass process of the deep neural network that traverses through the set of layers from the last layer to the first layer.

19. The computer-implemented method of claim 18 , wherein the receiving comprises receiving at least the portion of the data via a half-precision floating-point format.

20. The computer-implemented method of claim 18 , wherein the receiving comprises receiving at least the portion of the data via a compression scheme.

21. A computer program product for model support in deep learning, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a graphics processing unit to cause the graphics processing unit to:
process, by the graphics processing unit using a graphics processing unit cache memory of the graphics processing unit, training data to train a deep neural network that comprises a set of layers; and
provide, by the graphics processing unit to a central processing unit for storage in a central processing unit cache memory of the central processing unit, during a forward pass process of training the deep neural network that traverses through the set of layers from a first layer of the set of layers to a last layer of the set of layers that provides a set of outputs for the deep neural network, input data used as input for a layer from the set of layers during the forward pass proce

Assignees

• IBM

Inventors

• Cho Minsik
• Finkler Ulrich Alfons
• Zolotov Vladimir
• Kung David S

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06F13/4282 Primary

  on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247) · CPC title

• G06N3/09

  Supervised learning · CPC title