Sub-graph in frequency domain and dynamic selection of convolution implementation on a GPU

Patent metadata

Field	Value
Publication number	US-10762685-B2
Application number	US-201916670749-A
Country	US
Kind code	B2
Filing date	Oct 31, 2019
Priority date	Apr 8, 2017
Publication date	Sep 1, 2020
Grant date	Sep 1, 2020

Abstract

In an example, an apparatus comprises a plurality of execution units; and logic, at least partially including hardware logic, to determine a sub-graph of a network that can be executed in a frequency domain and apply computations in the sub-graph in the frequency domain. Other embodiments are also disclosed and claimed.

First claim

What is claimed is:

1. An apparatus comprising processing circuitry to:
obtain a sub-graph of a convolutional neural network to be executed at least in part in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply one or more convolutional computations in the sub-graph in the frequency domain, wherein the one or more convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using a first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using a second set of execution resources, different from the first set of execution resources, wherein:
the first set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations; and
the second set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations.

2. The apparatus of claim 1 , the processing circuitry to:
dynamically select a convolutional implementation based at least in part on executing a short comparison of one or more convolutions in the convolutional neural network.

3. The apparatus of claim 1 , the processing circuitry to:
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network using training sample statistics for the convolutional neural network.

4. The apparatus of claim 3 , the processing circuitry to:
update the predicted level of activation sparsity when operating on the convolutional neural network in inference mode.

5. The apparatus of claim 1 , the processing circuitry to:
expose one or more embedded cast operations in a load/store instruction to support loading data for the one or more convolutional computations in a variable integer precision.

6. The apparatus of claim 5 , the processing circuitry to:
select between a 2-bit precision, a 3-bit precision, and a 7-bit precision level.

7. A method, comprising:
obtaining a sub-graph of a convolutional neural network to be executed at least in part in a frequency domain;
generating a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
applying one or more convolutional computations in the sub-graph in the frequency domain, wherein the one or more convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using a first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using a second set of execution resources, different from the first set of execution resources, wherein:
the first set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations; and
the second set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations.

8. The method of claim 7 , further comprising:
dynamically selecting a convolutional implementation based at least in part on executing a short comparison of one or more convolutions in the convolutional neural network.

9. The method of claim 7 , further comprising:
generating a predicted level of activation sparsity for one or more layers of the convolutional neural network using training sample statistics for the convolutional neural network.

10. The method of claim 9 , further comprising:
updating the predicted level of activation sparsity when operating on the convolutional neural network in inference mode.

11. The method of claim 7 , further comprising:
exposing one or more embedded cast operations in a load/store instruction to support loading data for the one or more convolutional computations in a variable integer precision.

12. The method of claim 7 , further comprising:
selecting between a 2-bit precision, a 3-bit precision, and a 7-bit precision level.

13. One or more non-transitory computer readable media comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
obtain a sub-graph of a convolutional neural network to be executed at least in part in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply one or more convolutional computations in the sub-graph in the frequency domain, wherein the one or more convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using a first set of execution resources, while internal computations are performed in a baseline precision level of 8-bits or 16-bits using a second set of execution resources, different from the first set of execution resources, wherein:
the first set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations; and
the second set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations.

14. The one or more non-transitory computer readable media of claim 13 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
dynamically select a convolutional implementation based at least in part on executing a short comparison of one or more convolutions in the convolutional neural network.

15. The one or more non-transitory computer readable media of claim 13 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network using training sample statistics for the convolutional neural network.

16. The one or more non-transitory computer readable media of claim 15 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
update the predicted level of activation sparsity when operating on the convolutional neural network in inference mode.

17. The one or more non-transitory computer readable media of claim 13 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
expose one or more embedded cast operations in a load/store instruction to support loading data for the one or more convolutional computations in a variable integer precision.

Assignees

• Intel Corp

Inventors

• Sarel Uzi
• Cohen Ehud
• Schwartz Tomer
• Armon Amitai
• Shadmiy Yahav
• Ben-Ari Itamar
• Bleiweiss Amit
• Faivishevsky Lev
• Bar-On Tomer
• Fais Yaniv
• Subag Jacob
• Behar Michael
• Jacob Guy
• Leibovich Gal
• Dreyfuss Jeremie

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title