Sub-graph in frequency domain and dynamic selection of convolution implementation on a GPU

Patent metadata

Field	Value
Publication number	US-11600035-B2
Application number	US-202217669126-A
Country	US
Kind code	B2
Filing date	Feb 10, 2022
Priority date	Apr 8, 2017
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

In an example, an apparatus comprises a plurality of execution units; and logic, at least partially including hardware logic, to determine a sub-graph of a network that can be executed in a frequency domain and apply computations in the sub-graph in the frequency domain. Other embodiments are also disclosed and claimed.

First claim

What is claimed is:

1. An apparatus comprising processing circuitry to:
obtain a sub-graph of a convolutional neural network to be executed at least in part in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply one or more convolutional computations in the sub-graph in the frequency domain, wherein the one or more convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using a first set of execution resources, while internal computations are performed in a baseline precision level using a second set of execution resources, different from the first set of execution resources.

2. The apparatus of claim 1 , the processing circuitry to:
dynamically select a convolutional implementation based at least in part on executing a comparison of one or more convolutions in the convolutional neural network.

3. The apparatus of claim 1 , the processing circuitry to:
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network using training sample statistics for the convolutional neural network.

4. The apparatus of claim 3 , the processing circuitry to:
update the predicted level of activation sparsity when operating on the convolutional neural network in inference mode.

5. The apparatus of claim 1 , wherein:
the first set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations; and
the second set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations.

6. The apparatus of claim 2 , the processing circuitry to:
expose one or more embedded cast operations in a load/store instruction to support loading data for the one or more convolutional computations in a variable integer precision.

7. The apparatus of claim 6 , the processing circuitry to:
select between a 2-bit precision, a 3-bit precision, and a 7-bit precision level.

8. A method, comprising:
obtaining a sub-graph of a convolutional neural network to be executed at least in part in a frequency domain;
generating a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
applying one or more convolutional computations in the sub-graph in the frequency domain, wherein the one or more convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using a first set of execution resources, while internal computations are performed in a baseline precision level using a second set of execution resources, different from the first set of execution resources.

9. The method of claim 8 , further comprising:
dynamically selecting a convolutional implementation based at least in part on executing a comparison of one or more convolutions in the convolutional neural network.

10. The method of claim 8 , further comprising:
generating a predicted level of activation sparsity for one or more layers of the convolutional neural network using training sample statistics for the convolutional neural network.

11. The method of claim 10 , further comprising:
updating the predicted level of activation sparsity when operating on the convolutional neural network in inference mode.

12. The method of claim 8 , wherein:
the first set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations; and
the second set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations.

13. The method of claim 8 , further comprising:
exposing one or more embedded cast operations in a load/store instruction to support loading data for the one or more convolutional computations in a variable integer precision.

14. The method of claim 8 , further comprising:
selecting between a 2-bit precision, a 3-bit precision, and a 7-bit precision level.

15. One or more non-transitory computer readable media comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
obtain a sub-graph of a convolutional neural network to be executed at least in part in a frequency domain;
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network; and
apply one or more convolutional computations in the sub-graph in the frequency domain, wherein the one or more convolutional computations are performed at variable levels of integer precisions based at least in part on the predicted level of activation sparsity of a layer in the convolutional neural network using a first set of execution resources, while internal computations are performed in a baseline precision level using a second set of execution resources, different from the first set of execution resources.

16. The one or more non-transitory computer readable media of claim 15 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
dynamically select a convolutional implementation based at least in part on executing a comparison of one or more convolutions in the convolutional neural network.

17. The one or more non-transitory computer readable media of claim 15 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
generate a predicted level of activation sparsity for one or more layers of the convolutional neural network using training sample statistics for the convolutional neural network.

18. The one or more non-transitory computer readable media of claim 17 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
update the predicted level of activation sparsity when operating on the convolutional neural network in inference mode.

19. The one or more non-transitory computer readable media of claim 15 , wherein:
the first set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations; and
the second set of execution resources comprises a first plurality of processing elements configured to perform dense convolution operations.

20. The one or more non-transitory computer readable media of claim 15 , further comprising instructions which, when executed by processing circuitry, configure the processing circuitry to:
expose one or more embedded cast operations in a load/store instruction to support loading data for the one or more convolutional computations in a variable integer precision.

Assignees

• Intel Corp

Inventors

• Sarel Uzi
• Cohen Ehud
• Schwartz Tomer
• Armon Amitai
• Shadmiy Yahav
• Ben-Ari Itamar
• Bleiweiss Amit
• Faivishevsky Lev
• Bar-On Tomer
• Fais Yaniv
• Subag Jacob
• Behar Michael
• Jacob Guy
• Leibovich Gal
• Dreyfuss Jeremie

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/02

  Neural networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title