Per kernel Kmeans compression for neural networks

Patent metadata

Field	Value
Publication number	US-11055604-B2
Application number	US-201715702193-A
Country	US
Kind code	B2
Filing date	Sep 12, 2017
Priority date	Sep 12, 2017
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

Methods and apparatus relating to techniques for incremental network quantization. In an example, an apparatus comprises logic, at least partially comprising hardware logic to determine a plurality of weights for a layer of a convolutional neural network (CNN) comprising a plurality of kernels; organize the plurality of weights into a plurality of clusters for the plurality of kernels; and apply a K-means compression algorithm to each of the plurality of clusters. Other embodiments are also disclosed and claimed.

First claim

The invention claimed is:

1. A general-purpose graphics processing device comprising:
an instruction cache to receive a stream of instructions;
an instruction unit to execute the stream of instructions;
a general-purpose graphics processing compute block comprising a plurality of processing resources;
a shared cache memory communicatively coupled to the plurality of processing resources to receive data representing one or more layers of a convolutional neural network; and
the plurality of processing resources of the graphics processing compute block to:
determine a plurality of weights for a layer of a convolutional neural network (CNN) comprising a plurality of kernels;
organize the plurality of weights into a plurality of clusters on a per-kernel basis for the plurality of kernels;
compute a center point for the plurality of weights on a per-kernel basis in each cluster; and
apply a K-means compression algorithm to each of the plurality of clusters.

2. The general-purpose graphics processing device of claim 1 , the plurality of processing resources of the graphics processing compute block to:
encode the plurality of weights as a 1D tensor.

3. The general-purpose graphics processing device of claim 2 , the plurality of processing resources of the graphics processing compute block to:
determine an index associated with the center point for the plurality of weights in each cluster.

4. The general-purpose graphics processing device of claim 3 , wherein the plurality of weights are compressed to 4 bits.

5. The general-purpose graphics processing device of claim 3 , the plurality of processing resources of the graphics processing compute block to:
store the index in the shared cache memory.

6. An electronic device, comprising:
a general purpose processor having one or more processor cores; and
a graphics processing device, comprising:
an instruction cache to receive a stream of instructions;
an instruction unit to execute the stream of instructions;
a general-purpose graphics processing compute block comprising a plurality of processing resources;
a shared cache memory communicatively coupled to the plurality of processing resources to receive data representing one or more layers of a convolutional neural network; and
the plurality of processing resources of the graphics processing compute block to:
determine a plurality of weights for a layer of a convolutional neural network (CNN) comprising a plurality of kernels;
organize the plurality of weights into a plurality of clusters on a per-kernel basis for the plurality of kernels;
compute a center point for the plurality of weights on a per-kernel basis in each cluster; and
apply a K-means compression algorithm to each of the plurality of clusters.

7. The electronic device of claim 6 , the plurality of processing resources of the graphics processing compute block to:
encode the plurality of as a 1D tensor.

8. The electronic device of claim 7 , the plurality of processing resources of the graphics processing compute block to:
determine an index associated with the center point for the plurality of weights in each cluster.

9. The electronic device of claim 8 , wherein the plurality of weights are compressed to 4 bits.

10. The electronic device of claim 8 , the plurality of processing resources of the graphics processing compute block to:
store the index in the shared cache memory.

Assignees

• Intel Corp

Inventors

• Glesner Yonatan
• Novik Gal
• Vainbrand Dmitri
• Leibovich Gal

Classifications

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0495 Primary

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title