Decomposing convolution operation in neural networks

Patent metadata

Field	Value
Publication number	US-2016019455-A1
Application number	US-201414526018-A
Country	US
Kind code	A1
Filing date	Oct 28, 2014
Priority date	Jul 16, 2014
Publication date	Jan 21, 2016
Grant date	—

Abstract

A method of operating a neural network includes determining a complexity, such as a number) of separable filters approximating a filter. The method further includes selectively applying a decomposed convolution to the filter based on the determined number of separable filters.

First claim

What is claimed is:

1 . A method of operating a neural network, comprising:
determining a complexity of separable filters approximating a filter in the neural network; and selectively applying a decomposed convolution to the filter based at least in part on the determining.

2 . The method of claim 1 , in which the determining is based at least in part on a rank of the filter.

3 . The method of claim 1 , further comprising:
replacing the filter with a low rank approximation based at least in part on a performance metric.

4 . The method of claim 1 , in which the determining is based at least in part on a singular value decomposition (SVD) of the filter.

5 . The method of claim 4 , further comprising:
replacing the filter with a low rank approximation based at least in part on singular values obtained via the singular value decomposition.

6 . The method of claim 1 , in which the determined complexity comprises a number of separable filters sufficient to approximate the filter.

7 . An apparatus for operating a neural network, comprising:
a memory; and at least one processor coupled to the memory, the at least one processor being configured: to determine a complexity of separable filters approximating a filter in the neural network; and to selectively apply a decomposed convolution to the filter based at least in part on the determined complexity of separable filters.

8 . The apparatus of claim 7 , in which the at least one processor is further configured to determine the complexity of separable filters based at least in part on a rank of the filter.

9 . The apparatus of claim 7 , in which the at least one processor is further configured to replace the filter with a low rank approximation based at least in part on a performance metric.

10 . The apparatus of claim 7 , the at least one processor is further configured to determine the complexity of separable filters based at least in part on a singular value decomposition (SVD) of the filter.

11 . The apparatus of claim 10 , in which the at least one processor is further configured to replace the filter with a low rank approximation based at least in part on singular values obtained via the singular value decomposition.

12 . The apparatus of claim 7 , in which the determined complexity comprises a number of separable filters sufficient to approximate the filter.

13 . An apparatus for operating a neural network, comprising:
means for determining a complexity of separable filters approximating a filter in the neural network; and means for selectively applying a decomposed convolution to the filter based at least in part on the determined complexity of separable filters.

14 . The apparatus of claim 13 , in which the determined complexity of separable filters is based at least in part on a rank of the filter.

15 . The apparatus of claim 13 , further comprising:
means for replacing the filter with a low rank approximation based at least in part on a performance metric.

16 . The apparatus of claim 13 , in which the determined complexity of separable filters is based at least in part on a singular value decomposition (SVD) of the filter.

17 . The apparatus of claim 16 , further comprising:
means for replacing the filter with a low rank approximation based at least in part on singular values obtained via the singular value decomposition.

18 . A computer program product for operating a neural network, comprising:
a non-transitory computer readable medium having encoded thereon program code, the program code comprising: program code to determine a complexity of separable filters approximating a filter in the neural network; and program code to selectively apply a decomposed convolution to the filter based at least in part on the determined complexity of separable filters.

19 . The computer program product of claim 18 , further comprising program code to determine the complexity of separable filters based at least in part on a rank of the filter.

20 . The computer program product of claim 18 , further comprising program code to replace the filter with a low rank approximation based at least in part on a performance metric.

21 . The computer program product of claim 18 , further comprising program code to determine the complexity of separable filters based at least in part on a singular value decomposition (SVD) of the filter.

22 . The computer program product of claim 21 , further comprising program code to replace the filter with a low rank approximation based at least in part on singular values obtained via the singular value decomposition.

Assignees

• Qualcomm Inc

Inventors

• Annapureddy Venkata Sreekanta Reddy

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/049 Primary

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/02

  Neural networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title