Processor and memory transparent convolutional lowering and auto zero padding for deep neural network implementations

Patent metadata

Field	Value
Publication number	US-10565285-B2
Application number	US-201715845390-A
Country	US
Kind code	B2
Filing date	Dec 18, 2017
Priority date	Dec 18, 2017
Publication date	Feb 18, 2020
Grant date	Feb 18, 2020

Abstract

A convolutional lowering component (CoLor component) between processor and memory units (or within a memory hierarchy) maps location in a lowered matrix to an equivalent location in a non-lowered matrix and provides auto zero padding in computational heavy convolutional layers. An identification component identifies processing components that execute computations in deep neural networks (DNNs) in which convolutions are realized as general matrix to matrix multiplications (GEMM) operations, and identifies a subset of the processing components that store deep neural network (DNN) features in a non-lowered form component that determines output for successively larger neural networks of a set. An address translation component translates address requests, generated by the subset of processing components to a memory subsystem, from a lowered index form to a non-lowered index form.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores computer executable components;
a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise:
an identification component that identifies processing components that execute computations in deep neural networks (DNNs) in which convolutions are realized as general matrix to matrix multiplications (GEMM) operations, and identifies a subset of the processing components that store deep neural network (DNN) features in a non-lowered form; and
an address translation component that translates address requests, generated by the subset of processing components to a memory subsystem, from a lowered index form to a non-lowered index form.

2. The system of claim 1 further comprising a mapping component that coalesces lowered index requests that map to a same non-lowered index.

3. The system of claim 1 , further comprising a padding component wherein if an index in lowered matrix maps to a location beyond boundaries in a non-lowered version then out of boundary locations are zero-padded.

4. The system of claim 3 , wherein the address translation component returns a zero to the processor without passing a request to the memory subsystem.

5. The system of claim 1 , wherein the processor implements a convolution as general matrix to matrix multiplications, wherein the memory does not store a lowered matrix.

6. The system of claim 1 , wherein memory requests from the processor pass through the address translation component before reaching the memory sub-system.

7. The system of claim 1 , wherein the address translation component converts the lowered matrix index form to the non-lowered index form in an input feature space.

8. The system of claim 1 , wherein input data is read and not written.

9. The system of claim 7 , wherein the address translation component performs an inverse hash function in connection with translation of the lowered matrix index form and the non-lowered index form.

10. A computer-implemented method, comprising:
identifying by a system operatively coupled to a processor that is operatively coupled to a memory, processing components that execute computations in deep neural networks (DNNs) in which convolutions are realized as general matrix to matrix multiplications (GEMM) operations, and identifies a subset of the processing components that store deep neural network (DNN) features in a non-lowered form; and
translating by the system, address requests, generated by the subset of processing components to a memory subsystem, from a lowered index form to a non-lowered index form.

11. The method of claim 10 further comprising coalescing, by the system, lowered index requests that map to a same non-lowered index.

12. The method of claim 10 , further comprising wherein if an index in lowered matrix maps to a location beyond boundaries in a non-lowered version then out of boundary locations are zero-padded by the system.

13. The method of claim 12 , wherein the system does not pass the request to the memory sub-system.

14. The system of claim 10 , further comprising implementing a convolution as general matrix to matrix multiplications, wherein the memory does not store a lowered matrix.

15. The system of claim 10 , wherein memory requests from the processor pass through an address translation component before reaching the memory sub-system.

16. A non-transitory computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by processor to cause the processor to:
identify by a system operatively coupled to a processor that is operatively coupled to a memory, processing components that execute computations in deep neural networks (DNNs) in which convolutions are realized as general matrix to matrix multiplications (GEMM) operations, and identifies a subset of the processing components that store deep neural network (DNN) features in a non-lowered form; and
translate by the system, address requests, generated by the subset of processing components to a memory subsystem, from a lowered index form to a non-lowered index form.

17. The non-transitory computer program product of claim 16 , the program instructions further cause the processor to: coalesce, by the system, lowered index requests that map to a same non-lowered index.

18. The non-transitory computer program product of claim 16 , the program instructions further cause the processor to: wherein if an index in lowered matrix maps to a location beyond boundaries in a non-lowered version then out of boundary locations are zero-padded by the system.

19. The non-transitory computer program product of claim 12 , the program instructions further cause the processor to implement a convolution as general matrix to matrix multiplications, wherein the memory does not store a lowered matrix.

20. The non-transitory computer program product of claim 12 , the program instructions further cause the processor to pass memory requests from the processor through an address translation component before reaching the memory sub-system.

Assignees

• IBM

Inventors

• Choi Jungwook
• Fleischer Bruce
• Srinivasan Vijayalakshmi
• Venkataramani Swagath

Classifications

• G06F17/153 Primary

  Multidimensional correlation or convolution · CPC title

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N7/02

  using fuzzy logic (computing arrangements based on biological models G06N3/00; computing arrangements using knowledge-based models G06N5/00) · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title