Indirectly accessing sample data to perform multi-convolution operations in a parallel processing system

Patent metadata

Field	Value
Publication number	US-2016162402-A1
Application number	US-201514951588-A
Country	US
Kind code	A1
Filing date	Nov 25, 2015
Priority date	Dec 4, 2014
Publication date	Jun 9, 2016
Grant date	—

Abstract

In one embodiment of the present invention, a convolution engine configures a parallel processing pipeline to perform multi-convolution operations. More specifically, the convolution engine configures the parallel processing pipeline to independently generate and process individual image tiles. In operation, for each image tile, the pipeline calculates source locations included in an input image batch based on one or more start addresses and one or more offsets. Subsequently, the pipeline copies data from the source locations to the image tile. The pipeline then performs matrix multiplication operations between the image tile and a filter tile to generate a contribution of the image tile to an output matrix. To optimize the amount of memory used, the pipeline creates each image tile in shared memory as needed. Further, to optimize the throughput of the matrix multiplication operations, the values of the offsets are precomputed by a convolution preprocessor.

First claim

The invention claimed is:

1 . A computer-implemented method for performing a multi-convolution operation, the method comprising:
selecting a first start address based on a first destination address included in a first image tile that is stored in a first memory; identifying a first offset based on the first destination address; computing a first source address included in an image batch that is stored in a second memory based on the first start address and the first offset; copying data from the first source address to the first destination address; and after copying the data, performing one or more matrix multiplication operations between the first image tile and a first filter tile.

2 . The computer-implemented method of claim 1 , wherein the first memory comprises a shared memory, and the second memory comprises a parallel processing memory.

3 . The computer-implemented method of claim 1 , wherein the first filter tile is stored in the first memory, and further comprising:
computing a filter source address based on the first destination address; and copying data stored in a filter stack at the filter source address to a filter destination address included in the first filter tile.

4 . The computer-implemented method of claim 1 , wherein selecting the first start address comprises:
associating the first destination address with a column of a virtual image matrix; and performing one or more operations that map the column to an address included in the image batch.

5 . The computer-implemented method of claim 1 , wherein identifying the first offset comprises:
associating the first destination location with a row of a virtual image matrix; and retrieving a value included in an offset sequence based on the row.

6 . The computer-implemented method of claim 5 , further comprising generating the offset sequence based on a deterministic relationship between the image batch and the virtual image matrix.

7 . The computer-implemented method of claim 1 , further comprising assigning the first image tile to a first thread group, and configuring at least one thread in the thread group to compute the first source address.

8 . The computer-implemented method of claim 7 , further comprising assigning a second image tile to a second thread group, and configuring at least one thread in the second thread group to compute a second source address included in the image batch based on a second start address and the first offset.

9 . The computer-implemented method of claim 8 , wherein the first source address and the second source address are computed substantially in parallel.

10 . A non-transitory, computer-readable storage medium including instructions that, when executed by a processor, cause the processor to perform a multi-convolution operation, by performing the steps of:
selecting a first start address based on a first destination address included in a first image tile that is stored in a first memory; identifying a first offset based on the first destination address; computing a first source address included in an image batch that is stored in a second memory based on the first start address and the first offset; copying data from the first source address to the first destination address; and after copying the data, performing one or more matrix multiplication operations between the first image tile and a first filter tile.

11 . The non-transitory computer-readable storage medium of claim 10 , wherein the first memory comprises a shared memory, and the second memory comprises a parallel processing memory.

12 . The non-transitory computer-readable storage medium of claim 10 , wherein the first filter tile is stored in the first memory, and further comprising:
computing a filter source address based on the first destination address; and copying data stored in a filter stack at the filter source address to a filter destination address included in the first filter tile.

13 . The non-transitory computer-readable storage medium of claim 10 , wherein selecting the first start address comprises:
associating the first destination address with a column of a virtual image matrix; and performing one or more operations that map the column to an address included in the image batch.

14 . The non-transitory computer-readable storage medium of claim 10 , wherein identifying the first offset comprises:
associating the first destination location with a row of a virtual image matrix; and retrieving a value included in an offset sequence based on the row.

15 . The non-transitory computer-readable storage medium of claim 14 , further comprising generating the offset sequence based on a deterministic relationship between the image batch and the virtual image matrix.

16 . The non-transitory computer-readable storage medium of claim 10 , further comprising configuring at least one thread in a second thread group to compute a second source address included in the image batch based on the first start address and a second offset.

17 . The non-transitory computer-readable storage medium of claim 10 , further comprising performing one or more output formatting operations based on the output matrix to generate an output batch.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein a first layer included in a convolutional neural network includes at least the image batch, and a second layer included in the convolution neural network includes at least the output batch.

19 . A system configured to perform a multi-convolution operation, the system comprising:
a first memory; a second memory; and a convolution engine coupled to both the first memory and the second memory, and configured to:
identify a first offset included in an offset sequence based on a first destination address included in a first image tile that is stored in the first memory;
compute a first source address included in an image batch that is stored in the second memory based on the first offset;
copy data from the first source address to the first destination address; and
after copying the data, perform one or more matrix multiplication operations between the first image tile and a first filter tile.

20 . The system of claim 19 , wherein the first memory comprises a shared memory, and the second memory comprises a parallel processing memory.

Assignees

• Nvidia Corp

Inventors

• Woolley Jr John Clifton
• Tran John

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06V40/172 Primary

  Classification, e.g. identification · CPC title

• G06V10/56

  relating to colour · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06V10/95 Primary

  structured as a network, e.g. client-server architectures · CPC title