Hardware double buffering using a special purpose computational unit

Patent metadata

Field	Value
Publication number	US-10496326-B2
Application number	US-201916240459-A
Country	US
Kind code	B2
Filing date	Jan 4, 2019
Priority date	Jul 5, 2017
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

Methods, systems, and apparatus, including an apparatus for transferring data using multiple buffers, including multiple memories and one or more processing units configured to determine buffer memory addresses for a sequence of data elements stored in a first data storage location that are being transferred to a second data storage location. For each group of one or more of the data elements in the sequence, a value of a buffer assignment element that can be switched between multiple values each corresponding to a different one of the memories is identified. A buffer memory address for the group of one or more data elements is determined based on the value of the buffer assignment element. The value of the buffer assignment element is switched prior to determining the buffer memory address for a subsequent group of one or more data elements of the sequence of data elements.

First claim

What is claimed is:

1. An apparatus for transferring data, the apparatus comprising:
a plurality of memories; and
one or more processors arranged to:
determine buffer memory addresses for data of an N-dimensional tensor stored in a first data storage location that is being transferred to a second data storage location, wherein N is an integer that is equal to or greater than two, the determining comprising:
identifying a current value of a buffer assignment element that can be switched between a plurality of values each corresponding to a different one of the plurality of memories; and
assigning a first portion of the data of the N-dimensional tensor to the memory corresponding to the current value of the buffer assignment element until the memory corresponding to the current value of the buffer assignment element is full, including:
determining the buffer memory address for each data element of the first portion of data based on at least on a combination of (i) a base address for the plurality of memories, (ii) a memory address offset value for the memory corresponding to the current value of the buffer assignment element, and (iii) a memory offset value for the data element, wherein the memory offset value for each data element is based on current index values of multiple loops in a loop nest used to traverse the N-dimensional tensor;

transfer the first portion of the data of the N-dimensional tensor to a respective memory location of the memory corresponding to the current value of the buffer assignment element using the determined buffer memory address for each data element; and
switch the value of the buffer assignment element prior to determining buffer memory addresses for a next portion of the data of the N-dimensional tensor.

2. The apparatus of claim 1 , wherein the one or more processors are arranged to switch the value of the buffer assignment element in response to determining that the memory corresponding to the current value of the buffer assignment element is full.

3. The apparatus of claim 1 , wherein the memory address offset value for one of the memories is zero and the memory address offset value for each other memory is non-zero.

4. The apparatus of claim 1 , wherein:
each memory of the plurality of memories are buffers that each have a first data storage capacity; and
the first data storage location and the second data storage location each comprise at least a second data storage capacity that is greater than the first data storage capacity.

5. The apparatus of claim 1 , wherein the one or more processors are arranged to transfer the first portion of the data of the N-dimensional tensor from the respective memory location of the memory corresponding to the current value of the buffer assignment element to the second data storage location.

6. The apparatus of claim 1 , wherein:
the value of the buffer assignment element is a loop variable for a loop used to switch the value of the buffer assignment element; and
switching the value of the buffer assignment element prior to determining buffer memory addresses for a next portion of the data of the N-dimensional tensor comprises iterating the loop variable in response to determining that the memory corresponding to the current value of the buffer assignment element is full.

7. A method performed by a computing system for transferring data, the method comprising:
determining buffer memory addresses for data of an N-dimensional tensor stored in a first data storage location that is being transferred to a second data storage location, wherein N is an integer that is equal to or greater than two, the determining comprising:
identifying a current value of a buffer assignment element that can be switched between a plurality of values each corresponding to a different one of a plurality of memories; and
assigning a first portion of the data of the N-dimensional tensor to the memory corresponding to the current value of the buffer assignment element until the memory corresponding to the current value of the buffer assignment element is full, including:
determining the buffer memory address for each data element of the first portion of data based on at least on a combination of (i) a base address for the plurality of memories, (ii) a memory address offset value for the memory corresponding to the current value of the buffer assignment element, and (iii) a memory offset value for the data element, wherein the memory offset value for each data element is based on current index values of multiple loops in a loop nest used to traverse the N-dimensional tensor;

transferring the first portion of the data of the N-dimensional tensor to a respective memory location of the memory corresponding to the current value of the buffer assignment element using the determined buffer memory address for each data element; and
switching the value of the buffer assignment element prior to determining buffer memory addresses for a next portion of the data of the N-dimensional tensor.

8. The method of claim 7 , wherein the value of the buffer assignment element is switched in response to determining that the memory corresponding to the current value of the buffer assignment element is full.

9. The method of claim 7 , wherein the memory address offset value for one of the memories is zero and the memory address offset value for each other memory is non-zero.

10. The method of claim 7 , wherein:
each memory of the plurality of memories are buffers that each have a first data storage capacity; and
the first data storage location and the second data storage location each comprise at least a second data storage capacity that is greater than the first data storage capacity.

11. The method of claim 7 , further comprising transferring the first portion of the data of the N-dimensional tensor from the respective memory location of the memory corresponding to the current value of the buffer assignment element to the second data storage location.

12. The method of claim 7 , wherein:
the value of the buffer assignment element is a loop variable for a loop used to switch the value of the buffer assignment element; and
switching the value of the buffer assignment element prior to determining buffer memory addresses for a next portion of the data of the N-dimensional tensor comprises iterating the loop variable in response to determining that the memory corresponding to the current value of the buffer assignment element is full.

13. A system for transferring data, the system comprising:
a plurality of memories; and
one or more processing units that include one or more math units, the one or more processing units configured to:
determine buffer memory addresses for data of an N-dimensional tensor stored in a first data storage location that is being transferred to a second data storage location, wherein N is an integer that is equal to or greater than two, the determining comprising:
identifying a current value of a buffer assignment element that can be switched between a plurality of values each corresponding to a different one of the plurality of memories; and
assigning a first portion of the data of the N-dimensional tensor to the memory corresponding to the current value of the buffer assignment element until the memory corresponding to the current value of the buffer assignment element is full, including:
determining the buffer memory address for each data element of the first portion of data based on at least on a combination of (i) a base address for the plurality of memories, (ii) a memory address offset value for the memory corresponding to the current value of the buffer assignment element, and (ii

Assignees

• Google Llc

Inventors

• Temam Olivier
• Khaitan Harshit
• Narayanaswami Ravi
• Woo Dong Hyuk

Classifications

• G06F2212/1016

  Performance improvement · CPC title

• G06N3/063

  using electronic means · CPC title

• G06F2212/302

  In image processor or graphics adapter · CPC title

• G06F3/0656 Primary

  Data buffering arrangements · CPC title

• G06F12/0215

  with look ahead addressing means · CPC title