Approach for performing efficient memory operations using near-memory compute elements

Patent metadata

Field	Value
Publication number	US-12235756-B2
Application number	US-202117557568-A
Country	US
Kind code	B2
Filing date	Dec 21, 2021
Priority date	Dec 21, 2021
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

Near-memory compute elements perform memory operations and temporarily store at least a portion of address information for the memory operations in local storage. A broadcast memory command is then issued to the near-memory compute elements that causes the near-memory compute elements to perform a subsequent memory operation using their respective address information stored in the local storage. This allows a single broadcast memory command to be used to perform memory operations across multiple memory elements, such as DRAM banks, using bank-specific address information. In one implementation, the approach is used to process workloads with irregular updates to memory while consuming less command bus bandwidth than conventional approaches. Implementations include using conditional flags to selectively designate address information in local storage that is to be processed with the broadcast memory command.

First claim

The invention claimed is:

1. A memory module, comprising:
a first compute element configured to:
perform a first memory operation on a first memory element of a plurality of memory elements and store at least a portion of address information for the first memory operation in a local storage of the first compute element, and
perform a second memory operation on the first memory element using the at least a portion of address information for the first memory operation, wherein the second memory operation is performed in response to a broadcast memory command issued to the plurality of memory elements.

2. The memory module of claim 1 , wherein the at least a portion of address information for the first memory operation includes a column address.

3. The memory module of claim 1 , wherein the first compute element is further configured to
store, in the local storage of the first compute element, data that indicates that the at least a portion of address information for the first memory operation is to be used for the second memory operation.

4. The memory module of claim 1 , wherein:
the first memory operation produces a result, and
the first compute element is further configured to store the result in the local storage of the first compute element and use the result to perform the second memory operation.

5. The memory module of claim 1 , wherein the first compute element is further configured to:
determine second address information based upon the at least a portion of address information for the first memory operation and use the second address information to perform a third memory operation.

6. The memory module of claim 1 , wherein the first memory operation is performed in response to a first memory command issued to the first memory element.

7. The memory module of claim 6 , further comprising:
a second compute element configured to:
perform a third memory operation at a second memory element of the plurality of memory elements and store at least a portion of address information for the third memory operation in a local storage of the second compute element, wherein the third memory operation is performed in response to a third memory command issued to the second memory element and wherein the at least a portion of address information for the third memory operation is different than the at least a portion of address information for the first memory operation,
perform a fourth memory operation at the second memory element using the at least a portion of address information for the third memory operation, and
wherein the second memory operation and the fourth memory operation are performed in response to the broadcast memory command.

8. The memory module of claim 1 , wherein the local storage of the first compute element comprises one or more local registers or one or more buffers.

9. The memory module of claim 1 , wherein:
the first compute element is further configured to generate an error in response to determining that another memory command includes the at least a portion of the address information for the first memory operation and was received prior to a second memory command for the second memory operation.

10. A memory controller comprising:
processing logic configured to:
issue a first memory command to a first near-memory compute element, wherein the first memory command specifies that at least a portion of address information for the first memory command is to be stored in a first local storage for the first near-memory compute element,
issue a second memory command to a second near-memory compute element, wherein the second memory command specifies that at least a portion of address information for the second memory command is to be stored in a second local storage for the second near-memory compute element, wherein the at least a portion of address information for the second memory command is different than the at least a portion of address information for the first memory command, and
issue a broadcast memory command to the first and second near-memory compute elements, wherein processing of the broadcast memory command by the first near-memory compute element causes a first memory operation to be performed using the at least a portion of address information for the first memory command stored in the first local storage, and processing of the broadcast memory command by the second near-memory compute element causes a second memory operation to be performed using the at least a portion of address information for the second memory command stored in the second local storage.

11. The memory controller of claim 10 , wherein the at least a portion of address information for the first memory command and the at least a portion of address information for the second memory command includes a column address.

12. The memory controller of claim 10 , wherein the first memory command specifies that the at least a portion of address information for the first memory command is to be used for a subsequent broadcast memory command.

13. The memory controller of claim 10 , wherein:
processing of the broadcast memory command by the first near-memory compute element causes the first memory operation to be performed using a result of the first memory command and the at least a portion of address information for the first memory command stored in the first local storage, and
processing of the broadcast memory command by the second near-memory compute element causes the second memory operation to be performed using a result of the second memory command and the at least a portion of address information for the second memory command stored in the second local storage.

14. The memory controller of claim 10 , wherein:
the first memory operation specifies a particular row for the first near-memory compute element, and
the processing logic is further configured to perform one or more of:
keep the particular row for the first near-memory compute element open at least until the broadcast memory command is queued, or
reopen the particular row for the first near-memory compute element for the broadcast memory command.

15. The memory controller of claim 10 , wherein:
the first memory operation specifies a particular row for the first near-memory compute element, and
the processing logic is further configured to enqueue, with the first memory command, one or more other memory commands to the particular row for the first near-memory compute element.

16. The memory controller of claim 10 , wherein:
the first memory command specifies a particular row for the first near-memory compute element, and
the processing logic is further configured to not issue another memory command to the first near-memory compute element for the particular row after the first memory command and before the broadcast memory command.

17. The memory controller of claim 16 , wherein:
the first memory command further specifies a particular column for the first near-memory compute element, and
the processing logic is further configured to not issue another memory command to the first near-memory compute element for the particular row and column after the first memory command and before the broadcast memory command.

18. The memory controller of claim 10 , wherein the processing logic is further configured to:
track a number of near-memory compute commands that are issued to each of the first and second near-memory compute elements, wherein each of the near-memory compute commands specifies that a result and at least a portion of the address information for the near-memory

Assignees

• Advanced Micro Devices Inc

Inventors

• Aga Shaizeen
• Alsop Johnathan
• Jayasena Nuwan

Classifications

• G06F12/06 Primary

  Addressing a physical block of locations, e.g. base addressing, module addressing, memory dedication (G06F12/08 takes precedence) · CPC title

• G06F12/0284 Primary

  Multiple user address space allocation, e.g. using different base addresses (interprocessor communication G06F15/163) · CPC title