Processing-in-memory concurrent processing system and method

Patent metadata

Field	Value
Publication number	US-11468001-B1
Application number	US-202117217792-A
Country	US
Kind code	B1
Filing date	Mar 30, 2021
Priority date	Mar 30, 2021
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

A processing system includes a processing unit and a memory device. The memory device includes a processing-in-memory (PIM) module that performs processing operations on behalf of the processing unit. An instruction set architecture (ISA) of the PIM module has fewer instructions than an ISA of the processing unit. Instructions received from the processing unit are translated such that processing resources of the PIM module are virtualized. As a result, the PIM module concurrently performs processing operations for multiple threads or applications of the processing unit.

First claim

What is claimed is:

1. A memory device comprising:
a plurality of storage devices configured to store data on behalf of programs executing at a processing unit, wherein the processing unit is external to the memory device; and
a processing-in-memory (PIM) circuit configured to process PIM commands in response to kernel instructions corresponding to the programs from the processing unit, wherein an instruction set architecture (ISA) comprising the PIM commands and implemented by the PIM circuit has fewer instructions than an ISA implemented by the processing unit, and wherein processing resources of the PIM circuit are configured to be virtualized such that the PIM circuit concurrently processes PIM commands from multiple PIM kernels.

2. The memory device of claim 1 , further comprising:
a PIM command store (PCS) circuit configured to store the PIM commands and to provide the PIM commands to the PIM circuit, wherein the PCS circuit is configured to be virtualized.

3. The memory device of claim 2 , further comprising:
a PIM kernel scheduler configured to combine portions of at least two kernel instructions corresponding to at least two PIM kernels and store a single PIM command corresponding to the portions of the at least two kernel instructions at the PCS circuit.

4. The memory device of claim 3 , wherein:
the PIM kernel scheduler is further configured to combine the portions of the at least two kernel instructions based on a combination metric.

5. The memory device of claim 4 , wherein:
the combination metric is based on commonalities of respective kernel instructions between PIM kernels awaiting execution at the PIM circuit.

6. The memory device of claim 4 , wherein:
the combination metric is based on a weighted combination of kernel instructions based on expected resource usage of corresponding PIM kernels.

7. The memory device of claim 4 , wherein:
the combination metric is based on a dependency graph of the PIM kernels received from the processing unit.

8. The memory device of claim 3 , wherein:
the plurality of storage devices comprises a register file configured as a register bank.

9. The memory device of claim 8 , wherein:
the PIM kernel scheduler is further configured to translate virtual register addresses indicated by the kernel instructions into physical register addresses of the register file.

10. The memory device of claim 8 , wherein:
a first register of the register file is configured to be allocated to a first PIM kernel of the at least two PIM kernels while a second register of the register file is allocated to a second PIM kernel of the at least two PIM kernels.

11. The memory device of claim 3 , wherein:
the PIM kernel scheduler is further configured to schedule, based on the kernel instructions from the processing unit, the PIM commands for execution at the PIM circuit.

12. The memory device of claim 1 , wherein:
the PIM circuit is a PIM arithmetic logic unit (ALU).

13. A method comprising:
receiving, at a processing-in-memory (PIM) arithmetic logic unit (ALU) from a processing unit, kernel instructions for a plurality of kernels to be concurrently processed by the PIM ALU wherein an instruction set architecture (ISA) implemented by the PIM ALU has fewer instructions than an ISA implemented by the processing unit;
retrieving PIM ALU commands corresponding to the kernel instructions from a PIM command store (PCS) circuit, wherein at least one PIM ALU command corresponds to at least two kernels; and
performing the PIM ALU commands comprising storing data for at least two of the kernels in a register bank of a memory that includes the PIM ALU.

14. The method of claim 13 , wherein:
the kernel instructions include:
a respective command index that indicates a respective PIM ALU command and
at least one respective bank local address.

15. The method of claim 13 , further comprising:
prior to retrieving the PIM ALU commands from the PCS circuit, in response to detecting that at least two of the PIM ALU commands corresponding to the kernel instructions request a same operation, storing an indication of the at least two PIM ALU commands at a single entry of the PCS circuit.

16. The method of claim 13 , wherein:
retrieving the PIM ALU commands comprises delaying execution of at least one of the PIM ALU commands based on dependencies of corresponding PIM kernels.

17. A system comprising:
a processing unit configured to execute a plurality of programs; and
a memory device configured to receive kernel instructions from the processing unit, wherein the kernel instructions cause a processing-in-memory (PIM) circuit of the memory device to concurrently process PIM commands for a plurality of PIM kernels, wherein concurrently processing PIM commands comprises storing data for at least some of the PIM kernels in a plurality of storage devices of the memory device configured to act as a virtual register bank, and wherein an instruction set architecture (ISA) comprising the PIM commands and implemented by the PIM circuit has fewer instructions than an ISA implemented by the processing unit.

18. The system of claim 17 , wherein:
the memory device comprises a PIM command store (PCS) circuit configured to store the PIM commands and to provide the PIM commands to the PIM circuit; and
the processing unit comprises a portion of a PIM kernel scheduler that translates application addresses into PCS addresses and sends the PCS addresses as part of the kernel instructions.

19. The system of claim 17 , wherein:
the memory device comprises a second PIM circuit configured to concurrently process second PIM commands for a second plurality of PIM kernels.

20. The system of claim 17 , wherein:
the processing unit comprises a coprocessor configured to send the kernel instructions to the memory device.

Assignees

• Advanced Micro Devices Inc

Inventors

• Hassaan Muhammad Amber
• Chu Michael L
• Aji Ashwin

Classifications

• G06F15/7821 Primary

  Tightly coupled to memory, e.g. computational memory, smart memory, processor in memory · CPC title

• Y02D10/00

  Energy efficient computing, e.g. low power processors, power management or thermal management · CPC title

• G06F9/3004

  to perform operations on memory · CPC title

• G06F7/57

  Arithmetic logic units [ALU], i.e. arrangements or devices for performing two or more of the operations covered by groups G06F7/483 – G06F7/556 or for performing logical operations {(G06F7/49, G06F7/491 take precedence)} · CPC title

• G06F9/3877

  using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12) · CPC title