Neural network accelerator in DIMM form factor

Patent metadata

Field	Value
Publication number	US-12443837-B2
Application number	US-202016994990-A
Country	US
Kind code	B2
Filing date	Aug 17, 2020
Priority date	Aug 17, 2020
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

The technology relates to a neural network dual in-line memory module (NN-DIMM), a microelectronic system comprising a CPU and a plurality of the NN-DIMMs, and a method of transferring information between the CPU and the plurality of the NN-DIMMS. The NN-DIMM may include a module card having a plurality of parallel edge contacts adjacent to an edge of a slot connector thereof and configured to have the same command and signal interface as a standard dual in-line memory module (DIMM). The NN-DIMM may also include a deep neural network (DNN) accelerator affixed to the module card, and a bridge configured to transfer information between the DNN accelerator and the plurality of parallel edge contacts via a DIMM external interface.

First claim

The invention claimed is:

1. A neural network dual in-line memory module (NN-DIMM), comprising:
a module card having a plurality of parallel edge contacts adjacent an edge of a slot connector thereof and configured to have the same command and signal interface as a standard dual in-line memory module (DIMM);
a deep neural network (DNN) accelerator affixed to the module card; and
a bridge configured to transfer information between the DNN accelerator and the plurality of parallel edge contacts via a DIMM external interface, and
wherein the bridge includes control registers and on-chip memory are accessible as a standard DRAM to a host CPU, wherein the bridge is configured to map the control registers and the on-chip memory to one row of a plurality of banks of the NN-DIMM, and wherein access of the control registers or the on-chip memory is limited to the one row of the plurality of banks.

2. The NN-DIMM of claim 1 , comprising a DNN accelerator chip affixed to the module card, the DNN accelerator chip comprising the DNN accelerator, the bridge, and the DIMM external interface.

3. The NN-DIMM of claim 2 , wherein the DNN accelerator chip is an off-the-shelf DNN accelerator chip.

4. The NN-DIMM of claim 1 , further comprising one or more memory chips affixed to the module card and connected with the bridge via an interface.

5. The NN-DIMM of claim 4 , further comprising a memory controller connected with the one or more memory chips and the bridge.

6. The NN-DIMM of claim 1 , wherein the bridge includes a controller connected with the DNN accelerator, the control registers, the on-chip memory, and the DIMM external interface.

7. The NN-DIMM of claim 1 , further comprising a power connector affixed to the module card and configured to connect to a power cable from a motherboard.

8. The NN-DIMM of claim 1 , wherein the plurality of parallel edge contacts includes 288 parallel edge contacts comprising a standard DDR4 contact configuration.

9. A microelectronic system comprising a CPU affixed to a motherboard, the CPU having a plurality of memory channels each including a respective neural network dual in-line memory module (NN-DIMM) electronically connected with the CPU, each NN-DIMM comprising:
a module card having a plurality of parallel edge contacts adjacent an edge of a slot connector thereof and configured to have the same command and signal interface as a standard dual in-line memory module (DIMM);
a deep neural network (DNN) accelerator affixed to the module card; and
a bridge configured to transfer information between the DNN accelerator and the plurality of parallel edge contacts via a DIMM external interface, and
wherein the bridge includes control registers and on-chip memory are configured to be accessed as a standard DRAM to a host CPU, wherein the bridge is configured to map the control registers and the on-chip memory to one row of a plurality of banks of the NN-DIMM, and wherein the control registers or the on-chip memory access is limited to the one row in the plurality of banks.

10. The microelectronic system of claim 9 , wherein the slot connector of each NN-DIMM is mounted inside a corresponding socket connector affixed to the motherboard.

11. The microelectronic system of claim 9 , wherein each of the memory channels further includes a standard DIMM.

12. The microelectronic system of claim 9 , wherein the plurality of memory channels includes 16 DDR4 memory channels.

13. The microelectronic system of claim 9 , comprising a DNN accelerator chip affixed to the module card, the DNN accelerator chip comprising the DNN accelerator, the bridge, and the DIMM external interface.

14. The microelectronic system of claim 9 , wherein the DNN accelerator chip is an off-the-shelf DNN accelerator chip.

15. A method of transferring information between a host system CPU and a plurality of neural network dual in-line memory modules (NN-DIMMs), the method comprising:
electrically connecting each of the NN-DIMMs to the host system CPU, the CPU having a plurality of memory channels each including a respective one of the NN-DIMMs, each of the NN-DIMMs comprising a deep neural network (DNN) accelerator and a bridge configured to transfer the information between the DNN accelerator and the host system CPU via a DIMM external interface, wherein the bridge of each of the NN-DIMMs includes control registers and on-chip memory that are seen as a standard DRAM to the host system CPU and wherein the bridge is configured to map the control registers and the on-chip memory to one row of a plurality of memory banks of the NN-DIMM;
accessing, by the control registers or the on-chip memory, only the one row in the plurality of banks; and
transferring the information between the host system CPU and the plurality of NN-DIMMs.

16. The method of claim 15 , further comprising the host system controlling the NN-DIMMs through memory read and write operations.

17. The method of claim 15 , wherein each of the NN-DIMMs comprises a module card having a plurality of parallel edge contacts adjacent an edge of a slot connector thereof and having the same command and signal interface as a standard dual in-line memory module (DIMM), the slot connector of each NN-DIMM is mounted inside a corresponding socket connector affixed to a motherboard, and the host system CPU is affixed to the motherboard.

18. The method of claim 15 , wherein the plurality of memory channels includes 16 DDR4 memory channels.

19. The NN-DIMM of claim 1 , wherein the control registers or the on-chip memory accesses the predetermined row in the plurality of banks using only column read or write commands.

20. The NN-DIMM of claim 1 , wherein the plurality of banks comprise memory banks formed from the control registers or on-chip memory and that are accessed by the host CPU using column read or write commands.

Assignees

• Google Llc

Inventors

• Yoon Doe Hyun
• Nai Lifeng
• Ma Peter

Classifications

• G06N3/08

  Learning methods · CPC title

• G06F13/4027

  using bus bridges (G06F13/4022 takes precedence) · CPC title

• G06F9/3877

  using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12) · CPC title

• G06F9/30098

  Register arrangements · CPC title

• G06F15/7803

  System on board, i.e. computer system on one or more PCB, e.g. motherboards, daughterboards or blades · CPC title