System and method for modular hbm chiplet architecture

Patent metadata

Field	Value
Publication number	US-2025298771-A1
Application number	US-202519085669-A
Country	US
Kind code	A1
Filing date	Mar 20, 2025
Priority date	Mar 22, 2024
Publication date	Sep 25, 2025
Grant date	—

Abstract

A modular high-bandwidth memory (HBM) system and method are disclosed. The system includes a compute die including a memory controller, one or more die-to-die (D2D) channels coupled to the compute die; and one or more HBM chiplets coupled to the one or more D2D channels. The one or more chiplets are configured to receive a memory access request, and process the memory access request or forward the memory access request to a subsequent HBM chiplet.

First claim

What is claimed is:

1 . A modular high-bandwidth memory (HBM) system, comprising:
a compute die including a memory controller; one or more die-to-die (D2D) channels coupled to the compute die; and one or more HBM chiplets coupled to the one or more D2D channels and configured to: receive a memory access request, and process the memory access request or forward the memory access request to a subsequent HBM chiplet.

2 . The modular HBM system of claim 1 , wherein the one or more HBM chiplets are configured to:
process the memory access request in case the memory access request corresponds to a local memory address range.

3 . The modular HBM system of claim 1 , wherein the one or more HBM chiplets are configured to:
forward the memory access request to the subsequent HBM chiplet in case the memory access request does not correspond to a local memory address range.

4 . The modular HBM system of claim 1 , wherein the one or more HBM chiplets are connected in a daisy-chain configuration.

5 . The modular HBM system of claim 4 , wherein the memory controller in the compute die is configured to determine a number of nodes connected in the daisy-chain configuration by issuing a discovery command with a node number that is decremented by each of the one or more HBM chiplets connected in the daisy-chain configuration.

6 . The modular HBM system of claim 5 , wherein in a case in which the node number of the discovery command is equal to 0, the discovery command is processed locally.

7 . The modular HBM system of claim 5 , wherein in a case in which the node number of the discovery command is greater than 0, the discovery command is forwarded to a subsequent HBM chiplet in the daisy-chain configuration.

8 . The modular HBM system of claim 5 , wherein in a case in which the node number of the discovery command is an invalid value, an error message is transmitted to the compute die, enabling the compute die to determine the total number of nodes in the daisy-chain configuration.

9 . The modular HBM system of claim 1 , wherein the one or more HBM chiplets are connected in a network-grid configuration, and
wherein each of the one or more HBM chiplets connected in the network-grid configuration is configured to forward the memory access request along multiple routing paths.

10 . The modular HBM system of claim 9 , wherein the one or more HBM chiplets connected in the network-grid configuration include a built-in compute die configured to perform a compute-in-memory (CIM) operation.

11 . A method comprising:
transmitting, by a compute die, a memory access request via one or more die-to-die (D2D) channels to one or more high-bandwidth memory (HBM) chiplets; and receiving, by the one or more HBM chiplets, the memory access request, wherein the one or more HBM chiplets are configured to process the memory access request or forward the memory access request to a subsequent HBM chiplet.

12 . The method of claim 11 , further comprising:
processing the memory access request at the one or more HBM chiplets in response to determining that the memory access request corresponds to a local memory address range.

13 . The method of claim 11 , further comprising:
forwarding the memory access request from the one or more HBM chiplets to the subsequent HBM chiplet in response to determining that the memory access request does not correspond to a local memory address range.

14 . The method of claim 11 , wherein the one or more HBM chiplets are connected in a daisy-chain configuration.

15 . The method of claim 14 , further comprising:
determining, by a memory controller in the compute die, a number of nodes connected in the daisy-chain configuration by issuing a discovery command with a node number that is decremented by each of the one or more HBM chiplets connected in the daisy-chain configuration.

16 . The method of claim 15 , wherein in a case in which the node number of the discovery command is equal to 0, the discovery command is processed locally.

17 . The method of claim 15 , wherein in a case in which the node number of the discovery command is greater than 0, the discovery command is forwarded to a subsequent HBM chiplet in the daisy-chain configuration.

18 . The method of claim 15 , wherein in a case in which the node number of the discovery command is an invalid value, an error message is transmitted to the compute die, enabling the compute die to determine the total number of nodes in the daisy-chain configuration.

19 . The method of claim 11 , wherein the one or more HBM chiplets are connected in a network-grid configuration, and
wherein each of the one or more HBM chiplets connected in the network-grid configuration is configured to forward the memory access request along multiple routing paths.

20 . The method of claim 19 , wherein the one or more HBM chiplets connected in the network-grid configuration include a built-in compute die configured to perform a compute-in-memory (CIM) operation.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yao Chih-Wei
• Yang Li
• Quach Nhon

Classifications

• G06F13/4247 Primary

  on a daisy chain bus · CPC title

• G06F13/1668

  Details of memory controller · CPC title

• G06F13/4265 Primary

  on a point to point bus (G06F13/4247, G06F13/4282 take precedence) · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/0658

  Controller construction arrangements · CPC title