Method and apparatus for uniform memory access in a storage cluster

Patent metadata

Field	Value
Publication number	US-12411765-B2
Application number	US-202318468712-A
Country	US
Kind code	B2
Filing date	Sep 17, 2023
Priority date	Apr 25, 2016
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

A node in a network including a plurality of nodes comprises persistent memory; non-persistent memory, a node control device operatively coupled to the persistent memory and the non-persistent memory, a local interface for interfacing with the local server in the UMA node, and a network interface for interfacing with other nodes. The node control device is configured to translate between a local unified memory access (UMA) address space accessible by applications running on a local server and a global UMA address space that is mapped to a physical UMA address space. The physical UMA address space includes physical address spaces associated with different UMA nodes in the cluster of UMA nodes. Thus, a server in the UMA network can access the physical address spaces at other UMA nodes without going through the servers in the other UMA nodes.

First claim

We claim:

1. A node in a network including a plurality of nodes, comprising:
a processing unit and a memory unit, the memory unit including a first interface configured to interface with the processing unit and a second interface configured to interface with one or more memory units in one or more other nodes of the plurality of nodes without going through any processing unit in any of the one or more other nodes; and
wherein the memory unit includes a node control device configurable to:
receive a memory access request from the processing unit via the first interface;
in response to the memory access request including a first address in a local unified memory access (UMA) address space accessible by applications running on the processing unit, translate the first address to a second address in a global UMA address space, the global UMA address space being mapped to a physical UMA address space including physical address spaces associated with memory units in the plurality of nodes; and
transmit a first request packet via the second interface, the first request packet including the second address or a first target address for a first target node among the plurality of nodes; and

wherein the network further includes memory access paths, each of the memory access paths is between the second interface of one memory unit in one node and the second interface of another memory unit in another node, and none of the memory access paths go through any of the processing unit in any of the plurality of nodes.

2. The node of claim 1 , wherein the first interface includes one or more of:
a main memory interface coupled to a memory bus in the node;
a peripheral component interconnect (PCI) interface;
a PCI express (PCIe) interface;
a Fiber Channel (FC) interface; and
an Internet Protocol (IP) interface.

3. The node of claim 2 , wherein the second interface includes one or more of an Internet Protocol (IP) interface and a Fiber Channel (FC) interface.

4. The node of claim 3 , wherein the first request packet further includes an access type and a size of data to be transferred in response to the memory access request.

5. The node of claim 1 , wherein the node control device is further configurable to:
receive via the second interface an acknowledgement packet indicating availability of the first target node;
form a command packet including a packet type, a source address identifying the node, the first target address of the first target node, a memory address at the target node, and a data size;
transmit the first command packet via the second interface; and
communicate one or more data packets with the target node via the second interface subsequent to transmitting the command packet, each of the one or more data packets identifying the target node.

6. The node of claim 5 , wherein the node control device is further configurable to:
receive a second request packet via the second interface, the second request packet including a second target address and a requester address;
receive a second command packet via the second interface in response to the second target address indicating that the node is targeted by the second request packet; and
communicate one or more second data packets via the second interface in accordance with the second command packet.

7. The node of claim 6 , wherein the node control device is further configurable to:
receive a third request packet via the second interface, the third request packet including a third target address and a requester address; and
repeat the third request packet to a next node in the plurality of nodes in response to the third target address indicating that the node is not targeted by the third request packet.

8. The node of claim 1 , wherein the node control device is further configurable to:
modify the local UMA address space and the global UMA address space in response to a new node being inserted into the system; and
broadcast reservation of a space for the new node in the global UMA address space to other nodes in the system.

9. The node of claim 1 , wherein the node control device is further configurable to:
receive from an application running on the processing unit a request for a UMA storage area;
send the request to a designated UMA node;
receive a response from the designated UMA node that the UMA storage area has been allocated to the application;
inform the application that the UMA storage area has been allocated to the application; and
broadcast to other nodes in the system that the UMA storage area has been allocated to the application.

10. The node of claim 1 , wherein the memory unit includes dynamic random access memory (DRAM) and persistent memory, wherein:
the first interface includes a main memory interface coupled to a memory bus in the node, and a data signal router coupled to the main memory interface and controlled by the node control device; and
the data signal router is configurable by the node control device to selectively route data signals between the memory bus and the DRAM, between the DRAM and the node control device, or between the memory bus and the node control device.

11. A method, comprising:
in a node in a network including a plurality of nodes, the node including a processing unit and a memory unit, the memory unit including a first interface to interface with the processing unit and a second interface to interface with one or more memory units in one or more other nodes of the plurality of nodes without going through any processing unit in any of the one or more other nodes,
receiving a memory access request from the processing unit via the first interface;
in response to the memory access request including a first address in a local unified memory access (UMA) address space accessible by applications running on the processing unit, translating the first address to a second address in a global UMA address space, the global UMA address space being mapped to a physical UMA address space including physical address spaces associated with memory units in the plurality of nodes; and
transmitting a first request packet via the second interface, the first request packet including the second address or a first target address for a first target node among the plurality of nodes.

12. The method of claim 11 , wherein the first interface includes one or more of:
a main memory interface coupled to a memory bus in the node;
a peripheral component interconnect (PCI) interface;
a PCI express (PCIe) interface;
a Fiber Channel (FC) interface; and
an Internet Protocol (IP) interface.

13. The method of claim 12 , wherein the second interface includes one or more of an Internet Protocol (IP) interface and a Fiber Channel (FC) interface.

14. The method of claim 11 , wherein the first request packet further includes an access type and a size of data to be transferred in response to the memory access request.

15. The method of claim 11 , further comprising:
receiving via the second interface an acknowledgement packet indicating availability of the first target node;
forming a command packet including a packet type, a source address identifying the node, the first target address of the first target node, a memory address at the target node, and a data size;
transmitting the first command packet via the second interface; and
communicating one or more data packets with the target node via the second interface subsequent to transmitting the command packet, each of the one or more data packets identifying the target node.

16. The method of claim 15 , further comprising:
receiving a second re

Assignees

• Netlist Inc

Inventors

• Lee Hyun
• Ryu Junkil

Classifications

• H04L65/40

  Support for services or applications · CPC title

• G06F12/1054

  the data cache being concurrently physically addressed · CPC title

• G06F12/0284

  Multiple user address space allocation, e.g. using different base addresses (interprocessor communication G06F15/163) · CPC title

• G06F12/109

  for multiple virtual address spaces, e.g. segmentation (G06F12/1036 takes precedence) · CPC title

• G06F2212/1044

  Space efficiency improvement · CPC title