Direct data move between DRAM and storage on a memory module

Patent metadata

Field	Value
Publication number	US-10248328-B2
Application number	US-201715665246-A
Country	US
Kind code	B2
Filing date	Jul 31, 2017
Priority date	Nov 7, 2013
Publication date	Apr 2, 2019
Grant date	Apr 2, 2019

Abstract

A computer system comprises a processor, a memory module and input/output devices. The memory module includes a circuit board, a volatile memory unit mounted on the circuit board, a non-volatile memory unit mounted on the circuit board and a control circuit mounted on the circuit board. The volatile memory unit comprises DRAM devices, and the non-volatile memory unit comprises flash memory. The processor is configured to execute an operating system (OS) and an application program and to present a memory address space to the application program. The memory address space including a memory mapped input/output (MMIO) space mapped to the I/O devices, a pseudo MMIO (PMMIO) space mapped to the non-volatile memory unit, and a DRAM space mapped to the volatile memory unit, the PMMIO space including a system main memory local storage (MMLS) area and a memory channel storage area, wherein the DRAM space is partitioned into memory pages, and the MCS space is partitioned into storage blocks.

First claim

We claim:

1. A computer system, comprising:
a system memory bus providing a memory channel, the memory channel comprising a control/address (C/A) bus and a data bus;
a processor coupled to the system memory bus;
a memory module including a circuit board having electrical contacts coupled to the system memory bus, a volatile memory unit mounted on the circuit board, a non-volatile memory unit mounted on the circuit board and a control circuit mounted on the circuit board, the volatile memory unit comprising DRAM devices, and the non-volatile memory unit comprising flash memory;
input/output devices coupled to the system bus;
wherein the processor is configured to execute an operating system (OS) and an application program and to present a memory address space to the application program, the memory address space including a memory mapped input/output (MMIO) space mapped to the I/O devices, a pseudo MMIO (PMMIO) space mapped to the non-volatile memory unit, and a DRAM space mapped to the volatile memory unit, the PMMIO space including a system main memory local storage (MMLS) area and a memory channel storage (MCS) area, wherein the DRAM space is partitioned into memory pages, and the MCS area is partitioned into storage blocks;
wherein the operating system is configured to request direct data transfer for one or more storage blocks in the MCS area between the volatile memory unit and the non-volatile memory unit to store or retrieve files associated with the application program by causing a direct data transfer command to be sent to the memory module via the memory channel, the direct data transfer command including an address in the DRAM space and an address in the MCS space; and
wherein the control circuit is configured to receive the direct data transfer command from the memory channel, and to move the data for the one or more storage blocks between the volatile memory unit and the non-volatile memory unit without any of the data for the one or more storage blocks going through the memory channel;
wherein the processor is further configured to respond to a request from the application program to access a memory page not loaded in the volatile memory by causing a page-in command to be transmitted via the data bus to the memory module, the page-in command including information for the memory page and one or more address locations in the MMLS space where the requested data is to be retrieved;
wherein the control circuit is configured to receive the page-in command from the memory channel, to read the data for the memory page from the MMLS space without any of the data for the memory page going through the memory channel;
wherein the processor is further configured to cause a dummy write command to be transmitted to the volatile memory unit, the dummy write command including an address associated with the memory page;
wherein the volatile memory unit further comprises a registered control device (RCD) that is configured to receive the dummy write command from the memory channel, and wherein the DRAM devices are configured to receive the data for the memory page from the control circuit in response to control/address signals from the RCD, the control/address signals being derived from the dummy write command.

2. The computer system of claim 1 , wherein the memory module further comprises a serial presence detect (SPD) device coded with a memory density bigger than a memory density of the volatile memory unit so as to allow the processor to present the PMMIO space in addition to the DRAM space and the MMIO space.

3. The computer system of claim 1 , further comprising a router circuit controlled by the control circuit, wherein the direct data transfer command include read or write command transmitted over the C/A bus and a command packet transmitted over the data bus, wherein the control circuit is configured to recognize the read or write command as part of the direct data transfer command and set up the router circuit to route the command packet to the control circuit.

4. The computer system of claim 1 , wherein processor is further configured to perform one or more memory read or write operations after transmitting the page-in command and before transmitting the dummy write command.

5. The computer system of claim 4 , wherein the control circuit is further configured to notify the processor when the data for the memory page is ready to be written into the volatile memory unit.

6. The computer system of claim 1 , wherein the DRAM space is divided into a user space accessible by the application program and a kernel space accessible by the OS but not by the application program, and wherein the address in the DRAM space is in the user space.

7. The computer system of claim 1 , wherein the control circuit comprises status registers configured to store status data about an estimated wait time for the direct data transfer command, wherein the memory address space further includes a status register space, and wherein the OS is further configured to initiate a memory read operation addressed to the status registers after the direct data transfer command is sent to the memory module.

8. The computer system of claim 7 , wherein the status registers include ECC bits for storing an ECC code associated with the status data.

9. The computer system of claim 7 , further comprising a memory controller configured to write pre-defined status codes corresponding to possible status situations into the memory module.

10. The computer system of claim 1 , wherein the memory module further includes buffer memory coupled to the control circuit via a set of internal C/A signal lines and a set of internal data lines, wherein the direct data transfer command includes a read command, and wherein the control circuit is configured to store the data for the one or more storage blocks into the buffer memory after read the data for the one or more storage blocks from the non-volatile memory unit, and wherein the control circuit is further configured to perform error correction on the data for the one or more storage blocks by reading each portion of the data for the one or more storage blocks from the buffer memory and writing the portion of the data for the one or more storage blocks back into the buffer memory.

11. A computer system, comprising:
a system memory bus providing a memory channel, the memory channel comprising a control/address (C/A) bus and a data bus;
a memory controller coupled to the system memory bus;
a processor coupled to the memory controller;
a memory module coupled to the memory controller via the system memory bus, the memory module comprising:
a volatile memory unit coupled to the memory channel, the volatile memory unit including a registered control device (RCD) configured to receive input C/A signals from the C/A bus and memory devices configured to perform memory operations in response to output C/A signals from the RCD that are derived from the input C/A signals;
a non-volatile (NV) memory unit; and
a module control device coupled to the volatile memory unit, the non-volatile memory unit, and the memory channel;
wherein the processor is configured to execute an operating system (OS) including a memory module driver configured to cause the memory controller to issue an NV access request to the memory module via the memory channel to transfer first data from the non-volatile memory unit to the volatile memory unit, and to issue a series of dummy write memory commands via the C/A bus after issuing the NV access request, the series of dummy write memory commands including memory addresses related to the NV access operation;
wherein the module control device is configured to perform an NV access operati

Assignees

• Netlist Inc

Inventors

• Lee Hyun
• Wang Sheng

Classifications

• G06F13/4068

  Electrical coupling · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title

• G06F3/0679

  Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP] · CPC title

• G06F3/0644

  Management of space entities, e.g. partitions, extents, pools · CPC title

• G06F3/0646

  Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems · CPC title