Reliable distributed messaging using non-volatile system memory

Patent metadata

Field	Value
Publication number	US-9740606-B1
Application number	US-201314070362-A
Country	US
Kind code	B1
Filing date	Nov 1, 2013
Priority date	Nov 1, 2013
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

Methods and apparatus for reliable distributed messaging are described. A computer system includes a system memory coupled to one or more processors. The system memory comprises at least a non-volatile portion. A particular location within the non-volatile portion is designated as a target location to which a sender module participating in a communication protocol is granted write permission. A receiver module participating in the communication protocol, subsequent to a failure event that results in a loss of data stored in a volatile portion of the system memory, reads a data item written by the sender program at the target location prior to the failure event. The receiver module performs an operation based on contents of the data item.

First claim

What is claimed is:

1. A system, comprising:
one or more processors, and
a system memory for the one or more processors, wherein the system memory is linked to the one or more processors and comprises one or more non-volatile memory modules and one or more volatile memory modules;
a persistent storage device having a higher access latency than the system memory;
wherein the system memory comprises program instructions that when executed on the one or more processors:
implement a shared-memory based communication protocol for communications between at least a source program and at least a destination program in accordance with which the source program is granted at least a write permission to one or more target locations within the system memory, and the destination program is granted at least a read permission to the one or more target locations;
wherein the shared-memory based communication protocol allows the source program or the destination program to request, in a system call to an operating system, whether each of the one or more target locations are to be memory mapped in a volatile memory space or a non-volatile memory space;
map, by the operating system in response to the system call, the one or more target locations into the memory space requested in the system call, wherein the volatile memory space is mapped to at least one of the one or more volatile memory modules, and the non-volatile memory space is mapped to at least one of the one or more non-volatile memory modules; and
write, by the source program, a data item to a particular target location, and the particular target location is mapped to a non-volatile memory module of the one or more non-volatile memory modules;
read, by the destination program from the particular target location, subsequent to a failure event that results in a loss of data stored in the one or more volatile modules of the system memory, the data item written by the source program at the particular target location prior to the failure event; and
perform, by the destination program, one or more operations based at least in part on contents of the data item.

2. The system as recited in claim 1 , wherein the one or more non-volatile memory modules of the system memory comprise a non-volatile dual inline memory module (NV-DIMM).

3. The system as recited in claim 1 , wherein the source program is executed at a different computing device than the destination program, and wherein the source program transmits the data item to the particular target location via a network link.

4. The system as recited in claim 1 , wherein the communication protocol comprises an RDMA (remote direct memory access) protocol.

5. The system as recited in claim 1 , wherein the communication protocol comprises a store-and-forward messaging protocol, wherein the destination program is configured to transmit at least a portion of the data item to a different destination program.

6. A method, comprising:
performing, by one or more computing devices comprising one or more respective persistent storage devices and one or more respective processors linked to a respective system memory comprising one or more non-volatile memory modules and one or more volatile memory modules, the system memory having a lower access latency than the respective persistent storage device:
implementing a shared-memory based communication protocol for communications between at least a sender module and at least a receiver module in accordance with which the sender module is granted at least a write permission to one or more target locations within the system memory, and the receiver module is granted at least a read permission to the one or more target locations;
wherein the shared-memory based communication protocol allows the sender module or the receiver module to request, in a system call to an operating system, whether each of the one or more target locations are to be memory mapped in a volatile memory space or a non-volatile memory space;
mapping, by the operating system in response to the system call, the one or more target locations into the memory space requested in the system call, wherein the volatile memory space is mapped to at least one of the one or more volatile memory modules and the non-volatile memory space is mapped to at least one of the one or more non-volatile memory modules;
writing, by the sender module, a data item to a particular target location, wherein the particular target location is mapped to a non-volatile memory module of the one or more non-volatile memory modules;
reading, by the receiver module from the target location in accordance with the communication protocol, subsequent to a failure event that results in a loss of data stored in a volatile portion of the system memory, the data item written by the sender module at the particular location in accordance with the communication protocol prior to the failure event; and
performing, by the receiver module, one or more operations based at least in part on contents of the data item.

7. The method as recited in claim 6 , wherein the non-volatile portion of the system memory comprises resistive memory.

8. The method as recited in claim 6 , wherein the non-volatile portion of the system memory comprises phase-change memory.

9. The method as recited in claim 6 , wherein the system memory is part of a particular computing device of the one or more computing devices, and wherein the sender module is executed at a different computing device than the particular computing device, the method further comprising performing, by the one or more computing devices:
receiving the data item at the particular computing device over a network link from the different computing device.

10. The method as recited in claim 6 , wherein the communication protocol comprises an RDMA (remote direct memory access) protocol.

11. The method as recited in claim 6 , wherein the communication protocol comprises a store-and-forward messaging protocol, wherein performing said one or more operations based at least in part on contents of the data item comprises transmitting the data item to a different receiver module.

12. The method as recited in claim 6 , wherein the receiver module comprises an application running in user mode, further comprising performing, at the one or more computing devices:
generating, in response to a system call, a memory map of a subset of the non-volatile portion of the system memory within an address space of the application.

13. The method as recited in claim 6 , wherein the receiver module comprises a component of one of: (a) a Domain Name System (DNS) service, or (b) a directory service.

14. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
providing a write acknowledgement to the sender module;
wherein said performing the one or more operations is asynchronous with respect to said providing the write acknowledgment.

15. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
setting, by the receiver module, a read-complete indicator in the non-volatile portion of the system memory to indicate to the sender module that at least a portion of the target location is available for writing a different data item.

16. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:
disabling, subsequent to the failure event, write access to at least a sub-portion of the non-volatile portion of the system memory until the data item has been read by the receiver module.

Assignees

• Amazon Tech Inc

Inventors

• Mckelvie Samuel James
• Gupta Anurag Windlass

Classifications

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F3/0607

  by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device · CPC title

• G06F3/0655

  Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices · CPC title

• G06F12/0238 Primary

  Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory · CPC title

• G06F2212/152

  Virtualized environment, e.g. logically partitioned system · CPC title