Local direct storage class memory access

Patent metadata

Field	Value
Publication number	US-9311230-B2
Application number	US-201313975643-A
Country	US
Kind code	B2
Filing date	Aug 26, 2013
Priority date	Apr 23, 2013
Publication date	Apr 12, 2016
Grant date	Apr 12, 2016

Abstract

A queued, byte addressed system and method for accessing flash memory and other non-volatile storage class memory, and potentially other types of non-volatile memory (NVM) storage systems. In a host device, e.g., a standalone or networked computer, having attached NVM device storage integrated into a switching fabric wherein the NVM device appears as an industry standard OFED™ RDMA verbs provider. The verbs provider enables communicating with a ‘local storage peer’ using the existing OpenFabrics RDMA host functionality. User applications issue RDMA Read/Write directives to the ‘local peer (seen as a persistent storage) in NVM enabling NVM memory access at byte granularity. The queued, byte addressed system and method provides for Zero copy NVM access. The methods enables operations that establish application private Queue Pairs to provide asynchronous NVM memory access operations at byte level granularity.

First claim

What is claimed is:

1. A method for accessing a non volatile memory system comprising:
embedding, in a computing system, a virtual representation of a remotely networked peer device configured to communicate in accordance with a remote direct memory access (RDMA) infrastructure for data storage and transfer among multiple networked devices, said RDMA infrastructure for data storage and transfer among multiple networked devices operating according to a switched fabric technology infrastructure for a local non-volatile memory (NVM) using RDMA directives at byte length granularity;
establishing, via said embedded virtual representation of said remotely networked peer device, a RDMA infrastructure interface between an application run by a host processing unit and the local NVM at the computing system, said interface comprising an NVM device driver as a NVM verbs provider element which accesses the local NVM via a set of command queues, wherein NVP verbs are loaded directly in both an operating system O/S and NVM device driver components to enable direct access to the NVM device;
registering said local NVM with said remote direct memory access (RDMA) infrastructure for read and write local NVM access, and processing received RDMA infrastructure Read directives to read data via said interface from said local non volatile memory at said byte length granularity, and
processing received RDMA infrastructure Write directives to write data via said interface to said local non volatile memory at said byte length granularity, and
said processing of said Read directives and Write directives comprising translating said directives into a format for use by said NVM verbs provider to transfer data directly between a registered buffer in the local NVM to another registered buffer in a virtual memory associated with the host processing unit, said Read directives and Write directives including an address in said local NVM where to start reading/writing, and an address of a user memory buffer where the data bytes are to be transferred from/to, and the length (in byte(s)) of the data being transferred,
wherein a programmed processor unit is configured to initiate said embedding, said establishing, said registering, and said Read and Write directives processing.

2. The method of claim 1 , wherein said local NVM comprises a flash non-volatile memory device as being one of: attached to or integrated in said computing system.

3. The method of claim 2 , wherein said translating said directives into a format for use by said NVM verb provider to transfer data directly between a registered buffer in the local NVM to another registered buffer in a virtual memory comprises:
checking whether a requested access location is within the registered local NVM.

4. The method of claim 3 , wherein said translating said directives into a format for use by said NVM verb provider to transfer data directly between a registered buffer in the local NVM to another registered buffer in a virtual memory comprises:
determining existence of any permission level requirements for the requested access location; and
checking access permissions of both the registered buffer in the local NVM and the registered buffer in the virtual memory associated with the host processing unit.

5. The method of claim 2 , wherein said translating said directives into a format for use by said NVM verb provider to transfer data directly between a registered buffer in the local NVM to another registered buffer in the virtual memory comprises:
determining whether to split a Read request or Write request into smaller subrequests in accordance with a machine or hardware capability.

6. The method of claim 5 , wherein a machine or hardware capability comprises a transfer limit or memory alignment requirement when using a transport path to said local NVM.

7. The method of claim 2 , wherein said translating said directives into a format for use by said NVM verb provider to transfer data directly between a registered buffer in the local NVM to another registered buffer in the virtual memory comprises:
performing a data alignment check to determine if a page of memory in said NVM is crossed; and responsively,
splitting the request into smaller requests to respect page boundaries.

8. The method of claim 1 , wherein said switched fabric technology infrastructure is an OpenFabrics Enterprise Distribution (OFED) standard.

9. The method of claim 1 , wherein said registering comprises:
registering a target local NVM memory area of said NVM with said RDMA infrastructure for data storage and transfer among multiple networked devices, said Read directives enabling reading of data from said target local non volatile memory area via said interface; and
registering a source memory storage buffer with said RDMA infrastructure for data storage and transfer among multiple networked devices, said Write directives enabling writing of data content of said source memory storage buffer into said target local non volatile memory area.

10. The method of claim 9 , further comprising:
extending an operating system (O/S) with kernel level components in said O/S to embed said virtual representation of a remotely networked peer device, and establish said RDMA infrastructure interface,
providing a driver module loaded into the O/S for communicating with an attached local non volatile memory hardware component with the RDMA infrastructure; and
activating RDMA infrastructure-compliant application level library components which are loaded with an application to enable accessing said local non volatile memory via said RDMA infrastructure interface.

11. The method of claim 10 , further comprising:
providing, in a user client space, user level library components for translating said RDMA infrastructure Read and Write directives as issued by the application from and to referenced non volatile memory into Read and Write work requests placed into a send work request queue shared with the driver module.

12. The method of claim 11 , wherein said driver module performs:
translating said Read and Write work requests derived from a send queue shared with an application library into commands issued to the local non volatile hardware component,
wherein said Read directives are translated by said driver module to program the local non volatile hardware component to directly transfer non volatile hardware buffer content referenced by said Read directive into an application buffer also referenced by said Read directive without further operating system involvement, and
wherein said Write directives are translated by said driver module into programming the non volatile hardware component to directly transfer the content of application buffers referenced by said Write directive into an non volatile hardware buffer also referenced by said Write directive without further operating system involvement.

13. The method of claim 12 , wherein the programmed processor unit is further programmed to perform:
signaling a completion of Read directive and Write directive issued by an application to read or write to said local non volatile memory using said RDMA infrastructure, whereby a work completion queue is shared between said driver module and said user level library components, said method further comprising:
placing, by said driver module, work completions into said completion queue, and
obtaining, by said user level library components, work completions from said completion queue.

14. The method of claim 13 , wherein said RDMA infrastructure provides a control interface between the application and the driver module, said method comprising:
issuing comm

Assignees

• Globalfoundries Inc

Inventors

• Fitch Blake G
• Franceschini Michele M
• Schneidenbach Lars
• Metzler Bernaard

Classifications

• G06F12/109 Primary

  for multiple virtual address spaces, e.g. segmentation (G06F12/1036 takes precedence) · CPC title

• G06F12/1036

  for multiple virtual address spaces, e.g. segmentation (G06F12/1045 takes precedence) · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F15/17331

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title

• G06F12/0246 Primary

  in block erasable memory, e.g. flash memory · CPC title