Memory sharing for working data using rdma

Patent metadata

Field	Value
Publication number	US-2017277655-A1
Application number	US-201615081602-A
Country	US
Kind code	A1
Filing date	Mar 25, 2016
Priority date	Mar 25, 2016
Publication date	Sep 28, 2017
Grant date	—

Abstract

A server system may include a cluster of multiple computers that are networked for high-speed data communications. Each of the computers has a remote direct memory access (RDMA) network interface to allow high-speed memory sharing between computers. A relational database engine of each computer is configured to utilize a hierarchy of memory for temporary storage of working data, including in order of decreasing access speed (a) local main memory, (b) remote memory accessed via RDMS, and (c) mass storage. The database engine uses the local main memory for working data, and additionally uses the RDMA accessible memory for working data when the local main memory becomes depleted. The server system may include a memory broker to which individual computers report their available or unused memory, and which leases shared memory to requesting computers.

First claim

What is claimed is:

1 . A method performed by a relational database engine of a first computer, the method comprising:
detecting a depletion of memory of the first computer; communicating with a memory broker to receive an allocation of remote memory, wherein the remote memory comprises memory of a second computer that is accessible to the relational database engine using a remote direct memory access (RDMA) network interface of the first computer; accessing the remote memory through the RDMA network interface; and temporarily storing data in the remote memory for use by the relational database engine.

2 . The method of claim 1 , wherein detecting the depletion of memory comprises detecting depletion of randomly-accessible memory (RAM) of the first computer.

3 . The method of claim 1 , further comprising:
maintaining a primary cache in the memory of the first computer, the primary cache having multiple entries; maintaining a secondary cache in the remote memory; evicting a first entry from the primary cache; writing the first entry to the secondary cache; detecting a primary cache miss for requested data; and in response to detecting the primary cache miss, checking the secondary cache for the requested data.

4 . The method of claim 1 , the data comprising one or more of a table, a table variable, a cursor, a cache, a buffer pool, an index, a materialized view, or an intermediate result.

5 . The method of claim 1 , the data comprising a semantic cache.

6 . The method of claim 1 , further comprising, in response to being designated a primary replicated server, priming a buffer pool of the relational database engine by copying at least a portion of the buffer pool from remotely accessible memory through the RDMA network interface.

7 . A system, comprising:
multiple servers; each of the servers having a respective remote direct memory access (RDMA) network interface; each of the servers having randomly-accessible memory (RAM) that is remotely accessible by others of the database servers through RDMA; a memory broker that performs actions comprising:
communicating with a first of the servers to identify a region of RAM of the first server that is unused;
receiving a remote memory request from a second of the servers;
in response to receiving the remote memory request, allocating the region of RAM of the first server for use by the database server;
a relational database management system (RDBMS) that executes on the second server, the RDBMS utilizing at least three types of memory for temporary storage of data, the three types of memory comprising: (a) RAM of the second database server; (b) the allocated region of RAM of the first server; and (c) non-volatile mass storage; and the RDBMS being configured to access the allocated region of RAM of the first server using RDMA through the RDMA interface of the first server and the RDMA interface of the second database server.

8 . The system of claim 7 , the RDBMS being further configured to perform actions comprising:
detecting depletion of the RAM of the second database server; and sending the remote memory request to the memory broker in response to detecting the depletion of the RAM of the second database server.

9 . The system of claim 7 , the RDBMS being further configured to perform actions comprising:
maintaining a primary cache in the RAM of the second database server, the primary cache having multiple entries; maintaining a secondary cache in the allocated region of RAM of the first database server; evicting a first entry from the primary cache; writing the first entry to the secondary cache; detecting a primary cache miss for requested data; and in response to detecting the primary cache miss, checking the secondary cache for the requested data.

10 . The system of claim 7 , wherein the data comprises one or more of a table, a table variable, a cursor, or an intermediate result.

11 . The system of claim 7 , the RDBMS being further configured to maintain a semantic cache in the allocated region of RAM of the first server.

12 . The system of claim 7 , the RDBMS being further configured to maintain a semantic cache in the allocated region of RAM of the first server, wherein the semantic cache contains one or more of a non-clustered index, a partial index, or a materialized view.

13 . The system of claim 7 , wherein the RDBMS is further configured to perform an action in response to being designated as a primary replicated server, the action comprising priming a buffer pool by copying at least a portion of the buffer pool from RAM of a secondary replicated server using RDMA.

14 . One or more computer-readable media storing computer-executable instructions that, when executed by one or more processors of a first computer, cause the one or more processors to perform actions comprising:
managing a database that resides on one or more mass storage devices; receiving and responding to database queries; temporarily storing first data related to the database queries in local memory of the first computer; detecting depletion of the local memory of the first computer; in response to detecting the depletion of the local memory of the first computer, requesting an allocation of memory of a second computer; receiving the allocation of the memory of the second computer; and temporarily storing second data related to the database queries in the memory of the second computer.

15 . The one or more computer-readable media of claim 14 , the actions further comprising accessing the memory of the second computer using remote direct memory access (RDMA) through an RDMA network interface of the first computer.

16 . The one or more computer-readable media of claim 14 , the actions further comprising communicating with a memory broker to request the allocation of memory of the second computer.

17 . The one or more computer-readable media of claim 14 , the actions further comprising reporting unused memory of the first computer to a memory broker for allocation to other computers.

18 . The one or more computer-readable media of claim 14 , the actions further comprising:
maintaining a primary cache in the local memory of the first computer, the primary cache having multiple entries; maintaining a secondary cache in the memory of the second computer; evicting a first entry from the primary cache; writing the first entry to the secondary cache; detecting a primary cache miss for requested data; and in response to detecting the primary cache miss, checking the secondary cache for the requested data.

19 . The one or more computer-readable media of claim 14 , the second data comprising one or more of a table, a table variable, a cursor, a cache, a buffer pool, an index, a materialized view, or an intermediate result.

20 . The one or more computer-readable media of claim 14 , the second data comprising a semantic cache.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Das Sudipto
• Li Feng
• Syamala Manoj A
• Narasayya Vivek R

Classifications

• G06F9/5016 Primary

  the resource being the memory · CPC title

• G06F12/023

  Free address space management · CPC title

• G06F2212/163

  Server or database system · CPC title

• G06F2212/264

  Remote server · CPC title

• G06F15/17331 Primary

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title