Sharding of in-memory objects across NUMA nodes

Patent metadata

Field	Value
Publication number	US-9684682-B2
Application number	US-201514922077-A
Country	US
Kind code	B2
Filing date	Oct 23, 2015
Priority date	Sep 21, 2013
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

Techniques are provided for sharding objects across different compute nodes. In one embodiment, a database server instance generates, for an object, a plurality of in-memory chunks including a first in-memory chunk and a second in-memory chunk, where each in-memory chunk includes a different portion of the object. The database server instance assigns each in-memory chunk to one of a plurality of computer nodes including the first in-memory chunk to a first compute node and a second in-memory chunk to a second local memory of a second compute node. The database server instance stores an in-memory map that indicates a memory location for each in-memory chunk. The in-memory map indicates that the first in-memory chunk is located in the first local memory of the first compute node and that the second in-memory chunk is located in the second local memory of the second compute node.

First claim

What is claimed is:

1. A method comprising:
storing, by a database server instance that is executing on a particular machine, a plurality of in-memory chunks that contain data from an object;
wherein the plurality of in-memory chunks include a first in-memory chunk and a second in-memory chunk;
wherein the first in-memory chunk includes a first portion of the object;
wherein the second in-memory chunk includes a second portion of the object;
wherein the first portion of the object is different than the second portion of the object;
wherein each in-memory chunk of the plurality of in-memory chunks is stored in a corresponding memory area within volatile memory of the particular machine;
wherein the particular machine includes a plurality of compute nodes;
wherein the plurality of computer nodes includes a first compute node and a second computer node;
wherein each compute node of the plurality of compute nodes has direct access to all memory areas within the volatile memory of the particular machine;
wherein each compute node, of the plurality of compute nodes, has a local memory area, within the volatile memory of the particular machine, that the compute node is able to access more efficiently than other areas of the volatile memory of the particular machine;
wherein storing the plurality of in-memory chunks includes:
assigning, by the database server instance, each in-memory chunk of the plurality of in-memory chunks to one or more compute nodes of the plurality of compute nodes; and
causing each in-memory chunk of the plurality of in-memory chunks to be loaded into one or more local memory areas of the one or more compute nodes to which the in-memory chunk is assigned;

wherein the step of assigning each in-memory chunk includes:
assigning the first in-memory chunk to the first compute node; and
assigning the second in-memory chunk to the second compute node;

storing, by the database server instance, an in-memory map that indicates the one or more compute nodes to which each of the plurality of chunks is assigned.

2. The method of claim 1 ,
wherein the object corresponds to an on-disk object;
wherein the first in-memory chunk stores data from a first on-disk chunk of the on-disk object; and
wherein the second in-memory chunk stores data from a second on-disk chunk of the on-disk object.

3. The method of claim 2 , wherein the step of assigning each memory chunk comprises:
selecting, to store the first in-memory chunk in volatile memory, the first compute node from a plurality of compute nodes by applying a hash function to a first page address of the first on-disk chunk;
selecting, to store the second in-memory chunk in volatile memory, the second compute node from a plurality of compute nodes by applying the hash function to a second page address of the second on-disk chunk.

4. The method of claim 2 , wherein the in-memory map indicates that the first in-memory chunk corresponds to the first on-disk chunk and that the second in-memory chunk corresponds to the second on-disk chunk.

5. The method of claim 2 , wherein the first in-memory chunk stores data from the first on-disk chunk in a mirror format and the second in-memory chunk stores data from the second on-disk chunk in the mirror format.

6. The method of claim 1 , wherein the step of assigning each memory chunk comprises selecting the first compute node from a plurality of compute nodes and the second compute node from the plurality of compute nodes using a round-robin distribution wherein the plurality of in-memory chunks are evenly distributed across the plurality of compute nodes.

7. The method of claim 1 , further comprising:
allocating a plurality of stripes from volatile memory;
wherein the step of allocating the plurality of stripes from volatile memory includes:
allocating, from a first memory area that is local to the first compute node, a first set of stripes;
allocating, from a second memory area that is local to the second compute node, a second set of stripes;

wherein each stripe in the first set of stripes includes a contiguous chunk of memory of the first memory area;
wherein each stripe in the second set of stripes includes a contiguous chunk of memory of the second memory area.

8. The method of claim 7 , wherein the plurality of stripes are allocated across a plurality of compute nodes in a round-robin distribution such that each compute node of the plurality of compute nodes has approximately the same number of stripes.

9. The method of claim 7 , further comprising:
sending, by the database server instance to the first compute node, a first request to allocate space for the first in-memory chunk;
sending, by the database server instance to the second compute node, a second request to allocate space for the second in-memory chunk;
wherein the first compute node satisfies the first request using at least one stripe from the first set of stripes;
wherein the second compute node satisfies the second request using at least one stripe from the second set of stripes.

10. The method of claim 1 wherein the first compute node and the second compute node are non-uniform memory access (NUMA) nodes; wherein a first memory controller that controls access to a first memory area of the first compute node is connected to a second memory controller that controls access to a second memory area of the second compute node.

11. One or more non-transitory computer-readable media storing instructions, wherein the instructions include:
instructions which, when executed by one or more hardware processors, cause storing, by a database server instance that is executing on a particular machine, a plurality of in-memory chunks that contain data from an object;
wherein the plurality of in-memory chunks include a first in-memory chunk and a second in-memory chunk;
wherein the first in-memory chunk includes a first portion of the object;
wherein the second in-memory chunk includes a second portion of the object;
wherein the first portion of the object is different than the second portion of the object;
wherein each in-memory chunk of the plurality of in-memory chunks is stored in a corresponding memory area within volatile memory of the particular machine;
wherein the particular machine includes a plurality of compute nodes;
wherein the plurality of computer nodes includes a first compute node and a second computer node;
wherein each compute node of the plurality of compute nodes has direct access to all memory areas within the volatile memory of the particular machine;
wherein each compute node, of the plurality of compute nodes, has a local memory area, within the volatile memory of the particular machine, that the compute node is able to access more efficiently than other areas of the volatile memory of the particular machine;
wherein instruction for storing the plurality of in-memory chunks include:
instructions which, when executed by one or more hardware processors, cause assigning, by the database server instance, each in-memory chunk of the plurality of in-memory chunks to one or more compute nodes of the plurality of compute nodes; and
instructions which, when executed by one or more hardware processors, cause loading each in-memory chunk of the plurality of in-memory chunks into one or more local memory areas of the one or more compute nodes to which the in-memory chunk is assigned;

wherein the step of assigning each in-memory chunk includes:
assigning the first in-memory chunk to the first compute node; and
assigning the second in-memory chunk to the second compute node;

storing, by the database server instance, an in-memory map that indicates the one or more compute node

Assignees

• Oracle Int Corp

Inventors

• Mukherjee Niloy
• Ganesh Amit
• Marwah Vineet

Classifications

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F2212/2542

  Non-uniform memory access [NUMA] architecture · CPC title

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F2212/283

  Plural cache memories · CPC title