System and method of optimization of in-memory data grid placement

Patent metadata

Field	Value
Publication number	US-9405589-B2
Application number	US-201113154294-A
Country	US
Kind code	B2
Filing date	Jun 6, 2011
Priority date	Jun 6, 2011
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

A method, system, and program product for optimizing distribution and availability of data partitions is disclosed. Placement logic is run for data partition distribution which optimizes a configuration of a memory data grid. A performance impact of adding new members to the memory data grid is provided. Replication of data partitions in the memory data grid is minimized. Optimum data partition distribution in the memory data grid is determined.

First claim

What is claimed is:

1. A method for optimizing data partition distribution in a memory data grid which includes a plurality of data partitions and a plurality of containers which contain the data partitions, comprising:
running placement logic for optimizing configuration of the memory data grid;
providing performance impact of adding new members to the memory data grid for adding available space to the memory data grid;
determining a number of containers for the plurality of partitions based on network latency;
for a primary partition found in a container to be removed, converting a replica of the found primary partition to the found primary partition in a container other than the container to be removed, wherein a total number of containers is based on the determined optimum number of containers;
creating a replica partition of the converted replica partition in a different one of the containers from the converted replica partition;
for a replica partition found in the container to be removed, creating the found replica partition in one of the containers other than the container to be removed, wherein the created replica partition is not in a container that contains a primary partition corresponding to the created replica partition; and
removing the container to be removed;
determining an optimal data partition distribution in the memory data grid based on network topology;
creating policies for deciding whether synchronization of the optimal data partition distribution should be synchronous or asynchronous;
reducing invalidation traffic by placing the plurality of data partitions with the plurality of grid containers in the memory data grid; and
dynamically changing a data partition synchronization to an optimal data partition synchronization based on one of the policies.

2. The method of claim 1 , wherein the method includes minimizing internal communication among members of the memory data grid.

3. The method of claim 1 , wherein the optimum partition distribution in the memory data grid is determined by observing network speed and latency, frequency of membership changes detected by the memory data grid, and frequency of data partition access.

4. The method of claim 3 , wherein the optimum data partition distribution in the memory data grid is determined by observing existing desired quality of service goals, and internal communication between members of the memory data grid.

5. The method of claim 1 , wherein optimizing data partition distribution in the memory data grid includes analyzing data partition usage patterns and co-locating data partition data most frequently accessed.

6. The method of claim 1 , wherein optimizing data partition distribution in the memory data grid includes placing replica data partitions in a plurality of different data centers.

7. The method of claim 1 , wherein optimizing data partition distribution in the memory data grid includes data partition placement that reduces group service communications between members of the memory data grid.

8. A computer program product for optimizing data partition distribution in a memory data grid which includes a plurality of data partitions and a plurality of containers which contain the data partitions, the computer program product comprising a
non-transitory computer readable storage medium having computer readable code embodied therewith, the computer readable program code being operable to:
run placement logic for optimizing configuration of the memory data grid;
provide performance impact of adding new members to the memory data grid for adding available space to the memory data grid;
determine a number of containers for the plurality of data partitions based on network latency;
for a primary partition found in a container to be removed, convert a replica of the found primary partition to the found primary partition in a container other than the container to be removed, wherein a total number of containers is based on the determined optimum number of containers;
create a replica partition of the converted replica partition in a different one of the containers from the converted replica partition;
for a replica partition found in the container to be removed, create the found replica partition in one of the containers other than the container to be removed, wherein the created replica partition is not in a container that contains a primary partition corresponding to the created replica partition;
remove the container to be removed;
determine an optimal data partition distribution in the memory data grid based on network topology;
create policies for deciding whether synchronization of the optimal data partition distribution should be synchronous or asynchronous;
reduce invalidation traffic by placing the plurality of data partitions with the plurality of grid containers in the memory data grid; and
dynamically change a data partition synchronization to an optimal data partition synchronization based on one of the policies.

9. The computer program product of claim 8 , including computer readable program code that minimizes internal communication among members of the memory data grid.

10. The computer program product of claim 8 , wherein the optimal data partition distribution in the memory data grid is determined by observing network speed and latency, frequency of membership changes detected by the memory data grid, and frequency of data partition access.

11. The computer program product of claim 10 , wherein the optimal data partition distribution in the memory data grid is determined by observing existing desired quality of service goals, and internal communication between members of the memory data grid.

12. The computer program product of claim 8 , wherein the optimizing data partition distribution in the memory data grid includes moving one of the plurality of data partitions from one container to another container in the memory data grid.

13. The computer program product of claim 8 , wherein the optimizing data partition distribution in the memory data grid includes analyzing data partition usage patterns and co-locating data partition data most frequently accessed.

14. The computer program product of claim 8 , wherein optimizing data partition distribution in the memory data grid includes placing replica data partitions in a plurality of different data centers.

15. The computer program product of claim 8 , wherein optimizing data partition distribution in the memory data grid includes data partition placement that reduces group service communications between members of the memory data grid.

16. A system comprising:
a computer including a computer processor for optimizing data partition distribution in a memory data grid which includes a plurality of data partitions and a plurality of containers which contain the data partitions; and
a data partition optimization engine for synchronizing the plurality of data partitions, the data partition optimization engine being operable to:
run placement logic for optimizing configuration of the memory data grid;
provide performance impact of adding new members to the memory data grid for adding available space to the memory data grid;
determine a number of containers for the plurality of data partitions based on network latency;
for a primary partition found in a container to be removed, convert a replica of the found primary partition to the found primary partition in a container other than the container to be removed, wherein a total number of containers is based on the determined optimum number of containers;
create a replica partition of the co

Assignees

• Gaur Nitin
• Bhogal Kulvir S
• Johnson Christopher D
• Kaplinger Todd E
• Berg Douglas C
• IBM

Inventors

• Gaur Nitin
• Bhogal Kulvir S
• Johnson Christopher D
• Kaplinger Todd E
• Berg Douglas C

Classifications

• G06F3/0674

  Disk device · CPC title

• G06F9/5072 Primary

  Grid computing · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F11/1425

  by reconfiguration of node membership · CPC title

• G06F3/061

  Improving I/O performance · CPC title