High throughput, high reliability data processing system

Patent metadata

Field	Value
Publication number	US-10025680-B2
Application number	US-201615019003-A
Country	US
Kind code	B2
Filing date	Feb 9, 2016
Priority date	Dec 16, 2015
Publication date	Jul 17, 2018
Grant date	Jul 17, 2018

Abstract

A data processing system configured to store a plurality of data entities in volatile memories of multiple different computing devices. The data processing system comprises a first computing device having a first volatile memory configured to store a first data entity; and a second computing device having a second volatile memory configured to store a copy of the first data entity. The first computing device is configured to perform: receiving an indication to update the first data entity; after receiving the indication, updating the first data entity in the first volatile memory, and providing to the second computing device an indication to update the copy of the first data entity; and providing an indication that the first data entity has been updated, after receiving information from the second computing device indicating that the copy of the first data entity has been updated in the second volatile memory.

First claim

What is claimed is:

1. A data processing system configured to store a plurality of data entities in volatile memories of multiple computing devices, the data processing system comprising:
a first computing device having a first volatile memory configured to store a first data entity, the first data entity associated with a first identifier; and
a second computing device, different from and communicatively coupled to the first computing device, having a second volatile memory configured to store a copy of the first data entity associated with a second identifier,
wherein the first computing device is configured to perform:
receiving an indication to update the first data entity;
after receiving the indication,
updating the first data entity in the first volatile memory, and
providing to the second computing device an indication to update the copy of the first data entity; and

providing an indication that the first data entity has been updated, after receiving information from the second computing device indicating that the copy of the first data entity has been updated in the second volatile memory,
wherein the first volatile memory comprises first on-heap memory, and wherein the first computing device is configured to store the first data entity in the first on-heap memory.

2. The system of claim 1 , wherein the first volatile memory further comprises first off-heap memory.

3. The system of claim 2 , wherein the second volatile memory comprises second on-heap memory and second off-heap memory, and wherein the second computing device is configured to store the copy of the first data entity in the second off-heap memory.

4. The system of claim 3 , wherein the first computing device is communicatively coupled to a third computing device having a third volatile memory, the third volatile memory comprising third off-heap memory configured to store another copy of the first data entity, wherein the first computing device is further configured to perform:
in response to the receiving, providing to the third computing device an indication to update the other copy of the first data entity; and
providing the indication that the first data entity has been updated only after receiving information from the third computing device indicating that the other copy of the first data entity has been updated in the third volatile memory.

5. The system of claim 1 , wherein the first computing device is communicatively coupled to multiple computing devices having respective volatile memories configured to store copies of the first data entity therein, the multiple computing devices including the second computing device, wherein the first computing device is further configured to perform:
in response to the receiving, providing an indication to the multiple computing devices to update their respective copies of the first data entity; and
providing the indication that the first data entity has been updated only after receiving information from each of the multiple computing devices indicating that its respective copy of the first data entity has been updated.

6. The system of claim 5 , further comprising a plurality of computing devices, the plurality of computing devices comprising the multiple computing devices configured to store copies of the first data entity therein, wherein the first computing device is further configured to perform:
identifying, based on the first data entity, the multiple computing devices from among the plurality of computing devices.

7. The system of claim 1 , wherein updating the first data entity in the first volatile memory is performed prior to providing to the second computing device an indication to update the copy of the first data entity.

8. The system of claim 1 , wherein the first volatile memory is configured to store a plurality of data entities including the first data entity, wherein the second computing device is configured to store copies of data entities in the plurality of data entities, and wherein the first computing device is further configured to perform:
receiving an indication to perform resynchronization;
in response to receiving the indication to perform resynchronization, providing to
the second computing device information comprising an identifier and a version number for each of the plurality of data entities;
receiving, from the second computing device, information identifying at least one data entity in the first volatile memory that is not synchronized with its respective at least one copy in the second volatile memory; and
in response to receiving the information identifying the at least one data entity, providing to the second computing device the at least one data entity stored in the first volatile memory.

9. The system of claim 8 , wherein the information identifying the at least one data entity consists of a first amount of information, and wherein the at least one data entity consists of a second amount of information that is at least at least ten thousand times greater than the first amount of information.

10. The system of claim 1 , wherein:
the data processing system is configured to manage a set of at least ten million data entities, each data entity comprising at least a kilobyte of data; and
the plurality of data entities comprise the set of at least ten million data entities.

11. The system of claim 1 , wherein the first identifier and the second identifier are the same identifier.

12. A method, performed by a first computing device having a first volatile memory configured to store a first data entity associated with a first identifier, the first computing device being communicatively coupled to a second computing device having a second volatile memory configured to store a copy of the first data entity associated with a second identifier, the second computing device being different from the first computing device, the method comprising:
receiving an indication to update the first data entity;
after receiving the indication,
updating the first data entity in the first volatile memory, and
providing to the second computing device an indication to update the copy of the first data entity; and

providing an indication that the first data entity has been updated, after receiving information from the second computing device indicating that the copy of the first data entity has been updated in the second volatile memory,
wherein the first volatile memory comprises first on-heap memory, and wherein the first computing device is configured to store the first data entity in the first on-heap memory.

13. The method of claim 12 , wherein the first computing device is communicatively coupled to multiple computing devices having respective volatile memories configured to store copies of the first data entity therein, the multiple computing devices including the second computing device, the method further comprising:
in response to the receiving, providing an indication to the multiple computing devices to update their respective copies of the first data entity; and
providing the indication that the first data entity has been updated only after receiving information from each of the multiple computing devices indicating that its respective copy of the first data entity has been updated.

14. The method of claim 13 , further comprising:
identifying, based on the first data entity, the multiple computing devices from among a plurality of computing devices.

15. The method of claim 12 , wherein updating the first data entity in the first volatile memory is performed prior to providing to the second computing device an indication to update the copy

Assignees

• Ab Initio Technology Llc

Inventors

• Thomas Ben

Classifications

• G06F2201/855

  Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F11/1666

  where the redundant component is memory or memory area · CPC title

• G06F12/023

  Free address space management · CPC title