Systems and methods for data replication synchronization

Patent metadata

Field	Value
Publication number	US-2024095259-A1
Application number	US-202318522007-A
Country	US
Kind code	A1
Filing date	Nov 28, 2023
Priority date	Dec 22, 2016
Publication date	Mar 21, 2024
Grant date	—

Abstract

Systems and methods are provided for storing a first data object comprising a first set of immutable components, the first data object being associated with a corresponding second data object stored by a remote replication system. A difference is determined between the first set of immutable components of the first data object and a second set of immutable components of the corresponding second data object. A subset of immutable components is identified from the first set of immutable components based on the difference. The subset of immutable components from the first set of immutable components is provided to the remote replication system over a communication network.

First claim

1 . A system comprising:
a first server communicatively coupled to different servers and to one or more clients; the different servers; the one or more clients; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the system to perform:
storing a first data object at the first server, the first data object being associated with a corresponding second data object stored by a second server of the one or more different servers and a corresponding third data object stored by a third server of the one or more different servers;
receiving an update to the second data object from the second server;
selectively scheduling one or more import actions from the second server to the first server and one or more export actions from the first server to the third server to synchronize, among the first server, the second server and the third server, the first data object, the second data object, and the third data object; and
hiding information regarding any first associated clients of the first server from the one or more different servers.

2 . The system of claim 1 , wherein information of any second associated clients associated with the second server and information of any third associated clients associated with the third server are invisible from a perspective of the first server.

3 . The system of claim 1 , wherein the selective scheduling comprises refraining from reordering a priority of an import action or an export action with respect to a different import action.

4 . The system of claim 1 , wherein the selective scheduling comprises refraining from reordering a first export action to occur ahead of a second export action.

5 . The system of claim 1 , wherein the selective scheduling is based on a predicted saturation state of a network connection resulting from an export action.

6 . The system of claim 1 , wherein the instructions further cause the system to:
hold, at the first server, separate version clock acknowledgements for each of different data objects stored within the first server, and for different updates from each of the different servers; compare a version vector associated with an update to the second data object or the third data object to a local version vector of the first server associated with an update to the first data object to determine any concurrent changes; upon detecting a difference between the version vector associated with the update, hold the update to the first data object from replication to the second server and to the third server; resolve a conflict associated with the concurrent changes; and in response to resolving the conflict, update the local version vector of the first server and transmit the updated first data object and the updated local version vector to the second server or the third server.

7 . The system of claim 1 , wherein a client is communicatively coupled to only one of the first server, the second server, or the third server.

8 . The system of claim 1 , wherein the instructions further cause the system to update the first data object at the first server in response to the update to the second data object and propagate the updated first data object to a first client without propagating the updated first data object to any other clients.

9 . The system of claim 1 , wherein the selectively scheduling comprises:
determining that a local update to the first data object takes precedence over the update received from the second server; updating the first data object based on the local update; propagating the updated first data object to the second server; and propagating the updated first data object to the third server without propagating the updated second data object.

10 . The system of claim 1 , wherein the selectively scheduling is based on a predicted transfer time of propagating the second update to the first server and the third server, the selectively scheduling comprising refraining from scheduling in response to the predicted transfer time being longer than a threshold time.

11 . A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:
storing, at a first server, a first data object, the first data object being associated with a corresponding second data object stored by a second server of the one or more different servers and a corresponding third data object stored by a third server of the one or more different servers; receiving an update to the second data object from the second server; selectively scheduling one or more import actions from the second server to the first server and one or more export actions from the first server to the third server to synchronize, among the first server, the second server and the third server, the first data object, the second data object, and the third data object; and hiding information regarding any first associated clients of the first server from the one or more different servers.

12 . The method of claim 11 , wherein information of any second associated clients associated with the second server and information of any third associated clients associated with the third server is invisible from a perspective of the first server.

13 . The method of claim 11 , wherein the selective scheduling comprises refraining from reordering a priority of an import action or an export action with respect to a different import action.

14 . The method of claim 11 , wherein the selective scheduling comprises refraining from reordering a first export action to occur ahead of a second export action.

15 . The method of claim 11 , wherein the selective scheduling is based on a predicted saturation state of a network connection resulting from an export action.

16 . The method of claim 11 , further comprising:
holding, at the first server, separate version clock acknowledgements for each of different data objects stored within the first server, and for different updates from each of the different servers; comparing a version vector associated with an update to the second data object or the third data object to a local version vector of the first server associated with an update to the first data object to determine any concurrent changes; upon detecting a difference between the version vector associated with the update, holding the update to the first data object from replication to the second server and to the third server; resolving a conflict associated with the concurrent changes; and in response to resolving the conflict, updating the local version vector of the first server and transmit the updated first data object and the updated local version vector to the second server or the third server.

17 . The method of claim 11 , wherein a client is communicatively coupled to only one of the first server, the second server, or the third server.

18 . The method of claim 11 , further comprising:
updating the first data object at the first server in response to the update to the second data object and propagating the updated first data object to a first client without propagating the updated first data object to any other clients.

19 . The method of claim 11 , wherein the selectively scheduling comprises:
determining that a local update to the first data object takes precedence over the update received from the second server; updating the first data object based on the local update; propagating the updated first data object to the second s

Assignees

• Palantir Technologies Inc

Inventors

• Freiberg Stephen
• Landau Alexander
• Greene Andrew
• Dorne Brian
• Offutt Bryan
• Zeidman Ernest
• Nepomnyaschchiy Ilya
• Garrod John
• Brainard Katherine
• Purcell Kolin
• Levin Michael
• Swanson Simon
• Stecko Spencer

Classifications

• G06F16/273 Primary

  Asynchronous replication or reconciliation · CPC title

• G06F7/02

  Comparing digital values (G06F7/06, {G06F7/22,} G06F7/38 take precedence) · CPC title

• G06F16/178 Primary

  Techniques for file synchronisation in file systems · CPC title