Coherent distributed logging

Patent metadata

Field	Value
Publication number	US-10158642-B2
Application number	US-201615148516-A
Country	US
Kind code	B2
Filing date	May 6, 2016
Priority date	May 6, 2016
Publication date	Dec 18, 2018
Grant date	Dec 18, 2018

Abstract

Log entries are provided with unique entry identifiers, which may be sequenced in an incremental or decremental order, to create or to update a distributed log which may be replicated and distributed to multiple servers in a network. The entry identifiers may be appended to their respective log entries. Files, records or data which are identified by their respective log entries may be transmitted regardless of the sequence of the entry identifiers.

First claim

The invention claimed is:

1. A method of generating coherent log entries in a distributed log in a network comprising a plurality of servers and a plurality of client devices, wherein the client devices access a client library, the method comprising:
allocating, by the client library, a first entry identifier corresponding to a first log entry of a plurality of log entries created by a first client process of a plurality of client processes, wherein the first client process runs on a first client device of the plurality of client devices;
allocating, by the client library, a second entry identifier corresponding to a second log entry of the plurality of log entries created by a second client process of the plurality of client processes, wherein the second client process runs on a second client device of the plurality client devices concurrently with running of the first client process on the first client device, and wherein the second entry identifier is allocated subsequent to the first entry identifier being allocated;
authorizing, by the client library, transmission of the first log entry by the first client device and the second log entry by the second client device to one or more of the plurality of servers in the network, wherein the authorizing comprises authorizing transmission of the second log entry prior to transmission of the first log entry;
receiving, by the client library, acknowledgement of the first log entry and acknowledgement of the second log entry from one or more of the plurality of servers that replicate the first log entry and the second log entry in a journal that includes a background thread checkpoint, maintain a copy of the journal in cache memory, update the journal by flushing at least the first log entry and the second log entry up until a predetermined checkpoint, and delete one or more other journals older than the predetermined checkpoint; and
transmitting, by the client library to the client devices, the acknowledgement of the second log entry subsequent to the acknowledgement of the first log entry.

2. The method of claim 1 , wherein the first client device is the client device that includes the client library.

3. The method of claim 1 , wherein the second client device is the client device that includes the client library.

4. The method of claim 1 , wherein the first client device and the second client device are separate from the client device that includes the client library.

5. The method of claim 1 , further comprising batching a plurality of log entries including the first log entry and the second log entry by one of the client devices.

6. The method of claim 1 , further comprising:
allocating, by the client library, a third entry identifier corresponding to a third log entry of the plurality of log entries created by a third client process of the plurality of client processes, wherein the third client process runs on a third client device of the plurality client devices concurrently with running of the first client process on the first client device and running of the second client process on the second client device, and wherein the third entry identifier is allocated subsequent to the second entry identifier being allocated;
authorizing, by the client library, transmission of the third log entry by the third client device to one or more of the plurality of servers in the network prior to transmission of the first log entry;
receiving, by the client library, acknowledgement of the third log from one or more of the plurality of servers; and
transmitting, by the client library to the client devices, the acknowledgement of the third log entry subsequent to the acknowledgement of the second log entry.

7. The method of claim 6 , wherein the authorizing, by the client library, transmission of the third log entry comprises authorizing transmission of the third log entry prior to transmission of the second log entry.

8. The method of claim 6 , further comprising batching a plurality of log entries including the second log entry and the third log entry by the second client device and transmitting the second log entry and the third log entry prior to transmission of the first log entry to one or more of the plurality of servers in the network.

9. The method of claim 1 , wherein the second entry identifier is a sequential increment of the first entry identifier.

10. A client device in a network that comprises a plurality of servers and a plurality of client devices, the client device comprising:
a processor; and
a memory communicably coupled to the processor, the memory including computer instructions configured to cause the processor to:
allocate a first entry identifier corresponding to a first log entry of a plurality of log entries created by a first client process of a plurality of client processes, wherein the first client process runs on a first client device of the plurality of client devices;
allocate a second entry identifier corresponding to a second log entry of the plurality of log entries created by a second client process of the plurality of client processes, wherein the second client process runs on a second client device of the plurality client devices concurrently with running of the first client process on the first client device, and wherein the second entry identifier is allocated subsequent to the first entry identifier being allocated;
authorize transmission of the first log entry by the first client device and the second log entry by the second client device to one or more of the plurality of servers in the network, wherein the authorizing comprises authorizing transmission of the second log entry prior to transmission of the first log entry;
receive acknowledgement of the first log entry and acknowledgement of the second log entry from one or more of the plurality of servers that replicate the first log entry and the second log entry in a journal that includes a background thread checkpoint, maintain a copy of the journal in cache memory, update the journal by flushing at least the first log entry and the second log entry up until a predetermined checkpoint, and delete one or more other journals older than the predetermined checkpoint; and
transmit the acknowledgement of the second log entry subsequent to the acknowledgement of the first log entry.

11. The client device of claim 10 , wherein the computer instructions further comprise computer instructions to cause the processor to batch a plurality of log entries including the first log entry and the second log entry by one of the client devices.

12. The client device of claim 10 , wherein the computer instructions further comprise computer instructions configured to cause the processor to:
allocate a third entry identifier corresponding to a third log entry of the plurality of log entries created by a third client process of the plurality of client processes, wherein the third client process runs on a third client device of the plurality client devices concurrently with running of the first client process on the first client device and running of the second client process on the second client device, and wherein the third entry identifier is allocated subsequent to the second entry identifier being allocated;
authorize transmission of the third log entry by the third client device to one or more of the plurality of servers in the network prior to transmission of the first log entry;
receive acknowledgement of the third log from one or more of the plurality of servers; and
transmit the acknowledgement of the third log entry subsequent to the acknowledgement of the second log entry.

13. The client device of claim 12 , wherein transmissio

Assignees

• Salesforce Com Inc

Inventors

• Jujjuri Venkateswararao
• Tiwari Sameer
• Seeger Jr James John
• Helland Patrick James

Classifications

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• H04L67/1095

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H04L67/42

  Electricity · mapped topic

• H04L63/102 Primary

  Entity profiles · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title