Method and apparatus for a mechanism of disaster recovery and instance refresh in an event recordation system

Patent metadata

Field	Value
Publication number	US-10783044-B2
Application number	US-201816222925-A
Country	US
Kind code	B2
Filing date	Dec 17, 2018
Priority date	Sep 24, 2018
Publication date	Sep 22, 2020
Grant date	Sep 22, 2020

Abstract

A method and apparatus for a mechanism of disaster recovery and instance refresh in an event recordation system are described. A first request for events associated with a first topic is received. A first set of events associated with the first topic are retrieved from a primary event recordation system. Each event from the first set of events includes a first commit identifier indicating the order with which each event is stored in the primary event recordation system. For each event from the first set of events a first replay identifier is determined based on the first commit identifier and a first value of a low commit identifier. The first replay identifier is exposed to event consumers for identifying the event. In response to the first request each event from the first set of events is transmitted with the replay identifier.

First claim

What is claimed is:

1. A method in an event recordation and distribution system, the method comprising:
receiving a first request for events associated with a first topic;
retrieving from a primary event recordation system a first set of events associated with the first topic, wherein each event from the first set of events includes a first commit identifier indicating the order with which each event is stored in the primary event recordation system;
determining for each event from the first set of events, based on the first commit identifier and a first value of a low commit identifier, a first replay identifier that is exposed to event consumers for identifying the event, wherein the first value of the low commit identifier identifies the last event stored in a second event recordation system before the first set of events and the second event recordation system is different from the primary event recordation system; and
transmitting in response to the first request each event from the first set of events with the first replay identifier.

2. The method of claim 1 , wherein when no event is stored in the second event recordation system for the first topic before the first set of events, for each event from the first set of events, the first replay identifier is identical to the first commit identifier identifying the order with which each event is stored in the primary event recordation system.

3. The method of claim 1 , further comprising:
responsive to determining that the primary event recordation system is not available, performing the following:
retrieving from a target event recordation system a second set of events associated with the first topic, wherein each event from the second set of events includes a second commit identifier indicating the order with which each event is stored in the target event recordation system, and wherein the second set of events are stored in the target event recordation system after the first set of events are stored in the primary event recordation system,
determining for each event of the second set of events, based on the second commit identifier and a second value of the low commit identifier, a second replay identifier that is to be exposed to the event consumers for identifying each event of the second set, wherein the second value of the low commit identifier identifies the last event from the first set of events stored in the second event recordation system, and
transmitting each event from the second set of events with the second replay identifier, wherein second replay identifiers associated with the events of the second set succeed first replay identifiers associated with the events of the first set and an event consumer is to receive uninterrupted stream of events for the first topic including the first set of events and the second set of events ordered based on the first and the second replay identifiers.

4. The method of claim 3 , further comprising:
storing each one from the first set of events in the second event recordation system with the first replay identifier.

5. The method of claim 4 , further comprising:
receiving a replay request including a third replay identifier identifying an event from the first set; and
responsive to determining that the primary event recordation system is not available, performing the following:
retrieving, based on the third replay identifier, a subset of events from the first set of events as stored in the second event recordation system,
retrieving, based on the third replay identifier, the second set of events from the target event recordation system, and
transmitting, in response to the replay request, the subset of events and the second set of events.

6. The method of claim 5 , wherein performing further includes:
determining a third commit identifier that corresponds to the third replay identifier, wherein the third commit identifier identifies an event as stored in at least one of the primary event recordation system and the target event recordation system;
determining that the third commit identifier identifies an event from the first set of events;
wherein retrieving the subset of events from the first set of events includes retrieving events with commit identifiers that are greater than the third commit identifier; and
wherein retrieving the second set of events from the target event recordation system is performed in response to determining that the third commit identifier identifies an event from the first set.

7. The method of claim 3 , wherein the primary event recordation system and the target event recordation system are short term storage systems.

8. The method of claim 1 , wherein the second event recordation system is a long-term storage recordation system.

9. The method of claim 1 , wherein the event recordation and distribution system is a multi-tenant system and the first topic is associated with a tenant identifier uniquely identifying a tenant.

10. The method of claim 1 , wherein the first request is a request to subscribe to the first topic.

11. A non-transitory computer readable storage medium including code which when executed by one or more processors in an event recordation and distribution system cause the event recordation and distribution system to perform operations comprising:
receiving a first request for events associated with a first topic;
retrieving from a primary event recordation system a first set of events associated with the first topic, wherein each event from the first set of events includes a first commit identifier indicating the order with which each event is stored in the primary event recordation system;
determining for each event from the first set of events, based on the first commit identifier and a first value of a low commit identifier, a first replay identifier that is exposed to event consumers for identifying the event, wherein the first value of the low commit identifier identifies the last event stored in a second event recordation system before the first set of events and the second event recordation system is different from the primary event recordation system; and
transmitting in response to the first request each event from the first set of events with the first replay identifier.

12. The non-transitory computer readable storage medium of claim 11 , wherein when no event is stored in the second event recordation system for the first topic before the first set of events, for each event from the first set of events, the first replay identifier is identical to the first commit identifier identifying the order with which each event is stored in the primary event recordation system.

13. The non-transitory computer readable storage medium of claim 11 , further comprising:
responsive to determining that the primary event recordation system is not available, performing the following:
retrieving from a target event recordation system a second set of events associated with the first topic, wherein each event from the second set of events includes a second commit identifier indicating the order with which each event is stored in the target event recordation system, and wherein the second set of events are stored in the target event recordation system after the first set of events are stored in the primary event recordation system,
determining for each event of the second set of events, based on the second commit identifier and a second value of the low commit identifier, a second replay identifier that is to be exposed to the event consumers for identifying each event of the second set, wherein the second value of the low commit identifier identifies the last event from the first set of events

Assignees

• Salesforce Com Inc
• Salesforce Inc

Inventors

• Zhao Yingwu
• Jain Samarpan
• Hildebrand Hal Scott
• Syomichev Alexey
• Gerba Emin Eliot
• Pesenson Igor
• Spragg David William
• Hurst Jay
• Bandyopadhyay Soumen

Classifications

• G06F2201/805

  Real-time · CPC title

• G06F2201/82

  Solving problems relating to consistency · CPC title

• G06F11/1471 Primary

  involving logging of persistent data for recovery · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/2023 Primary

  Failover techniques · CPC title