Watermarking of events of an ordered event stream

Patent metadata

Field	Value
Publication number	US-11360992-B2
Application number	US-202016915762-A
Country	US
Kind code	B2
Filing date	Jun 29, 2020
Priority date	Jun 29, 2020
Publication date	Jun 14, 2022
Grant date	Jun 14, 2022

Abstract

Watermarking of an ordered event stream (OES) is disclosed. Writing events to an OES can advance populating of the OES, which can be correspond to progress values. Progress values can enable a reader to know where it is in an OES. A reported progress value can be determined from the progress values based on an event being committed to the OES. Moreover, an effective progress value can be determined from the progress values and monitoring the OES for events with that defer being committed to the OES. A watermark for the OES can be determined from the effective progress value such that reading beyond the effective progress value is restricted. This can provide confidence that the reader has read out properly committed events and has not omitted any events having deferred commitment.

First claim

What is claimed is:

1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
receiving raw progress data for an event write operation corresponding to an advancement of an event population of an ordered event stream of an ordered event stream storage system;
determining effective progress data for the event write operation based on the raw progress data and a commitment state of a first event of the ordered event stream, wherein the commitment state is inherited from a transaction object;
determining a watermark for the ordered event stream based on the effective progress data for the event write operation; and
setting a first permission to read a second event of the ordered event stream based on the watermark.

2. The system of claim 1 , wherein the first event of the ordered event stream is a different event than the second event of the ordered event stream.

3. The system of claim 1 , wherein the first event of the ordered event stream is a same event as the second event of the ordered event stream.

4. The system of claim 1 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a second progress of the advancement of the event population of the ordered event stream, and wherein the first progress lags the second progress according to a third progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

5. The system of claim 1 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a second progress of the advancement of the event population of the ordered event stream, and wherein the first progress is same as the second progress according to a third progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

6. The system of claim 1 , wherein the transaction object corresponds to storage of the first event of the ordered event stream.

7. The system of claim 6 , wherein the transaction object corresponds to storage of the first event of the ordered event stream and a second event of the ordered event stream.

8. The system of claim 1 , wherein the raw progress data corresponds to a time-based advancement of the event population of the ordered event stream.

9. The system of claim 1 , wherein the raw progress data corresponds to a count-based advancement of the event population of the ordered event stream.

10. The system of claim 1 , wherein the raw progress data corresponds to a distance-based advancement of the event population of the ordered event stream.

11. The system of claim 1 , wherein the operations further comprise, in response to determining reported progress data based on the raw progress data and commitment of a third event of the ordered event stream wherein the third event of the ordered stream is not permitted to be read according to the first permission, setting a second permission to read the third event of the ordered stream.

12. The system of claim 11 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the reported progress data indicates a second progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a third progress of the advancement of the event population of the ordered event stream, and wherein the third progress lags the second progress that lags the first progress according to a fourth progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

13. The system of claim 11 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the reported progress data indicates a second progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a third progress of the advancement of the event population of the ordered event stream, and wherein the third progress, the second progress, and the first progress are the same progress according to a fourth progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

14. A method, comprising:
writing, by a system comprising a processor, a first event to an ordered event stream of an ordered event stream storage system, wherein the writing of the event advances a progression of the event population of the ordered event stream, and wherein the progression corresponds to a raw progress value of raw progress values corresponding to event write operations;
determining, by the system, an effective progress value corresponding to an event write operation of the event write operations based on a deferred commitment of a second event to the ordered event stream, wherein the deferred commitment state is inherited from a transaction object;
determining, by the system, a watermark for the ordered event stream based on the effective progress value corresponding to the event write operation; and
enabling, by the system, reading of a portion of events from the ordered event stream based on the watermark.

15. The method of claim 14 , wherein the enabling the reading of the portion of events from the ordered event stream based on the watermark permits reading of the events up to the effective progress value.

16. The method of claim 15 , wherein the effective progress value is less than the raw progress value.

17. The method of claim 14 , wherein the progression of the event population of the ordered event stream is selected from a group of progressions comprising a time-based progression, a count-based progression, and a distance-based progression.

18. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
receiving a raw progress value corresponding to an event write operation, wherein the raw progress value was determined in response to a first event being written to an ordered event stream of an ordered event stream storage system, wherein the first event being written advanced a progression of the event population of the ordered event stream, and wherein the raw progress value corresponds the progression of the event population at the writing of the first event;
determining an effective progress value corresponding to the event write operation based on a deferred commitment of a second event to the ordered event stream, wherein the deferred commitment state is inherited from a transaction object;
indicating a watermark for the ordered event stream based on the effective progress value; and
altering a permission related to reading of a portion of events from the ordered event stream based on the watermark.

19. The non-transitory machine-readable medium of claim 18 , wherein the operations further comprise:
in response to the altering the permission not enabling reading a second portion of events from the ordered event

Assignees

• Emc Ip Holding Co Llc

Inventors

• Danilov Mikhail
• Altaye Yohannes

Classifications

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G06F16/25 Primary

  Integrating or interfacing systems involving database management systems · CPC title

• G11B20/00905

  multiple watermarks used in combination · CPC title