Watermarking of Events of an Ordered Event Stream

Patent metadata

Field	Value
Publication number	US-2021406273-A1
Application number	US-202016915762-A
Country	US
Kind code	A1
Filing date	Jun 29, 2020
Priority date	Jun 29, 2020
Publication date	Dec 30, 2021
Grant date	—

Abstract

Watermarking of an ordered event stream (OES) is disclosed. Writing events to an OES can advance populating of the OES, which can be correspond to progress values. Progress values can enable a reader to know where it is in an OES. A reported progress value can be determined from the progress values based on an event being committed to the OES. Moreover, an effective progress value can be determined from the progress values and monitoring the OES for events with that defer being committed to the OES. A watermark for the OES can be determined from the effective progress value such that reading beyond the effective progress value is restricted. This can provide confidence that the reader has read out properly committed events and has not omitted any events having deferred commitment.

First claim

What is claimed is:

1 . A system, comprising:
a processor; and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
receiving raw progress data corresponding to an advancement of an event population of an ordered event stream of an ordered event stream storage system;
determining effective progress data based on the raw progress data and a commitment state of a first event of the ordered event stream;
determining a watermark for the ordered event stream based on the effective progress data; and
setting a first permission to read a second event of the ordered event stream based on the watermark.

2 . The system of claim 1 , wherein the first event of the ordered event stream is a different event than the second event of the ordered event stream.

3 . The system of claim 1 , wherein the first event of the ordered event stream is a same event as the second event of the ordered event stream.

4 . The system of claim 1 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a second progress of the advancement of the event population of the ordered event stream, and wherein the first progress lags the second progress according to a third progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

5 . The system of claim 1 , The system of claim 1 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a second progress of the advancement of the event population of the ordered event stream, and wherein the first progress is same as the second progress according to a third progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

6 . The system of claim 1 , wherein the commitment state of the first event of the ordered event stream is inherited from a transaction object that corresponds to storage of the first event of the ordered event stream.

7 . The system of claim 6 , wherein the commitment state of the first event of the ordered event stream is inherited from a transaction object that corresponds to storage of the first event of the ordered event stream and a second event of the ordered event stream.

8 . The system of claim 1 , wherein the raw progress data corresponds to a time based advancement of the event population of the ordered event stream.

9 . The system of claim 1 , wherein the raw progress data corresponds to a count based advancement of the event population of the ordered event stream.

10 . The system of claim 1 , wherein the raw progress data corresponds to a distance based advancement of the event population of the ordered event stream.

11 . The system of claim 1 , wherein the operations further comprise, in response to determining reported progress data based on the raw progress data and commitment of a third event of the ordered event stream wherein the third event of the ordered stream is not permitted to be read according to the first permission, setting a second permission to read the third event of the ordered stream.

12 . The system of claim 11 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the reported progress data indicates a second progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a third progress of the advancement of the event population of the ordered event stream, and wherein the third progress lags the second progress that lags the first progress according to a fourth progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

13 . The system of claim 11 , wherein the raw progress data indicates a first progress of the advancement of the event population of the ordered event stream, wherein the reported progress data indicates a second progress of the advancement of the event population of the ordered event stream, wherein the effective progress data indicates a third progress of the advancement of the event population of the ordered event stream, and wherein the third progress, the second progress, and the first progress are the same progress according to a fourth progress of the advancement of the event population of the ordered event stream corresponding to the commitment state of the first event of the ordered event stream.

14 . A method, comprising:
writing, by a system comprising a processor, a first event to an ordered event stream of an ordered event stream storage system, wherein the writing of the event advances a progression of the event population of the ordered event stream, and wherein the progression corresponds to a raw progress value of raw progress values; determining, by the system, an effective progress value based on a deferred commitment of a second event to the ordered event stream; determining, by the system, a watermark for the ordered event stream based on the effective progress value; and enabling, by the system, reading of a portion of events from the ordered event stream based on the watermark.

15 . The method of claim 14 , wherein the enabling the reading of the portion of events from the ordered event stream based on the watermark permits reading of the events up to the effective progress value.

16 . The method of claim 15 , wherein the effective progress value is less than the raw progress value.

17 . The method of claim 14 , wherein the progression of the event population of the ordered event stream is selected from a group of progressions comprising a time based progression, a count based progression, and a distance based progression.

18 . A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
receiving a raw progress value, wherein the raw progress value was determined in response to a first event being written to an ordered event stream of an ordered event stream storage system, wherein the first event being written advanced a progression of the event population of the ordered event stream, and wherein the raw progress value corresponds the progression of the event population at the writing of the first event; determining an effective progress value based on a deferred commitment of a second event to the ordered event stream; indicating a watermark for the ordered event stream based on the effective progress value; and altering a permission related to reading of a portion of events from the ordered event stream based on the watermark.

19 . The non-transitory machine-readable medium of claim 18 , wherein the operations further comprise:
in response to the altering the permission not enabling reading a second portion of events from the ordered event stream, determining a reported progress value based on a commitment of a third event to the ordered event stream; and enabling reading of the second portion of the events from the ordered event stream based on the reported progress value.

20 . The non-transitory mach

Assignees

• Emc Ip Holding Co Llc

Inventors

• Danilov Mikhail
• Altaye Yohannes

Classifications

• G06F16/25 Primary

  Integrating or interfacing systems involving database management systems · CPC title

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G11B20/00905

  multiple watermarks used in combination · CPC title