Stream retention in a data storage system

Patent metadata

Field	Value
Publication number	US-11599545-B2
Application number	US-202016795104-A
Country	US
Kind code	B2
Filing date	Feb 19, 2020
Priority date	Feb 19, 2020
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

Stream retention in a data storage system is described herein. A method as described herein can include receiving, by a device operatively coupled to a processor, a hold for a portion of a data stream, wherein the data stream comprises respective events indexed by time, and wherein the portion of the data stream is bounded by a first event of the data stream that is associated with a first time and a second event of the data stream that is associated with a second time; generating, by the device, a synthetic stream comprising the portion of the data stream; and preventing, by the device, events of the data stream that are included in the synthetic stream from alteration.

First claim

What is claimed is:

1. A data storage system, comprising:
a memory that stores computer executable components; and
a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise:
a hold processing component that receives a command to implement a hold on a portion of a data stream, wherein the data stream comprises respective events indexed by time, and wherein the portion of the data stream is bounded by a first event of the data stream that is associated with a first time and a second event of the data stream that is associated with a second time;
a stream generation component that constructs a synthetic stream from the portion of the data stream;
a data retention component that prevents alteration of events of the data stream that are included in the synthetic stream; and
a truncation component that, in response to a truncation of the data stream being initiated, causes an event, of the respective events that are associated with the hold, to be hidden from the data stream, wherein the truncation component preserves the event in the synthetic stream.

2. The data storage system of claim 1 , wherein the command is a first command to implement a first hold on a first portion of the data stream, and wherein the hold processing component receives a second command to implement a second hold on a second portion of the data stream bounded by a third event of the data stream that is associated with a third time and a fourth event of the data stream that is associated with a fourth time.

3. The data storage system of claim 2 , wherein the synthetic stream is a first synthetic stream, and wherein the stream generation component constructs a second synthetic stream from the second portion of the data stream.

4. The data storage system of claim 3 , wherein the data retention component prevents alteration of an event of the data stream that is included in the first synthetic stream and the second synthetic stream pending removal of the first synthetic stream and the second synthetic stream.

5. The data storage system of claim 1 , wherein the hold is a first hold, and wherein the computer executable components further comprise:
a hold logging component that maintains information relating to respective holds implemented on the data stream, the respective holds comprising the first hold.

6. The data storage system of claim 5 , wherein the computer executable components further comprise:
a hold reporting component that provides the information relating to the respective holds to a requesting system in response to receiving a request from the requesting system.

7. The data storage system of claim 1 , wherein the command is a first command, wherein the hold processing component receives a second command to remove the hold on the portion of the data stream, and wherein the stream generation component removes the synthetic stream in response to the hold on the portion of the data stream being removed.

8. A method, comprising:
receiving, by a device operatively coupled to a processor, a hold for a portion of a data stream, wherein the data stream comprises respective events indexed by time, and wherein the portion of the data stream is bounded by a first event of the data stream that is associated with a first time and a second event of the data stream that is associated with a second time;
generating, by the device, a synthetic stream comprising the portion of the data stream;
preventing, by the device, events of the data stream that are included in the synthetic stream from alteration; and
preserving, by the device in response to a truncation of the data stream being initiated, an event, of the respective events that are associated with the hold, in the synthetic stream while removing the event from the data stream.

9. The method of claim 8 , wherein the command is a first command, and wherein the method further comprises:
receiving, by the device, a second command to remove the hold; and
removing the synthetic stream in response to the hold being removed.

10. The method of claim 8 , wherein the hold is a first hold for a first portion of the data stream, and wherein the method further comprises:
receiving, by the device, a second hold for a second portion of the data stream, wherein the second portion of the data stream is bounded by a third event of the data stream that is associated with a third time and a fourth event of the data stream that is associated with a fourth time.

11. The method of claim 10 , wherein the synthetic stream is a first synthetic stream, and wherein the method further comprises:
constructing, by the device, a second synthetic stream from the second portion of the data stream.

12. The method of claim 11 , further comprising:
preventing, by the device, alteration of an event of the data stream that is included in the first synthetic stream and the second synthetic stream pending removal of the first synthetic stream and the second synthetic stream.

13. The method of claim 8 , wherein the hold is a first hold, and wherein the method further comprises:
recording, by the device, information relating to respective holds implemented on the data stream, wherein the respective holds comprise the first hold.

14. The method of claim 13 , further comprising:
providing, by the device in response to receiving a request from a requesting system, the information relating to the respective holds to the requesting system.

15. A non-transitory machine-readable storage medium comprising computer executable instructions that, when executed by a processor of a data storage system, facilitate performance of operations, the operations comprising:
initiating a hold for a portion of a data stream, wherein the data stream comprises respective events indexed by time, and wherein the portion of the data stream is bounded by a first event of the data stream that is associated with a first time and a second event of the data stream that is associated with a second time;
adding the portion of the data stream to a synthetic data stream;
preventing alteration of events of the data stream that are included in the synthetic data stream; and
in response to a pruning operation for the data stream being initiated, removing an event, of the respective events that are associated with the hold, from view within the data stream while preserving the event in the synthetic data stream.

16. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:
discarding the synthetic data stream in response to the hold being removed.

17. The non-transitory machine-readable storage medium of claim 15 , wherein the hold is a first hold for a first portion of the data stream, wherein the synthetic data stream is a first synthetic data stream, and wherein the operations further comprise:
initiating a second hold for a second portion of the data stream, wherein the second portion of the data stream is bounded by a third event of the data stream that is associated with a third time and a fourth event of the data stream that is associated with a fourth time; and
adding the second portion of the data stream to a second synthetic data stream.

18. The non-transitory machine-readable storage medium of claim 17 , wherein the operations further comprise:
preventing alteration of events of the data stream that are included in the first synthetic data stream and the second synthetic data stream pending the first synthetic data stream and the second synthetic data stream b

Assignees

• Emc Ip Holding Co Llc

Inventors

• Danilov Mikhail
• Medvedev Igor

Classifications

• G06F16/2322

  using timestamps · CPC title

• G06F16/24568 Primary

  Data stream processing; Continuous queries · CPC title

• G06F16/1734

  Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs · CPC title

• G06F16/2228

  Indexing structures · CPC title

• G06F16/254

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title