Managing dependent delete operations among data stores

Patent metadata

Field	Value
Publication number	US-11567899-B2
Application number	US-201916702346-A
Country	US
Kind code	B2
Filing date	Dec 3, 2019
Priority date	Dec 3, 2019
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

Example distributed storage systems, delete managers, and methods provide for managing dependent delete operations among data stores. Dependent data operation entries and corresponding dependency sets may be identified in an operations log. Dependent data operations may be identified in each shard and data operation entries. A delete process for the data objects in the dependency set may be delayed until the delete process for the dependent data object completes.

First claim

What is claimed is:

1. A system, comprising:
a processor;
a memory;
a first operation log stored in the memory and configured to store operation entries for a first storage node;
a barrier engine, stored in the memory and executable by the processor to:
determine a first dependent operation entry in the first operations log indicating a first dependent data object at a first timestamp, wherein the first dependent operation entry includes a barrier object identifier; and
determine at least one data operation entry in the first operations log for a first dependency set of data objects, wherein:
a validity of the first dependent data object depends on the first dependency set of data objects; and
the at least one data operation entry for the first dependency set of data objects does not include the barrier object identifier; and

a delete manager, stored in the memory and executable by the processor to:
determine, based on the barrier object identifier, a first dependent delete request for the first dependent data object;
determine at least one dependency delete request for at least one data object in the first dependency set of data objects;
execute, responsive to the barrier object identifier, a delete process for the first dependent delete request for the first dependent data object, wherein the delete process includes prioritizing the first dependent delete request in at least one operation queue; and
delay, based on at least one time interval and until the delete process for the first dependent delete request completes, delete processing of the at least one dependency delete request responsive to the at least one data operation entry not including the barrier object identifier.

2. The system of claim 1 , wherein:
the barrier engine is further executable to monitor a first time interval from a first time to a second time;
the first timestamp is no earlier than the first time and prior to the second time; and
the at least one time interval includes the first time interval.

3. The system of claim 2 , further comprising:
a second storage node; and
a replication manager, stored in the memory and executable by the processor to replicate data objects between the first storage node and the second storage node by:
replicating the first dependent delete request prior to the second time; and
replicating, between the first time and the second time, dependency delete requests with timestamps prior to the first time.

4. The system of claim 3 , wherein the replication manager is further executable to replicate data objects between the first storage node and second storage node by:
replicating, after the second time, the at least one dependency delete request for the first dependent data object.

5. The system of claim 2 , wherein:
the barrier engine is further executable to:
determine a second dependent operation entry in the first operations log indicating a second dependent data object at a second timestamp;

the first dependent data object is a snapshot of the first storage node at the first time; and
the second dependent data object is a snapshot of the first storage node at the second time.

6. The system of claim 5 , wherein the barrier engine is further executable to:
generate the first dependent data object at the first time;
determine the first time interval between the first time and the second time using a predetermined snapshot interval; and
generate the second dependent data object at the second time.

7. The system of claim 6 , wherein the barrier engine is further executable to:
increase, until replication of the first dependent delete request to a second storage node, the first time interval; and
delay, responsive to increasing the first time interval, generation of the second dependent data object.

8. The system of claim 1 , wherein the delete manager is further executable to:
generate, at a first time of the first timestamp, a first operation queue of dependent delete requests, including the first dependent delete request; and
generate, at the first time, a second operation queue of delete requests, excluding the first dependent delete request and including the at least one dependency delete request for the at least one data object in the first dependency set of data objects.

9. The system of claim 1 :
further comprising:
a second operations log stored in the memory and configured to store operation entries for a second storage node; and
a replication manager stored in memory and executable by the processor to replicate data objects between the first storage node and the second storage node;

wherein:
the barrier engine is further executable to determine the first dependent operation entry in the second operation log indicating the first dependent data object; and
the delete manager is further executable to:
execute, based on the barrier object identifier, a delete process for the first dependent delete request for the first dependent data object in both the first operations log and the second operations log; and
delay, until the delete process for the first dependent delete request completes in the first operation log and the second operation log, delete processing of the at least one dependency delete request.

10. A computer-implemented method, comprising:
storing a first operations log associated with a first storage node;
determining a first dependent operation entry in the first operations log indicating a first dependent data object at a first timestamp, wherein the first dependent operation entry includes a barrier object identifier;
determining at least one data operation entry in the first operations log for a first dependency set of data objects, wherein:
a validity of the first dependent data object depends on the first dependency set of data objects; and
the at least one data operation entry does not include the barrier object identifier;

determining, based on the barrier object identifier, a first dependent delete request for the first dependent data object;
determining at least one dependency delete request for at least one data object in the first dependency set of data objects;
executing, responsive to the barrier object identifier, a delete process for the first dependent delete request for the first dependent data object, wherein the delete process includes prioritizing the first dependent delete request in at least one operation queue; and
delaying, based on at least one time interval and until the delete process for the first dependent delete request completes, the delete process for the at least one dependency delete request responsive to the at least one data operation entry not including the barrier object identifier.

11. The computer-implemented method of claim 10 , further comprising:
monitoring a first time interval from a first time to a second time, wherein:
the first timestamp is no earlier than the first time and prior to the second time; and
the at least one time interval includes the first time interval.

12. The computer-implemented method of claim 11 , further comprising:
replicating data objects between the first storage node and a second storage node by:
replicating, prior to the second time, the first dependent delete request; and
replicating, prior to the second time, dependency delete requests with timestamps prior to the first time.

13. The computer-implemented method of claim 12 , wherein replicating data objects between the first storage node and second storage node further includes:
replicating, after the second time, the at least one dependency delete request for the first dependent data object.

Assignees

• Western Digital Tech Inc

Inventors

• De Schrijver Frederik
• Demoor Thomas
• D'Halluin Carl

Classifications

• G06F16/1824

  implemented using Network-attached Storage [NAS] architecture (distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097) · CPC title

• G06F16/162 Primary

  Delete operations (erasing in storage systems G06F3/0652) · CPC title

• G06F16/1844

  Management specifically adapted to replicated file systems · CPC title

• G06F16/2322

  using timestamps · CPC title

• G06F16/1734

  Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs · CPC title