Optimized filesystem walk for backup operations

Patent metadata

Field	Value
Publication number	US-10521309-B1
Application number	US-201314138525-A
Country	US
Kind code	B1
Filing date	Dec 23, 2013
Priority date	Dec 23, 2013
Publication date	Dec 31, 2019
Grant date	Dec 31, 2019

Abstract

The system for backing up data comprises an input interface configured to receive an input set of files. The system for backing up data additionally comprises a processor configured to add modified objects to a list of modified objects in the set of files, and, in the event it is determined to provide the list of modified objects to a backup process, provide the list to the backup process.

First claim

What is claimed is:

1. A system for backing up data, comprising:
an input interface configured to receive a set of files; and
a processor configured to:
add modified objects to a list of modified objects in the set of files, wherein to add modified objects to the list of modified objects comprises:
in response to a determination that a modified object of the modified object is identified, determining whether the identified modified object is within a defined proximity of other modified objects included on the list of modified objects, and
in response to a determination that the identified modified object is determined to not be within the defined proximity the other modified objects, creating a new list of modified objects and adding the identified modified object to the new list of modified objects;

determine whether to provide the list of modified objects to a backup process based at least in part on one or more of a threshold amount of data or a threshold number of files included in the list of modified objects; and
in response to a determination to provide the list of modified objects to the backup process:
provide the list to the backup process for backup, wherein the backup process processes the list in connection with backing up one or more of the modified objects.

2. The system of claim 1 , wherein determining to provide the list of modified objects to a backup process is based at least in part on the total file size of the list.

3. The system of claim 2 , wherein the total file size of the list is compared to a size threshold.

4. The system of claim 1 , wherein determining to provide the list of modified objects to a backup process is based at least in part on the number of files in the list.

5. The system of claim 4 , wherein the number of files in the list is compared to a file number threshold.

6. The system of claim 1 , wherein determining to provide the list of modified objects to a backup process is based at least in part on a location of a volume boundary within the set of files.

7. The system of claim 1 , wherein the processor is further configured to determine whether there are more files of the set of files to process.

8. The system of claim 7 , wherein in response to a determination that more files of the set of files to process are to be processed, then the processor is further configured to select a next file for processing.

9. The system of claim 1 , wherein the backup process queues the modified objects in the list to be read.

10. The system of claim 9 , wherein the modified objects in the list are read in a single access to storage device.

11. The system of claim 9 , wherein the modified objects in the list are read in multiple reads in a sequence.

12. The system of claim 11 , wherein the sequence targets a minimum seek time.

13. The system of claim 11 , wherein the sequence targets a minimum access time.

14. The system of claim 1 , wherein the set of files comprises one or more of the following: a complete system, a list of filesystems, a single filesystem, a mount point, a logical object that is used to aggregate files, or a folder.

15. The system of claim 1 , wherein the one or more modified objects are stored on a backup server in connection with an incremental backup.

16. The system of claim 1 , wherein the processor is further configured to determine whether to provide the new list of modified objects to the backup process based at least in part on one or more of the threshold amount of data or the threshold number of files included in the list.

17. The system of claim 1 , wherein the list of modified objects and the new list of modified objects are associated with different backup buffers.

18. The system of claim 1 , wherein determining, in connection with walking of a set of files, whether to add the modified objects to the list of modified objects in the set of files based on whether the modified objects have changed since a last backup.

19. The system of claim 1 , wherein determining whether the identified modified objects are within the defined proximity of other modified objects comprises determining that the identified modified objects are from a same volume as the other modified objects included on the list of modified objects.

20. The system of claim 1 , wherein the determining whether the identified modified object is within the defined proximity of the other modified objects included on the list of modified objects includes determining whether the identified object is stored on a volume that is the same as one or more volumes on which the other modified objects included in the list of modified objects are stored.

21. The system of claim 1 , wherein the processor is further configured to determine whether to add the modified objects to the list of modified objects to be provided to the backup process based at least in part on a determination of whether a backup storage system comprises the corresponding modified object.

22. The system of claim 1 , wherein the processing the list in connection with the backing up the one or more of the modified objects comprises:
reading a plurality of objects in the list of modified objects from a storage volume on which the plurality of objects are stored, the plurality of objects being read in a sequential manner based on a corresponding physical storage locations of the plurality of objects.

23. The system of claim 22 , wherein the reading of the plurality of objects does not comprise random access.

24. The system of claim 23 , wherein the one or more modified objects are read from storage in a single access to a storage device on which the one or more modified objects are stored.

25. The system of claim 22 , wherein the plurality of objects are read in a sequence determined according to the corresponding physical storage locations of the plurality of objects.

26. The system of claim 22 , wherein the plurality of objects are read in a manner based on a determination that minimizes one or more of seek times and access times.

27. The system of claim 1 , wherein the backup process queues the one or more modified objects in the list such that the backup process reads the one or more modified objects from storage in a single access to a storage device on which the one or more modified objects are stored.

28. The system of claim 1 , wherein the backup process queues the one or more modified objects in the list such that the backup process reads the one or more modified objects from storage in a sequential manner based on a corresponding physical storage locations of the plurality of objects.

29. A method for backing up data, comprising:
receiving a set of files;
adding, using a processor, modified objects to a list of modified objects in the set of files, wherein adding modified objects to the list of modified objects comprises:
in response to a determination that a modified object of the modified object is identified, determining whether the identified modified object is within a defined proximity of other modified objects included on the list of modified objects, and
in response to a determination that the identified modified object is determined to not be within the defined proximity of the other modified objects, creating a new list of modified objects and adding the identified modified object to the new list of modified objects;

determining whether to

Assignees

• Emc Ip Holding Co Llc

Inventors

• Mandic Vladimir

Classifications

• G06F11/1458 Primary

  Management of the backup or restore process · CPC title

• G06F11/1451 Primary

  by selection of backup contents · CPC title

• G06F2201/81

  Threshold · CPC title