Object-based storage replication and recovery

Patent metadata

Field	Value
Publication number	US-2016210308-A1
Application number	US-201615081520-A
Country	US
Kind code	A1
Filing date	Mar 25, 2016
Priority date	Mar 18, 2014
Publication date	Jul 21, 2016
Grant date	—

Abstract

A method and system for replicating file system entities in a manner that preserves block-level access and file system efficiency mechanisms such as deduplication are disclosed. In an embodiment, a replication engine receives a stream of file system entities that include a file system inodes and file system data blocks. The replication engine generates object-based storage (OBS) objects based on data and reference information specified by the file system entities. As part of generating the OBS objects, the replication engine generates at least one inode file object that associates file block numbers of a file system inode file and the inode numbers. The replication engine uses inode information to generate reference objects that logically associate file block numbers with data block numbers in per inode manner. The replication engine further generates data objects that contains the file system data blocks and that associates the data blocks with corresponding data block numbers.

First claim

What is claimed is:

1 . A method for managing file system entities in an object-based storage (OBS), said method comprising:
receiving a stream of file system entities that include,
inodes contained within an inode file and having respective assigned file index numbers utilized to index file system entities; and
data blocks including a data block that is logically associated within at least one of the inodes with at least two different file block numbers (FBNs); and
mapping the file system entities to OBS objects, said mapping comprising,
generating at least one inode file object that logically associates FBNs of the inode file with the file index numbers;
generating at least one indirection object that logically associates FBNs of indexed file system entities with respective ones of data block numbers corresponding to the received data blocks; and
generating at least one data object that contains the data blocks and that logically associates the data block numbers with the data blocks.

2 . The method of claim 1 , wherein said generating at least one inode file object comprises:
generating an inode file reference object that includes an ordered list of FBNs of the inode file within which the received inodes are contained; and generating an inode file data object that includes a per-inode, ordered list of metadata content, wherein each position within the ordered list of FBNs uniquely corresponds to a respective position in the ordered list of metadata content.

3 . The method of claim 1 , wherein said generating at least one indirection object comprises assigning to each indirection object, an object ID that corresponds to a respective one of the file index numbers and that is used as an object access key.

4 . The method of claim 1 , wherein said generating at least one data object comprises:
generating a data reference object that includes an ordered list of the data block numbers; and generating a data content object that includes a per-block, ordered list of data content of the respective data blocks, wherein each position within the ordered list of the data block numbers uniquely corresponds to a respective position in the ordered list of the data block content.

5 . The method of claim 4 , further comprising:
in response to a request to access an indexed file system entity within the OBS,
accessing the at least one inode file object to determine a file index number based on an inode file FBN corresponding to an inode that represents a file system entity that is within the file system pathname of the indexed file system entity;
identifying an indirection object based on the determined file index number; and
accessing the identified indirection object to identify one or more data block numbers that are associated, within the identified indirection object, with the file system entity that is within a file system pathname of the indexed file system entity.

6 . The method of claim 5 , wherein said accessing the identified indirection object comprises identifying the data block numbers based on at least one FBN specified by the access request.

7 . The method of claim 5 , further comprising:
in response to the request to access the indexed file system entity, selectively accessing one or more entries within the at least one data object based on the identified one or more data block numbers.

8 . The method of claim 7 , wherein said selectively accessing one or more entries within the at least one data object comprises reading the data reference object to locate the identified one or more data block numbers, and accessing the data content object to locate, based on the locations of the one or more data block numbers within the data reference object, one or more data blocks corresponding to the data block numbers.

9 . An apparatus for managing file system entities in an object-based storage (OBS), said apparatus comprising:
a processor; and a machine-readable medium having program code executable by the processor to cause the apparatus to,
receive a stream of file system entities that include,
inodes contained within an inode file and having respective assigned file index numbers utilized to index file system entities; and
data blocks including a data block that is logically associated within at least one of the inodes with at least two different file block numbers (FBNs); and

map the file system entities to OBS objects, said mapping comprising,
generating at least one inode file object that logically associates FBNs of the inode file with the file index numbers;
generating at least one indirection object that logically associates FBNs of indexed file system entities with respective ones of data block numbers corresponding to the received data blocks; and
generating at least one data object that contains the data blocks and that logically associates the data block numbers with the data blocks.

10 . The apparatus of claim 9 , wherein said generating at least one inode file object comprises:
generating an inode file reference object that includes an ordered list of FBNs of the inode file within which the received inodes are contained; and generating an inode file data object that includes a per-inode, ordered list of metadata content, wherein each position within the ordered list of FBNs uniquely corresponds to a respective position in the ordered list of metadata content.

11 . The apparatus of claim 9 , wherein said generating at least one indirection object comprises assigning to each indirection object, an object ID that corresponds to a respective one of the file index numbers and that is used as an object access key.

12 . The apparatus of claim 9 , wherein said generating at least one data object comprises:
generating a data reference object that includes an ordered list of the data block numbers; and generating a data content object that includes a per-block, ordered list of data content of the respective data blocks, wherein each position within the ordered list of the data block numbers uniquely corresponds to a respective position in the ordered list of the data block content.

13 . The apparatus of claim 12 , wherein the program code is further executable by the processor to cause the apparatus to:
in response to a request to access an indexed file system entity within the OBS,
access the at least one inode file object to determine a file index number based on an inode file FBN corresponding to an inode that represents a file system entity that is within the file system pathname of the indexed file system entity;
identify an indirection object based on the determined file index number; and
access the identified indirection object to identify one or more data block numbers that are associated, within the identified indirection object, with the file system entity that is within a file system pathname of the indexed file system entity.

14 . The apparatus of claim 13 , wherein said accessing the identified indirection object comprises identifying the data block numbers based on at least one FBN specified by the access request.

15 . The apparatus of claim 13 wherein the program code is further executable by the processor to cause the apparatus to:
in response to the request to access the indexed file system entity, selectively access one or more entries within the at least one data object based on the identified one or more data block numbers, wherein said selectively accessing one or more entries within the at least one data object comprises reading the data reference object to locate the identified one or more data block numbers, and acce

Assignees

• Netapp Inc

Inventors

• Shekhar Sisir
• Eisler Michael R
• Muthyala Kartheek
• Kumar Ranjit

Classifications

• G06F16/178

  Techniques for file synchronisation in file systems · CPC title

• G06F16/1844

  Management specifically adapted to replicated file systems · CPC title

• G06F16/174

  Redundancy elimination performed by the file system (error detection or correction of the data by redundancy in operations G06F11/14) · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F16/116 Primary

  Details of conversion of file system types or formats · CPC title