Hybrid cloud mirroring to facilitate performance, migration, and availability

Patent metadata

Field	Value
Publication number	US-10650035-B2
Application number	US-201715610400-A
Country	US
Kind code	B2
Filing date	May 31, 2017
Priority date	Jan 6, 2017
Publication date	May 12, 2020
Grant date	May 12, 2020

Abstract

Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.

First claim

What is claimed:

1. A method comprising:
receiving, from an application layer of a Z File System (ZFS) system and through a system call interface of an interface layer of the ZFS system, a request to perform a transaction with respect to a file;
based at least in part on the request, storing, via a mirror device operating at a virtual device layer of the ZFS system, data blocks and corresponding metadata in a storage pool of storage devices of the ZFS system, wherein:
the data blocks correspond to the file, and
the data blocks and the corresponding metadata are stored as logical blocks in a tree hierarchy so that a local version of the tree hierarchy is stored in the storage pool;

causing, via at least one cloud interface device of a plurality of virtual devices operating at the virtual device layer, storage of a set of cloud storage objects, the set of cloud storage objects comprising data of the data blocks and the corresponding metadata of the tree hierarchy so that a cloud version of the tree hierarchy is stored in a cloud object store;
receiving, from the application layer and through the system call interface, subsequent requests to perform transactions with respect to the file;
translating, by a data management unit of a transactional object layer of the ZFS system, the subsequent requests into I/O requests to perform I/O operations; and
upon receiving each I/O request of the I/O requests, synchronizing the local version of the tree hierarchy in the storage pool with the cloud version of the tree hierarchy in the cloud object store at least in part by:
causing, by the mirror device, a first I/O operation according to the respective I/O operation with respect to at least a portion of the local version of the tree hierarchy in the storage pool; and
causing, by the mirror device, the at least one cloud interface device to initiate a second I/O operation that mirrors the first I/O operation according to the respective I/O operation with respect to the cloud version of the tree hierarchy in the cloud object store.

2. The method of claim 1 , further comprising:
monitoring, by the mirror device and/or the at least one cloud interface device, results of each I/O request of the I/O requests for storage failures that cause the local version of the tree hierarchy and the cloud version of the tree hierarchy to become unsynchronized with one another; and
identifying a particular storage failure corresponding to a particular transaction.

3. The method of claim 2 , further comprising:
in response to the identifying the particular storage failure corresponding to the particular transaction, initiating recovery of one or more portions corresponding to the particular storage failure.

4. The method of claim 3 , wherein the recovery of the one or more portions comprises:
causing the at least one cloud interface device to initiate one or more read operations to retrieve a set of one or more cloud storage objects from the cloud version of the tree hierarchy; and
updating the local version of the tree hierarchy based at least in part on the set of one or more cloud storage objects retrieved, so that the local version of the tree hierarchy becomes synchronized with the cloud version of the tree hierarchy.

5. The method of claim 3 , wherein the recovery of the one or more portions comprises:
performing one or more read operations to retrieve a set of one or more blocks from the local version of the tree hierarchy; and
updating the cloud version of the tree hierarchy based at least in part on the set of one or more blocks retrieved, so that the cloud version of the tree hierarchy becomes synchronized with the local version of the tree hierarchy.

6. The method of claim 3 , wherein the identifying the particular storage failure corresponding to the particular transaction comprises:
identifying an unsynchronized state of the local version of the tree hierarchy with respect to the cloud version of the tree hierarchy based at least in part on at least one checksum stored in at least one non-leaf node of the local version and/or at least one checksum stored in at least one non-leaf node of the cloud version.

7. The method of claim 1 , further comprising:
receiving, from the application layer and through the system call interface, an additional request to perform an additional transaction with respect to the file;
translating, by the data management unit of the transactional object layer of the ZFS system, the additional request into a first I/O request to perform a read operation;
performing the read operation at least partially by:
checking the local version for at least one data block corresponding to the read operation;
attempting to validate the at least one data block retrieved from the local version; and
consequent to not successfully validating the at least one data block, attempting to perform the read operation based at least in part on requesting at least one cloud object from the cloud version.

8. A system comprising:
one or more processors communicatively coupled to memory, the one or more processors to facilitate:
receiving, from an application layer of a Z File System (ZFS) system and through a system call interface of an interface layer of the ZFS system, a request to perform a transaction with respect to a file;
based at least in part on the request, storing, via a mirror device operating at a virtual device layer of the ZFS system, data blocks and corresponding metadata in a storage pool of storage devices of the ZFS system, wherein:
the data blocks correspond to the file, and
the data blocks and the corresponding metadata are stored as logical blocks in a tree hierarchy so that a local version of the tree hierarchy is stored in the storage pool;

causing, via at least one cloud interface device of a plurality of virtual devices operating at the virtual device layer, storage of a set of cloud storage objects, the set of cloud storage objects comprising data of the data blocks and the corresponding metadata of the tree hierarchy so that a cloud version of the tree hierarchy is stored in a cloud object store;
receiving, from the application layer and through the system call interface, subsequent requests to perform transactions with respect to the file;
translating, by a data management unit of a transactional object layer of the ZFS system, the subsequent requests into I/O requests to perform I/O operations; and
upon receiving each I/O request of the I/O requests, synchronizing the local version of the tree hierarchy in the storage pool with the cloud version of the tree hierarchy in the cloud object store at least in part by:
causing, by the mirror device, a first I/O operation according to the respective I/O operation with respect to at least a portion of the local version of the tree hierarchy in the storage pool; and
causing, by the mirror device, the at least one cloud interface device to initiate a second I/O operation that mirrors the first I/O operation according to the respective I/O operation with respect to the cloud version of the tree hierarchy in the cloud object store.

9. The system of claim 8 , the one or more processors further to facilitate:
monitoring, by the mirror device and/or the at least one cloud interface device, results of each I/O request of the I/O requests for storage failures that cause the local version of the tree hierarchy and the cloud version of the tree hierarchy to become unsynchronized with one another; and
identifying a particular storage failure corresponding to a particular transaction.

10. The system of claim 9 , the one or more processors further to facilitate:
in response to the identifying the particular storage failure corres

Assignees

• Oracle Int Corp

Inventors

• Maybee Mark
• Kremer James
• Ledru Pascal
• Dunlop Kenneth

Classifications

• G06F2201/81

  Threshold · CPC title

• G06F3/0623

  in relation to content · CPC title

• G06F2212/284

  being distributed · CPC title

• G06F16/182

  Distributed file systems · CPC title

• G06F12/0804

  with main memory updating (G06F12/0806 takes precedence) · CPC title