Distributed transaction log

Patent metadata

Field	Value
Publication number	US-10769036-B2
Application number	US-201715810650-A
Country	US
Kind code	B2
Filing date	Nov 13, 2017
Priority date	Aug 26, 2013
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

Embodiments of the disclosure provide techniques for updating a distributed transaction log on a previously offline resource object component using distributed transaction logs from active host computer nodes from separate RAID mirror configurations. Each component object maintains a journal (log) where distributed transactions are recorded. If a component object goes offline and subsequently returns (e.g., if the node hosting the component object reboots), the component object is marked as stale. To return the component object to an active state, a distributed resources module retrieves the journals from other resource component objects from other RAID configurations where the data is mirrored. The module filters corresponding data that is missing in the journal of the previously offline corresponding object and merges the filtered data to the journal.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving an input/output (I/O) operation request from a virtual machine at a hypervisor of a host computer node, the I/O operation request for a corresponding virtual disk of the virtual machine, wherein the virtual disk is associated with a distributed storage system;
determining particular resource component objects of the virtual disk that are subject to the I/O operation, wherein each resource component object is associated with particular physical storage resources of the distributed storage system;
performing a plurality of distributed transactions on the particular resource component objects to carry out the I/O operation;
logging each transaction in a journal for each respective resource component object, wherein each entry includes a sequence identifier; and
reconciling the entries on different journals such that lost transactions from an offline resource component object are derived from comparing the content of the journals using the respective sequence identifiers.

2. The computer-implemented method of claim 1 , wherein the I/O request includes an identifier for an object representing the virtual disk, wherein the hypervisor uses the identifier to identify a coordinator node of the virtual disk object for carrying out the I/O request.

3. The computer-implemented method of claim 1 , wherein the particular resource component objects are hosted on one or more host computer nodes in a cluster.

4. The computer-implemented method of claim 1 , wherein active resource component objects of the particular resource component objects correspond to a mirrored redundant array of independent disks (RAID) configuration.

5. The computer-implemented method of claim 1 , further comprising:
determining that one of the particular resource component objects is stale; and
using the content of the journals to resynchronize the stale resource component object.

6. The computer-implemented method of claim 5 , wherein using the content of the journals to resynchronize the state resource component object includes determining missing transactions for the stale resource component object using the sequence identifiers and applying any corresponding transactions to the state resource component object.

7. A system comprising:
one or more computers and one or more storage devices on which are stored data that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving an input/output (I/O) operation request from a virtual machine at a hypervisor, the I/O operation request for a corresponding virtual disk of the virtual machine, wherein the virtual disk is associated with a distributed storage system;
determining particular resource component objects of the virtual disk that are subject to the I/O operation, wherein each resource component object is associated with particular physical storage resources of the distributed storage system;
performing a plurality of distributed transactions on the particular resource component objects to carry out the I/O operation;
logging each transaction in a journal for each respective resource component object, wherein each entry includes a sequence identifier; and
reconciling the entries on different journals such that lost transactions from an offline resource component object are derived from comparing the content of the journals using the respective sequence identifiers.

8. The system of claim 7 , wherein the I/O request includes an identifier for an object representing the virtual disk, wherein the hypervisor uses the identifier to identify a coordinator node of the virtual disk object for carrying out the I/O request.

9. The system of claim 7 , wherein the particular resource component objects are hosted on one or more host computer nodes in a cluster.

10. The system of claim 7 , wherein active resource component objects of the particular resource component objects correspond to a mirrored redundant array of independent disks (RAID) configuration.

11. The system of claim 7 , further comprising:
determining that one of the particular resource component objects is stale; and
using the content of the journals to resynchronize the stale resource component object.

12. The system of claim 11 , wherein using the content of the journals to resynchronize the state resource component object includes determining missing transactions for the stale resource component object using the sequence identifiers and applying any corresponding transactions to the state resource component object.

13. A non-transitory computer-readable storage medium storing instructions, which, when executed on a processor, performs operations comprising:
receiving an input/output (I/O) operation request from a virtual machine at a hypervisor, the I/O operation request for a corresponding virtual disk of the virtual machine, wherein the virtual disk is associated with a distributed storage system;
determining particular resource component objects of the virtual disk that are subject to the I/O operation, wherein each resource component object is associated with particular physical storage resources of the distributed storage system;
performing a plurality of distributed transactions on the particular resource component objects to carry out the I/O operation;
logging each transaction in a journal for each respective resource component object, wherein each entry includes a sequence identifier; and
reconciling the entries on different journals such that lost transactions from an offline resource component object are derived from comparing the content of the journals using the respective sequence identifiers.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the I/O request includes an identifier for an object representing the virtual disk, wherein the hypervisor uses the identifier to identify a coordinator node of the virtual disk object for carrying out the I/O request.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the particular resource component objects are hosted on one or more host computer nodes in a cluster.

16. The non-transitory computer-readable storage medium of claim 13 , wherein active resource component objects of the particular resource component objects correspond to a mirrored redundant array of independent disks (RAID) configuration.

17. The non-transitory computer-readable storage medium of claim 13 , further comprising:
determining that one of the particular resource component objects is stale; and
using the content of the journals to resynchronize the stale resource component object.

18. The non-transitory computer-readable storage medium of claim 17 , wherein using the content of the journals to resynchronize the state resource component object includes determining missing transactions for the stale resource component object using the sequence identifiers and applying any corresponding transactions to the state resource component object.

19. A system comprising:
one or more computers and one or more storage devices on which are stored data that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
preventing previously offline resource component objects in a distributed system from servicing operations if the resource component objects are stale, the system performing transactions without the resource component objects, the resource component ob

Assignees

• Vmware Inc

Inventors

• Earl William
• Karamanolis Christos
• Knauft Eric
• Renauld Pascal

Classifications

• G06F16/1815 Primary

  Journaling file systems · CPC title

• G06F16/1774

  Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files · CPC title

• G06F2009/45579

  I/O management, e.g. providing access to device drivers or storage · CPC title

• G06F11/1471 Primary

  involving logging of persistent data for recovery · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title