Log-structured storage device format

Patent metadata

Field	Value
Publication number	US-10402374-B2
Application number	US-201414469418-A
Country	US
Kind code	B2
Filing date	Aug 26, 2014
Priority date	Aug 26, 2013
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

Embodiments of the disclosure provide techniques managing a log-structured solid state drive (SSD) format in a distributed storage system. SSDs in the distributed storage system maintains a journal of logical changes to storage objects to persist prepared and committed changes in the latency path. The journal includes metadata entries that describe changes and reference data pages. Dense data structures (such as a logical block addressing table) index the metadata entries. To reduce the amount of overhead in I/O operations, the distributed storage system maintains the dense data structures in memory rather than on disk.

First claim

We claim:

1. A method for populating journal entries in a non-volatile storage device of a distributed resources system comprising a plurality of host computer nodes including a first host computer node and a second host computer node, wherein the non-volatile storage device backs one or more resource objects of the first host computer node and resides in the second host computer node and another non-volatile storage device backs the one or more resource objects of the first host computer node and resides in a third host computer node,
the method comprising:
upon an operation occurring on a block of the other non-volatile storage device in the third host computer node, inserting a current entry corresponding to the operation into a journal,
wherein the journal is stored persistently in the non-volatile storage device and maintains entries reflecting logical changes to the one or more resource objects of the first host computer node which are backed by the non-volatile storage device in the second host computer node and the other non-volatile storage device in the third host computer node, and
wherein each entry includes sequence data indicating a sequence of the entry in the journal and a metadata record specifying the logical changes to the one or more resource objects of the first host computer node associated with the operation; and
upon determining a type of the metadata record of the current entry, inserting the metadata record into a datastore corresponding to the journal and the type, wherein the datastore resides in memory of the first host computer node.

2. The method of claim 1 , further comprising, upon determining that the current entry corresponds to an overwritten block on the other non-volatile storage device, removing a previous metadata record corresponding to the overwritten block from the datastore.

3. The method of claim 1 , wherein the metadata record describes a change to one or more of the logical changes to the one or more resource objects.

4. The method of claim 1 , wherein the distributed resources system is a software-based virtual storage area network.

5. The method of claim 1 , wherein the type corresponds to a distributed transaction phase of the entry.

6. The method of claim 1 , further comprising:
upon detecting that the first host computer node has restarted, iterating through records of each of the plurality of entries;
determining the type of each record; and
inserting each record into the datastore corresponding to the type of each record.

7. The method of claim 6 , further comprising, while inserting each record into the datastore, upon determining that a current record corresponds to an overwritten block, removing a previous record corresponding to the overwritten block from the datastore.

8. A non-transitory computer readable storage medium storing instructions, which, when executed on a processor, performs a method for populating journal entries in a non-volatile storage device of a distributed resources system comprising a plurality of host computer nodes including a first host computer node and a second host computer node,
wherein the non-volatile storage device backs one or more resource objects of the first host computer node and resides in the second host computer node and another non-volatile storage device backs the one or more resource objects of the first host computer node and resides in a third computer node,
the method comprising:
upon an operation occurring on a block of the other non-volatile storage device in the third host computer node, inserting a current entry corresponding to the operation into a journal,
wherein the journal is stored persistently in the non-volatile storage device and maintains entries reflecting logical changes to the one or more resource objects of the first host computer node which are backed by the non-volatile storage device in the second host computer node and the other non-volatile storage device in the third host computer node, and
wherein each entry includes sequence data indicating a sequence of the entry in the journal and a metadata record specifying the logical changes to the one or more resource objects of the first host computer node associated with the operation; and
upon determining a type of the metadata record of the current entry, inserting the metadata record into a datastore corresponding to the journal and the type, wherein the datastore resides in memory of the first host computer node.

9. The computer-readable storage medium of claim 8 , wherein the method further comprises, upon determining that the current entry corresponds to an overwritten block on the other non-volatile storage device, removing a previous metadata record corresponding to the overwritten block from the datastore.

10. The computer-readable storage medium of claim 8 , wherein the metadata record describes a change to one or more of the logical changes to the one or more resource objects.

11. The computer-readable storage medium of claim 8 , wherein the distributed resources system is a software-based virtual storage area network.

12. The computer-readable storage medium of claim 8 , wherein the type corresponds to a distributed transaction phase of the entry.

13. The computer-readable storage medium of claim 8 , wherein the method further comprises:
upon detecting that the first host computer node has restarted, iterating through records of each of the plurality of entries;
determining the type of each record; and
inserting each record into the datastore corresponding to the type of each record.

14. The computer-readable storage medium of claim 13 , wherein the method further comprises, while inserting each record into the datastore, upon determining that a current record corresponds to an overwritten block, removing a previous record corresponding to the overwritten block from the datastore.

15. A system, comprising:
a processor; and
a memory hosting an application, which, when executed on the processor, performs a method for populating journal entries in a non-volatile storage device of a distributed resources system comprising a plurality of host computer nodes including a first host computer node and a second host computer node, wherein the non-volatile storage device backs one or more resource objects of the first host computer node and resides in the second host computer node and another non-volatile storage device backs the one or more resource objects of the first host computer node and resides in a third host computer node,
the method comprising:
upon an operation occurring on a block of the other non-volatile storage device in the third host computer node, inserting a current entry corresponding to the operation into a journal,
wherein the journal is stored persistently in the non-volatile storage device and maintains entries reflecting logical changes to the one or more resource objects of the first host computer node which are backed by the non-volatile storage device in the second host computer node and the other non-volatile storage device in the third host computer node, and
wherein each entry includes sequence data indicating a sequence of the entry in the journal and a metadata record specifying the logical changes to the one or more resource objects of the first host computer node associated with the operation; and
upon determining a type of the metadata record of the current entry, inserting the metadata record into a datastore corresponding to the journal and the type, wherein the datastore resides in memory of the first host computer node.

16. The system of claim 15 , wherein the metho

Assignees

• Vmware Inc

Inventors

• Earl William
• Karamanolis Christos
• Joshi Kiran

Classifications

• G06F16/1734 Primary

  Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs · CPC title

• G06F11/1471

  involving logging of persistent data for recovery · CPC title