Asynchronous input and output for snapshots of virtual machines

Patent metadata

Field	Value
Publication number	US-11429418-B2
Application number	US-201916527470-A
Country	US
Kind code	B2
Filing date	Jul 31, 2019
Priority date	Jul 31, 2019
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

A data management system having a storage appliance configured to store a snapshot of a virtual machine; and one or more processors in communication with the storage appliance. The one or more processors are configured to perform operations including: identifying a plurality of shards of the virtual machine; requesting a shard snapshot of each of the plurality of shards; receiving the shard snapshots asynchronously; ordering the received shard snapshots sequentially into a results queue; and storing a single snapshot of the virtual machine based on the ordered shard snapshots. The operations may further include maintaining a flow control queue that limits a number of the requested shard snapshots.

First claim

The invention claimed is:

1. A computer-implemented method at a data management system, the method comprising:
sharding an entire virtual machine into a plurality of shards, wherein each of the shards corresponds to a different portion of the entire virtual machine;
transmitting separate requests for shard snapshots corresponding to at least a first subset of shards of the plurality of shards;
receiving, in response to the separate requests for the shard snapshots, the requested shard snapshots out of order;
maintaining an offset-slot mapping indicating ordering of the requested shard snapshots, wherein the offset-slot mapping uses disk addresses of the entire virtual machine to indicate the ordering of the requested shard snapshots;
storing a shard snapshot that is not maintained in the offset-slot mapping in an early received map, wherein the early received map is configured to hold shard snapshot data until the ordering indicated by the offset-slot mapping is updated based at least in part on a receive queue;
ordering the received out of order shard snapshots and the shard snapshot that is stored in the early received map in order into a results queue using the offset-slot mapping that is maintained for the requested shard snapshots and updated to include the shard snapshot based at least in part on the receive queue; and
storing a single snapshot of the entire virtual machine in a computer memory based at least in part on the ordered shard snapshots.

2. The method of claim 1 , further comprising:
maintaining a flow control queue that limits a number of the requested shard snapshots, wherein the flow control queue stores a set of tokens that limit the number of the requested shard snapshots and receives a token from the receive queue upon storage of a shard snapshot in the results queue.

3. The method of claim 2 , further comprising:
maintaining the receive queue and transferring a token from the flow control queue to the receive queue upon receiving one of the requested shard snapshots of the plurality of shards.

4. The method of claim 3 , further comprising:
updating the maintained offset-slot mapping with ordering of the shard snapshot not in the offset-slot mapping; and
moving the shard snapshot not in the offset-slot mapping into the results queue after the updating.

5. The method of claim 1 , further comprising presenting the ordered shard snapshots in order to a read Application Programming Interface for storage in the computer memory.

6. The method of claim 1 , further comprising enforcing a Secure Sockets Layer in the receiving the requested out of order shard snapshots.

7. A non-transitory, machine-readable medium storing instructions which, when read by a machine, cause the machine to perform operations comprising, at least:
sharding an entire virtual machine into a plurality of shards, wherein each of the shards corresponds to a different portion of the entire virtual machine;
transmitting separate requests for shard snapshots corresponding to at least a first subset of shards of the plurality of shards;
receiving, in response to the separate requests for the shard snapshots, the requested shard snapshots out of order;
maintaining an offset-slot mapping indicating ordering of the requested shard snapshots, wherein the offset-slot mapping uses disk addresses of the entire virtual machine to indicate the ordering of the requested shard snapshots;
storing a shard snapshot that is not maintained in the offset-slot mapping in an early received map, wherein the early received map is configured to hold shard snapshot data until the ordering indicated by the offset-slot mapping is updated based at least in part on a receive queue;
ordering the received out of order shard snapshots and the shard snapshot that is stored in the early received map in order into a results queue using the offset-slot mapping that is maintained for the requested shard snapshots and updated to include the shard snapshot based at least in part on the receive queue; and
storing a single snapshot of the entire virtual machine in a computer memory based at least in part on the ordered shard snapshots.

8. The medium of claim 7 , wherein the operations further include:
maintaining a flow control queue that limits a number of the requested shard snapshots, wherein the flow control queue stores a set of tokens that limit the number of the requested shard snapshots and receives a token from the receive queue upon storage of a shard snapshot in the results queue.

9. The medium of claim 8 , wherein the operations further include:
maintaining the receive queue and transferring a token from the flow control queue to the receive queue upon receiving one of the requested shard snapshots of the plurality of shards.

10. The medium of claim 9 , wherein the operations further include:
updating the maintained offset-slot mapping with ordering of the shard snapshot not in the offset-slot mapping; and
moving the shard snapshot not in the offset-slot mapping into the results queue after the updating.

11. The medium of claim 7 , wherein the operations further include presenting the ordered shard snapshots in order to a read Application Programming Interface for storage in the computer memory.

12. The medium of claim 7 , wherein the operations further include enforcing a Secure Sockets Layer in the receiving the requested out of order shard snapshots.

13. A data management system, comprising:
a computer memory configured to store a snapshot of an entire virtual machine;
one or more processors in communication with the computer memory, the one or more processors configured to perform operations including:
sharding the entire virtual machine into a plurality of shards, wherein each of the shards corresponds to a different portion of the entire virtual machine;
transmitting separate requests for shard snapshots corresponding to at least a first subset of shards of the plurality of shards;
receiving, in response to the separate requests for the shard snapshots, the requested shard snapshots out of order;
maintaining an offset-slot mapping indicating ordering of the requested shard snapshots, wherein the offset-slot mapping uses disk addresses of the entire virtual machine to indicate the ordering of the requested shard snapshots;
storing a shard snapshot that is not maintained in the offset-slot mapping in an early received map, wherein the early received map is configured to hold shard snapshot data until the ordering indicated by the offset-slot mapping is updated based at least in part on a receive queue;
ordering the received out of order shard snapshots and the shard snapshot that is stored in the early received map in order into a results queue using the offset-slot mapping that is maintained for the requested shard snapshots and updated to include the shard snapshot based at least in part on the receive queue; and
storing a single snapshot of the entire virtual machine in the computer memory based at least in part on the ordered shard snapshots.

14. The system of claim 13 , wherein the operations further include:
maintaining a flow control queue that limits a number of the requested shard snapshots, wherein the flow control queue stores a set of tokens that limit the number of the requested shard snapshots and receives a token from the receive queue upon storage of a shard snapshot in the results queue.

15. The system of claim 14 , wherein the operations further include:
maintaining the receive queue and transferring a token from the flow control queue to the receive queue upon receiving one of the requested shard

Assignees

• Rubrik Inc

Inventors

• Denny Christopher
• Ding Li
• Yu Linglin
• Chu Stephen
• Wu Ying

Classifications

• G06F11/1464

  for networked environments · CPC title

• G06F3/0664

  at device level, e.g. emulation of a storage device or system · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F11/1469

  Backup restoration techniques · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title