Method and system for implementing performance tier de-duplication in a virtualization environment

Patent metadata

Field	Value
Publication number	US-9342253-B1
Application number	US-201414466782-A
Country	US
Kind code	B1
Filing date	Aug 22, 2014
Priority date	Aug 23, 2013
Publication date	May 17, 2016
Grant date	May 17, 2016

Abstract

The present application provides an improved approach for managing performance tier de-duplication in a virtualization environment. A content cache is implemented on high performance tiers of storage in order to maintain a working set for the user virtual machines accessing the system, and associates fingerprints with data stored therein. During write requests from the user virtual machines, fingerprints are calculated for the data to be written. However, no de-duplication is performed during the write. During read requests, fingerprints corresponding to the data to be read are retrieved and matched with the fingerprints associated with the data in the content cache. Thus, while multiple pieces of data having the same fingerprints may be written to the lower performance tiers of storage, only one of those pieces of data having that fingerprint will be stored in the content cache for fulfilling read requests.

First claim

What is claimed is:

1. A computer-implemented method for performing data de-duplication for one or more virtual machines in a virtualization environment, comprising:
configuring the virtualization environment to perform de-duplication;
in response to one or more write requests from the one or more virtual machines to write data to one or more virtual disks:
generating one or more fingerprints corresponding to the data to be written; and
writing the data to one or more persistent storage devices associated with the one or more virtual disks,
wherein the data is written regardless of whether or not the one or more storage devices already contains data corresponding to fingerprints matching the one or more generated fingerprints and whether the virtualization environment is configured to perform de-deduplication, such that data on the one or more storage devices is not de-duplicated in response to or near a time of the one or more write requests;
in response to a read request from the one or more virtual machines to read at least a portion of the data:
identifying fingerprints corresponding to the data to be read;
accessing a content cache that stores a cached set of data for the one or more virtual machines and fingerprints corresponding to the cached set of data, wherein the content cache is configured such that it does not store more than one copy of data corresponding to a particular fingerprint;
determining if any of the identified fingerprints corresponding to the requested portion of data match any of the fingerprints corresponding to the cached set of data; and
if a match is found, retrieving data corresponding to the matched fingerprints from the cached set of data; and
at a later time, performing de-duplication on the one or more storage devices, such that that the one or more storage devices do not store more than a specified number of copies of data corresponding to a particular fingerprint, wherein the specified number is associated with a replication factor, and wherein the de-duplication is not performed in response to a particular write request.

2. The computer-implemented method of claim 1 , wherein the fingerprints comprise SHA1 fingerprints.

3. The computer-implemented method of claim 1 , wherein the generated fingerprints are associated with locations on the one or more virtual disks.

4. The computer-implemented method of claim 1 , wherein the content cache is implemented in RAM or flash.

5. The computer-implemented method of claim 1 , wherein the content cache associates fingerprints with working set of data using a hash table.

6. The computer-implemented method of claim 1 , wherein the content cache comprises one or more LRU chains.

7. The computer-implemented method of claim 6 , wherein the LRU chains comprise a single-touch LRU chain and a multi-touch LRU chain.

8. The computer-implemented method of claim 1 , further comprising:
reading the requested portion of data from the one or more persistent storage devices if there are no matching fingerprints are found;
writing the requested portion of data into the content cache.

9. A system for performance tier de-duplication for one or more virtual machines in a virtualization environment, comprising:
a processor;
a memory comprising computer code executed using the processor, in which the computer code implements:
configuring the virtualization environment to perform de-duplication;
in response to one or more write requests from the one or more virtual machines to write data to one or more virtual disks:
generating one or more fingerprints corresponding to the data to be written; and
writing the data to one or more persistent storage devices associated with the one or more virtual disks,
wherein the data is written regardless of whether or not the one or more storage devices already contains data corresponding to fingerprints matching the one or more generated fingerprints and whether the virtualization environment is configured to perform de-deduplication, such that data on the one or more storage devices is not de-duplicated in response to or near a time of the one or more write requests;
in response to a read request from the one or more virtual machines to read at least a portion of the data:
identifying fingerprints corresponding to the data to be read;
accessing a content cache that stores a cached set of data for the one or more virtual machines and fingerprints corresponding to the cached set of data, wherein the content cache is configured such that it does not store more than one copy of data corresponding to a particular fingerprint;
determining if any of the identified fingerprints corresponding to the requested portion of data match any of the fingerprints corresponding to the cached set of data; and
if a match is found, retrieving data corresponding to the matched fingerprints from the cached set of data; and
at a later time, performing de-duplication on the one or more storage devices, such that that the one or more storage devices do not store more than a specified number of copies of data corresponding to a particular fingerprint, wherein the specified number is associated with a replication factor, and wherein the de-duplication is not performed in response to a particular write request.

10. The system of claim 9 , wherein the fingerprints comprise SHA1 fingerprints.

11. The system of claim 9 , wherein the generated fingerprints are associated with locations on the one or more virtual disks.

12. The system of claim 9 , wherein the content cache is implemented in RAM or flash.

13. The system of claim 9 , wherein the content cache associates fingerprints with working set of data using a hash table.

14. The system of claim 9 , wherein the content cache comprises one or more LRU chains.

15. The system of claim 14 , wherein the LRU chains comprise a single-touch LRU chain and a multi-touch LRU chain.

16. The system of claim 9 , further comprising:
reading the requested portion of data from the one or more persistent storage devices if there are no matching fingerprints are found;
writing the requested portion of data into the content cache.

17. A computer program product embodied on a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for performance tier de-duplication for one or more virtual machines in a virtualization environment, the method comprising:
configuring the virtualization environment to perform de-duplication;
in response to one or more write requests from the one or more virtual machines to write data to one or more virtual disks:
generating one or more fingerprints corresponding to the data to be written; and
writing the data to one or more persistent storage devices associated with the one or more virtual disks,
wherein the data is written regardless of whether or not the one or more storage devices already contains data corresponding to fingerprints matching the one or more generated fingerprints and whether the virtualization environment is configured to perform de-deduplication, such that data on the one or more storage devices is not de-duplicated in response to or near a time of the one or more write requests;
in response to a read request from the one or more virtual machines to read at least a portion of the data:
identifying fingerprints corresponding to the data to be read;
accessing a content cache that stores a cached set of data for the one or more virtual machines and fingerprints

Assignees

• Nutanix Inc

Inventors

• Muthukkaruppan Kannan
• Ranganathan Karthik

Classifications

• G06F3/0641 Primary

  De-duplication techniques · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/0688

  Non-volatile semiconductor memory arrays · CPC title

• G06F2003/0694

  Physics · mapped topic

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title