Technique selection in a deduplication aware client environment

Patent metadata

Field	Value
Publication number	US-2016042008-A1
Application number	US-201414453173-A
Country	US
Kind code	A1
Filing date	Aug 6, 2014
Priority date	Aug 6, 2014
Publication date	Feb 11, 2016
Grant date	—

Abstract

Techniques and mechanisms described herein facilitate the transmission of a data stream to a networked storage system. According to various embodiments, a determination may be made as to whether an amount of available computing resources at a client device meets or exceeds a computing resource availability threshold at the client device. A processing operation on a data stream may be performed at the client device to produce a pre-processed data stream when the amount of available computing resources meets or exceeds the computing resource availability threshold. The pre-processed data stream may be transmitted to a networked storage system for storage via a network. The networked storage system may be operable to store deduplicated data for retrieval via the network.

First claim

What is claimed is:

1 . A method comprising:
determining whether an amount of available computing resources at a client device comprising a processor and memory meets or exceeds a computing resource availability threshold at the client device; performing a processing operation on a data stream at the client device to produce a pre-processed data stream when it is determined that the amount of available computing resources meets or exceeds the computing resource availability threshold; and transmitting the pre-processed data stream for storage to a networked storage system via a network, the networked storage system operable to store deduplicated data for retrieval via the network.

2 . The method recited in claim 1 , wherein the processing operation comprises:
deduplicating the data stream.

3 . The method recited in claim 1 , wherein the processing operation comprises compressing the data stream.

4 . The method recited in claim 1 , wherein the processing operation comprises encrypting the data stream.

5 . The method recited in claim 1 , wherein the processing operation comprises:
parsing the data stream to identify one or more data chunks, and for one or more of the identified data chunks, performing an operation selected from the group consisting of: deduplicating the identified data chunk, compressing the identified data chunk, and encrypting the identified data chunk.

6 . The method recited in claim 5 , wherein the one or more data chunks are identified via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams.

7 . The method recited in claim 5 , wherein the processing operation further comprises:
determining a respective fingerprint for each identified data chunk by applying a hash function to the identified data chunk.

8 . The method recited in claim 7 , wherein the processing operation further comprises:
determining whether each identified data chunk is stored at a networked storage system by transmitting each fingerprint to the networked storage system via the network.

9 . The method recited in claim 8 , wherein determining whether the identified chunk is stored at the networked storage system further comprises:
receiving a fingerprint status message from the networked storage system, the fingerprint status message indicating whether the chunk associated with the fingerprint is stored at the networked storage system.

10 . The method recited in claim 5 , wherein the processing operation further comprises:
transmitting a block map update request message to the networked storage system, the block map update request message including information for updating a block map at the networked storage system, the block map identifying a designated memory location at which each data encrypted chunk is stored at the networked storage system.

11 . The method recited in claim 1 , the method further comprising:
identifying an amount of available computing resources at the networked storage system.

12 . The method recited in claim 1 , wherein the data stream is generated at the client device via a network storage protocol selected from the group consisting of: the Network File System (NFS) protocol, the Common Internet File System (CIFS) protocol, and the Open Storage (OST) protocol.

13 . A device comprising:
memory operable to store a data stream; a processor operable to determine whether an amount of available computing resources at a client device meets or exceeds a computing resource availability threshold at the client device and perform a processing operation on the data stream to produce a pre-processed data stream when it is determined that the amount of available computing resources meets or exceeds the computing resource availability threshold; and a communications interface operable to transmit the pre-processed data stream for storage to a networked storage system via a network, the networked storage system operable to store deduplicated data for retrieval via the network.

14 . The device recited in claim 13 , wherein the processing operation is an operation selected from the group consisting of: deduplicating the data stream, encrypting the data stream, and compressing the data stream.

15 . The device recited in claim 13 , wherein the processing operation comprises:
parsing the data stream to identify one or more data chunks.

16 . The device recited in claim 15 , wherein the one or more data chunks are identified via a rolling hash parsing technique operable to generate at least some identical chunks when parsing different but overlapping data streams.

17 . The device recited in claim 15 , wherein the processing operation further comprises:
determining a respective fingerprint for each identified data chunk by applying a hash function to the identified data chunk.

18 . The device recited in claim 17 , wherein the processing operation further comprises:
determining whether each identified data chunk is stored at a networked storage system by transmitting each fingerprint to the networked storage system via the network.

19 . The device recited in claim 18 , wherein determining whether the identified chunk is stored at the networked storage system further comprises:
receiving a fingerprint status message from the networked storage system, the fingerprint status message indicating whether the chunk associated with the fingerprint is stored at the networked storage system.

20 . One or more computer readable media having instructions stored thereon for performing a method, the method comprising:
determining whether an amount of available computing resources at a client device comprising a processor and memory meets or exceeds a computing resource availability threshold at the client device; performing a processing operation on a data stream at the client device to produce a pre-processed data stream when it is determined that the amount of available computing resources meets or exceeds the computing resource availability threshold; and transmitting the pre-processed data stream for storage to a networked storage system via a network, the networked storage system operable to store deduplicated data for retrieval via the network.

Assignees

• Dell Products Lp

Inventors

• Tripathy Tarun K
• Smith Brian R
• Dinkar Abhijit S

Classifications

• G06F3/06

  Digital input from, or digital output to, record carriers {, e.g. RAID, emulated record carriers or networked record carriers} · CPC title

• G06F11/3017

  where the computing system is implementing multitasking (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L65/80

  Responding to QoS · CPC title

• G06F11/1453

  using de-duplication of the data · CPC title