Collaboration data proxy system in cloud computing platforms

Patent metadata

Field	Value
Publication number	US-9906597-B2
Application number	US-201514680757-A
Country	US
Kind code	B2
Filing date	Apr 7, 2015
Priority date	Apr 7, 2015
Publication date	Feb 27, 2018
Grant date	Feb 27, 2018

Abstract

In various embodiments, methods and systems for enhanced access to storage data based on a collaboration data proxy system are provided. A plurality of metadata tables on one or more peer nodes are referenced for data corresponding to a data request of a requesting node. The metadata tables indicate availability of chunks of data in the one or more peer nodes. A determination is made that the data corresponding to the data request is downloadable from the one or more node; the determination is based on the metadata tables. A download operation configuration instance is generated for a data request of a requesting node. The download operation configuration instance comprises configuration settings for downloading data corresponding to the data request from the one or more peer nodes. The chunk of data is downloaded from the corresponding one or more peer nodes where the chunk is located, using the configuration settings.

First claim

The invention claimed is:

1. A system for providing enhanced access to data in distributed storage systems, the system comprising:
one or more hardware processors;
memory configured for providing computer program instructions to the one or more hardware processors;
a collaboration data proxy component configured to execute on the one or more hardware processors for:
determining, using a metadata table stored at a requesting node, whether data corresponding to a data request of the requesting node is downloadable from one or more peer nodes of the requesting node, wherein each node in a peer group comprising the requesting node and the one or more peer nodes includes a metadata table that indicates availability of chunks of data in the peer group; and
downloading a chunk of the data from the one or more peer nodes, when the chunk is located on the one or more peer nodes, wherein downloading the data is based on a download operation; and

a long-tail mitigation component configured to execute on the one or more hardware processors for:
adjusting the download operation based on a long tail mitigation routine to facilitate downloading the chunk of data by using at least one of: a contention avoidance workflow or an increased download throughput workflow;

wherein the contention avoidance workflow comprises avoiding contention during the download operation based on a chunk selection scheme, wherein the chunk selection scheme comprises prioritizing downloading chunks having relatively fewer peer nodes downloading a selected chunk; and
wherein the increased download throughput workflow comprises a plurality of phases, each phase corresponding to a defined percentage of data chunks of the data downloaded, and each phase defining a different adjustment to the download operation.

2. The system of claim 1 , further comprising the collaboration data proxy component configured for:
determining whether the data request from the requesting node is directed to collaboration data, wherein collaboration data comprises data that is stored on at least one of: a cloud computing platform storage or one or more peer nodes;
forwarding the data request to a data source corresponding to the data request, when the request is not directed to collaboration data; and
processing the data request based on availability of the data on the cloud computing platform or the one or more peers nodes, when the request is directed to collaboration data.

3. The system of claim 1 , further comprising a metadata component configured to execute on the one or more hardware processors for:
initializing a selected node by retrieving metadata tables from the nodes in the peer group when the selected node joins the peer group;
detecting pre-cached chunks in the selected node to populate metadata entries in the metadata tables;
communicating updates in the metadata tables to the nodes in the peer group, wherein the updates comprise one of: an insertion change and a deletion change; and
maintaining the metadata tables based on maintenance triggers comprising negative acknowledgement communications.

4. The system of claim 1 , wherein the plurality of phases include a head phase having a defined head phase percent of data chunks of the data downloaded, wherein during the head phase the long-tail mitigation component is configured for:
upon determining that a node receiving the data is being throttled at a cloud computing platform storage, throttling a number of network calls for downloading from the cloud computing platform storage and triggering network calls for downloading from one or more peer nodes; and
upon determining that no throttling is occurring at the cloud computing platform storage, increasing a number of network calls to the cloud computing platform storage to download.

5. The system of claim 1 , wherein the plurality of phases include a body phase having a predefined body phase percent of data chunks of the data downloaded, wherein during the body phase the long-tail mitigation component is configured for:
determining that the predefined body phase percent of data chunks downloaded has been met;
reducing a number of network calls to the cloud computing platform storage; and
increasing a number of network calls to the one or more peer nodes.

6. The system of claim 1 , wherein the plurality of phases include a tail phase having a predefined body phase percent of data chunks of the data downloaded, wherein during the tail phase the long-tail mitigation component is configured for:
selecting a number of tail phase peer nodes in a tail phase peer node group, wherein the tail phase peer nodes are selected for downloading a selected tail phase chunk;
communicating a request for the selected tail phase chunk to each node in the tail phase node group; and
processing a chunk received earliest from one of the peer nodes in the tail phase node group.

7. The system of claim 1 , further comprising a flexibility configuration component configured to execute on the one or more hardware processors for:
generating a download operation configuration instance for the data request of the requesting node, wherein the download operation instance comprises configuration settings of the download operation for downloading data corresponding to the data request from the one or more peer nodes.

8. The system of claim 7 , wherein the download operation configuration instance comprises a plurality of configuration settings programmatically defined using template elements as constraints for performing the download operation.

9. One or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, causes the one or more processors to perform a method for enhanced access to data in distributed storage systems, the method comprising:
referencing metadata tables corresponding to a peer group comprising peer nodes for data corresponding to a data request, wherein each peer node in the peer group includes a metadata table that indicates availability of chunks of data in the peer group;
determining that the data corresponding to data request is downloadable from one or more of the peer nodes, wherein the determination is based on the metadata tables;
prioritizing downloading chunks of the data having relatively fewer peer nodes downloading a selected chunk;
generating a download operation configuration instance for the data request, wherein the download operation instance comprises configuration settings for downloading data corresponding to the data request; and
downloading a chunk of the data from the one or more peer nodes using the configuration settings of the download operation configuration instance.

10. The media of claim 9 , wherein metadata tables include each of a globally sorted chunk list, an insertion table, and a deletion table.

11. The media of claim 9 , wherein the download operation configuration instance comprises a plurality of configuration settings programmatically defined using template elements as constraints for performing the download operation.

12. The media of claim 11 , wherein an in-memory-cache-mode template element determines which chunks of data are cached in memory, and wherein an in-memory-cache-mode is selected from one of the following: a Least Recently Used mode, a Least Frequently Used mode, a First In First Out scheme, a Random Replacement scheme, a Most Globally Available Scheme.

13. The media of claim 9 , wherein the download operation configuration instance is associated with an account such that the configuration settings are identified based on cloud computing platform data access ser

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Peelen Nicolaas Deodorus
• Hui Wang
• Tang Jun
• Srinivasan Sridhar
• Xu Mingqiang
• Huang Yan

Classifications

• H04L67/06

  specially adapted for file transfer, e.g. file transfer protocol [FTP] · CPC title

• H04L67/28

  Electricity · mapped topic

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L67/104 Primary

  Peer-to-peer [P2P] networks · CPC title

• H04L47/12

  Avoiding congestion; Recovering from congestion · CPC title