Distributed file system

Patent metadata

Field	Value
Publication number	US-9723028-B2
Application number	US-201615193371-A
Country	US
Kind code	B2
Filing date	Jun 27, 2016
Priority date	Mar 31, 2011
Publication date	Aug 1, 2017
Grant date	Aug 1, 2017

Abstract

A distributed file system for devices is described. In an embodiment, each data element stored on one of the devices has an associated location and availability attribute. The location attribute is stored co-located with the data element. The availability attribute and a copy of the location attribute are stored by a metadata service. When a client on a device needs to access a data element, it sends a request to the metadata service to find the location of the data element. If the data element is available, this information is provided to the client and this may involve waking a dormant device which holds the data element. Where the data element is not available, read only access may be granted to a cached copy of the data element. Where replication is used and one of the devices holding a replica is unavailable, the system may use write off-loading.

First claim

The invention claimed is:

1. A method of operating a distributed data storage system, the method comprising:
storing, in a data store, a plurality of data elements, each data element having an associated location attribute identifying a location of the data element and availability attribute identifying availability of the data element, wherein the location attribute is stored in a location which is co-located with the data element and the availability attribute is stored by a metadata service in the distributed storage system;
sending, using a main processor, requests for location information to the metadata service and to send read/write requests to a remote device in the distributed storage system based on location information received from the metadata service, the requests being sent via a communication interface; and
maintaining, using a processing element, network connectivity when a device associated with the main processor is in a reduced power mode and the main processor is off and to wake the main processor on receipt of a read/write request from a remote device in the distributed storage system.

2. A method according to claim 1 , further comprising providing read only access to the plurality of data elements when communication with the metadata service is not possible.

3. A method according to claim 1 , wherein one of the plurality of data elements comprises one of a set of replicas of a data element, each replica being stored on a different device in the distributed data storage system and the replica stored in the data store being designated a primary replica, the method further comprising sending a message to the metadata service prior to entering the reduced power mode to trigger migration of the primary designation to another of the replicas of the data element.

4. A method according to claim 1 , further comprising:
writing data to a data element in the data store in response to receipt of a write request relating to the data element from a remote device;
determining if all remote devices storing replicas of the data element are available; and
sending the write request to a short-term store in the distributed storage system if all the remote devices are not available.

5. A method according to claim 1 , further comprising:
receiving a read/write request relating to a data element in the data store from a remote device
determining a user location associated with the write request;
determining if an access control policy associated with the data element is satisfied, the access control policy being defined in terms of at least one of the user location and the location attribute associated with the data element; and
servicing the read/write request if the access control policy is satisfied.

6. A method according to claim 1 , wherein the processing element is mounted within a module, the module further comprising a network interface and wherein the processing element is arranged to maintain network connectivity when the main processor is off via the network interface and to receive read/write requests via the network interface.

7. A method according to claim 1 , wherein the data store is further arranged to store a cached copy of metadata received from the metadata service, the metadata comprising the location and availability attributes associated with each data element stored in the distributed data storage system and wherein the main processor is arranged to send a request for updated metadata in response to a failure of a read/write request sent to a remote device based on location information from the cached copy of metadata.

8. A method according to claim 1 , wherein the device comprises a consumer device and wherein the distributed data storage system comprises a plurality of different consumer devices.

9. A device for use in a distributed data storage system, the device comprising:
a data store arranged to store a plurality of data elements, each data element having an associated location attribute identifying a location of the data element and availability attribute identifying availability of the data element, wherein the location attribute is stored in a location which is co-located with the data element and the availability attribute is stored by a metadata service in the distributed storage system;
a communication interface;
a main processor arranged to send requests for location information to the metadata service and to send read/write requests to a remote device in the distributed storage system based on location information received from the metadata service, the requests being sent via the communication interface; and
a processing element arranged to maintain network connectivity when the device is in a reduced power mode and the main processor is off and to wake the main processor on receipt of a read/write request from a remote device in the distributed storage system.

10. A device according to claim 9 , wherein the main processor is arranged to provide read only access to the plurality of data elements when communication with the metadata service is not possible.

11. A device according to claim 9 , wherein one of the plurality of data elements comprises one of a set of replicas of a data element, each replica being stored on a different device in the distributed data storage system and the replica stored in the data store being designated a primary replica, and wherein the main processor is further arranged to send a message to the metadata service prior to entering the reduced power mode to trigger migration of the primary designation to another of the replicas of the data element.

12. A device according to claim 9 , wherein the main processor is further arranged to:
write data to a data element in the data store in response to receipt of a write request relating to the data element from a remote device;
determine if all remote devices storing replicas of the data element are available; and
send the write request to a short-term store in the distributed storage system if all the remote devices are not available.

13. A device according to claim 9 , wherein the main processor is further arranged to:
receive a read/write request relating to a data element in the data store from a remote device
determine a user location associated with the write request;
determine if an access control policy associated with the data element is satisfied, the access control policy being defined in terms of at least one of the user location and the location attribute associated with the data element; and
service the read/write request if the access control policy is satisfied.

14. A device according to claim 9 , wherein the processing element is mounted within a module, the module further comprising a network interface and wherein the processing element is arranged to maintain network connectivity when the main processor is off via the network interface and to receive read/write requests via the network interface.

15. A device according to claim 9 , wherein the data store is further arranged to store a cached copy of metadata received from the metadata service, the metadata comprising the location and availability attributes associated with each data element stored in the distributed data storage system and wherein the main processor is arranged to send a request for updated metadata in response to a failure of a read/write request sent to a remote device based on location information from the cached copy of metadata.

16. A device according to claim 9 , which comprises a consumer device and wherein the distributed data storage system comprises a plurality of different

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Thereska Eno
• Gunawardena Dinan Srilal
• Scott James W
• Harper Richard

Classifications

• G06F17/30569

  Physics · mapped topic

• G06F17/30041

  Physics · mapped topic

• G06F17/30091

  Physics · mapped topic

• H04L63/205 Primary

  involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved (negotiation of communication capabilities H04L69/24) · CPC title

• G06F1/3203

  Power management, i.e. event-based initiation of a power-saving mode · CPC title