Distributed file system

Patent metadata

Field	Value
Publication number	US-10853329-B2
Application number	US-201715632389-A
Country	US
Kind code	B2
Filing date	Jun 25, 2017
Priority date	Mar 31, 2011
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

A distributed file system for devices is described. In an embodiment, each data element stored on one of the devices has an associated location and availability attribute. The location attribute is stored co-located with the data element. The availability attribute and a copy of the location attribute are stored by a metadata service. When a client on a device needs to access a data element, it sends a request to the metadata service to find the location of the data element. If the data element is available, this information is provided to the client and this may involve waking a dormant device which holds the data element. Where the data element is not available, read only access may be granted to a cached copy of the data element. Where replication is used and one of the devices holding a replica is unavailable, the system may use write off-loading.

First claim

The invention claimed is:

1. A method of operating a distributed data storage system, the method comprising:
storing, in a data store, a plurality of data elements, each data element having an associated location attribute identifying a location of the data element and availability attribute identifying availability of the data element;
sending, using a main processor, a request to send one or more read/write requests to a remote device in the distributed storage system based on location information associated with one of the plurality of data elements, the request being sent via a communication interface; and
maintaining, using a processing element, network connectivity of a device associated with the main processor when the main processor is off; and
waking the main processor on receipt of a read/write request from a remote device in the distributed storage system.

2. A method according to claim 1 , further comprising providing read only access to the plurality of data elements when communication with a metadata service is not possible.

3. A method according to claim 1 , wherein said one of the plurality of data elements comprises one of a set of replicas of a data element stored on a different device in the distributed data storage system, wherein a replica stored in the data store being designated a primary replica, and further comprising sending a message to a metadata service prior to the device entering a reduced power mode to trigger migration of the primary designation to another of the replicas of the data element.

4. A method according to claim 1 , further comprising:
writing data to a data element in the data store in response to receipt of a write request relating to the data element from a remote device;
determining remote devices storing replicas of the data element are not available; and
sending the write request to a short-term store in the distributed storage system incident to the remote devices storing the replicas not being available.

5. A method according to claim 1 , further comprising:
receiving a read/write request relating to a data element in the data store from a remote device
determining a user location associated with the write request;
determining if an access control policy associated with the data element is satisfied, the access control policy being defined in terms of at least one of the user location and the location attribute associated with the data element; and
servicing the read/write request if the access control policy is satisfied.

6. A method according to claim 1 , wherein the processing element is mounted within a module, the module further comprising a network interface and wherein the processing element is arranged to maintain network connectivity when the main processor is off via the network interface and to receive read/write requests via the network interface.

7. A method according to claim 1 , wherein the data store is further arranged to store a cached copy of metadata received from a metadata service in the distributed data storage system, the metadata comprising location attributes associated with the plurality of data elements, and wherein the main processor is arranged to send a request for updated metadata in response to a failure of a read/write request sent to a remote device based on location information from the cached copy of metadata.

8. A method according to claim 1 , wherein the device comprises a consumer device and wherein the distributed data storage system comprises a plurality of different consumer devices.

9. A device for use in a distributed data storage system, the device comprising:
a data store arranged to store a plurality of data elements, each data element having an associated location attribute identifying a location of the data element and availability attribute identifying availability of the data element;
a communication interface;
one or more processors arranged to send one or more requests for read/write requests to a remote device in the distributed storage system based on location information received associated with one of the plurality of data elements; and
a processing element arranged to maintain network connectivity when the device is in a reduced power mode and the main processor is off and to wake the main processor on receipt of a read/write request from a remote device in the distributed storage system.

10. A device according to claim 9 , wherein the main processor is arranged to provide read-only access to the plurality of data elements when communication with a metadata service is not possible.

11. A device according to claim 9 , wherein one of the plurality of data elements comprises one of a set of replicas of a data element, each replica being stored on a different device in the distributed data storage system and the replica stored in the data store being designated a primary replica, and wherein the main processor is further arranged to send a message to a metadata service prior to entering the reduced power mode to trigger migration of the primary designation to another of the replicas of the data element.

12. A device according to claim 9 , wherein the main processor is further arranged to:
write data to a data element in the data store in response to receipt of a write request relating to one of the data element from a remote device;
determine if all remote devices storing replicas of the data element are available; and
send the write request to a short-term store in the distributed storage system if all the remote devices are not available.

13. A device according to claim 9 , wherein the main processor is further arranged to:
receive a read/write request relating to a data element in the data store from a remote device
determine a user location associated with the write request;
determine if an access control policy associated with the data element is satisfied, the access control policy being defined in terms of at least one of the user location and the location attribute associated with the data element; and
service the read/write request if the access control policy is satisfied.

14. A device according to claim 9 , wherein the processing element is mounted within a module, the module further comprising a network interface and wherein the processing element is arranged to maintain network connectivity when the main processor is off via the network interface and to receive read/write requests via the network interface.

15. A device according to claim 9 , wherein the data store is further arranged to store a cached copy of metadata received from a metadata service, the metadata comprising the location and availability attributes associated with each data element stored in the distributed data storage system and wherein the main processor is arranged to send a request for updated metadata in response to a failure of a read/write request sent to a remote device based on location information from the cached copy of metadata.

16. A device according to claim 9 , which comprises a consumer device and wherein the distributed data storage system comprises a plurality of different consumer devices.

17. A system for managing a distributed data storage system, the system comprising:
memory arranged to store data elements and a location attribute associated with each data element, the location attribute indicating a geographical location of the data element; and
one or more processors;
a processing element arranged to maintain network connectivity when a in the distributed data storage system is in a reduced power mode and the main processor is off and to wake

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Thereska Eno
• Gunawardena Dinan Srilal
• Scott James W
• Harper Richard

Classifications

• G06F16/13 Primary

  File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) · CPC title

• G06F16/487

  using geographical or spatial information, e.g. location · CPC title

• G06F16/258

  Data format conversion from or to a database · CPC title

• G06F16/182

  Distributed file systems · CPC title

• H04L63/205

  involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved (negotiation of communication capabilities H04L69/24) · CPC title