Distributed storage system with web services client interface

Patent metadata

Field	Value
Publication number	US-11895188-B2
Application number	US-202217813004-A
Country	US
Kind code	B2
Filing date	Jul 15, 2022
Priority date	Dec 29, 2005
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

A distributed, web-services based storage system. A system may include a web services interface configured to receive, according to a web services protocol, a given client request for access to a given data object, the request including a key value corresponding to the object. The system may also include storage nodes configured to store replicas of the objects, where each replica is accessible via a respective unique locator value, and a keymap instance configured to store a respective keymap entry for each object. For the given object, the respective keymap entry includes the key value and each locator value corresponding to replicas of the object. A coordinator may receive the given client request from the web services interface, responsively access the keymap instance to identify locator values corresponding to the key value and, for a particular locator value, retrieve a corresponding replica from a corresponding storage node.

First claim

What is claimed is:

1. A system for providing arbitrary amounts of reliable storage accessible from any location, the system comprising:
at least one processor; and
at least one memory, wherein the at least one memory stores program instructions, and wherein executing the program instructions causes the system to:
receive web services calls from at least one client;
cause a data storage web service to store a data object in response to a web services call, wherein the web services call references a namespace corresponding to a user of the data storage web service and a key corresponding to the data object, and wherein the stored data object is addressable using the namespace and the key;
cause the data storage web service to store one or more replicas of the data object in one or more of a plurality of distributed storage nodes; and
cause an operation metrics service to store information characteristic of an actual usage of the data storage web service, and wherein the actual usage is based, at least in part, upon the web services call.

2. The system of claim 1 , wherein the web services calls comprise Representational State Transfer (REST)-style requests.

3. The system of claim 1 , wherein the web services calls are implemented according to an application programing interface (API).

4. The system of claim 1 , wherein the data object is stored in a bucket.

5. The system of claim 1 , wherein executing the program instructions causes the system to cause the operation metrics service to store information characteristic of a size of the data object or a storage utilization of the data storage web service.

6. The system of claim 1 , wherein executing the program instructions causes the system to cause the data storage web service to store the data object according to an encoding scheme.

7. The system of claim 6 , wherein the encoding scheme is a parity encoding scheme.

8. The system of claim 1 , wherein a keymap stores a relationship between the key and locators corresponding to locations of the data object and the one or more replicas of the data object.

9. A computer program product embodied on a non-transitory computer readable storage medium comprising:
computer code for receiving web services calls from at least one client;
computer code for causing a data storage web service to store a data object in response to a web services call, wherein the web services call references a namespace corresponding to a user of the data storage web service and a key corresponding to the data object, and wherein the stored data object is addressable using the namespace and the key;
computer code for causing the data storage web service to store one or more replicas of the data object in one or more of a plurality of distributed storage nodes; and
computer code for storing information characteristic of an actual usage of the data storage web service, wherein the actual usage is based, at least in part, upon the web services call.

10. The computer program product of claim 9 , wherein the web services calls comprise Representational State Transfer (REST)-style requests.

11. The computer program product of claim 9 , wherein the web services calls are implemented according to an application programing interface (API), and wherein an operation metrics service stores the information characteristic of the actual usage of the data storage web service.

12. The computer program product of claim 9 , wherein the data object is stored in a bucket.

13. The computer program product of claim 9 , wherein an operation metrics service stores information characteristic of a size of the data object or a storage utilization of the data storage web service.

14. The computer program product of claim 9 , wherein the data object is stored according to an encoding scheme.

15. The computer program product of claim 14 , wherein the encoding scheme is a parity encoding scheme.

16. The computer program product of claim 9 , wherein a keymap stores a relationship between the key and locators corresponding to locations of the data object and the one or more replicas of the data object.

17. A system for providing arbitrary amounts of reliable storage accessible from any location, the system comprising:
means for receiving web services calls from at least one client;
means for storing a data object in response to a web services call, wherein the web services call references a namespace corresponding to a user of the data storage web service and a key corresponding to the data object, and wherein the stored data object is addressable using the namespace and the key;
means for storing one or more replicas of the data object in one or more of a plurality of distributed storage nodes; and
means for storing information characteristic of an actual usage of the data storage web service, wherein the actual usage is based, at least in part, upon the web services call.

18. The system of claim 17 , wherein the web services calls comprise Representational State Transfer (REST)-style requests.

19. The system of claim 17 , wherein the web services calls are implemented according to an application programing interface (API), and wherein an operation metrics service stores the information characteristic of the actual usage of the data storage web service.

20. The system of claim 17 , wherein the data object is stored in a bucket.

Assignees

• Amazon Tech Inc

Inventors

• Vermeulen Allan Henry
• Atlas Alan B
• Barth David M
• Cormie John David
• Fischman Ami K
• Sorenson Iii James Christopher
• Wagner Eric M

Classifications

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/184 Primary

  implemented as replicated file system · CPC title

• G06F9/547

  Remote procedure calls [RPC]; Web services · CPC title

• H04L67/00

  Network arrangements or protocols for supporting network services or applications (user-to-user messaging H04L51/00; network arrangements, protocols or services for supporting real-time applications in data packet communications networks H04L65/00) · CPC title

• H04L67/01

  Protocols · CPC title