Local key-value database synchronization

Patent metadata

Field	Value
Publication number	US-10242084-B2
Application number	US-201715645897-A
Country	US
Kind code	B2
Filing date	Jul 10, 2017
Priority date	Oct 15, 2013
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

A remote distributed data store may be configured to process data updates received through invocation of a common API with reference to a common schema. A local data store may also be configured to process updates through the common API with reference to the common schema. Updates to the local data store may be mapped from the local data store schema to the common schema, and applied to the distributed data store. Updates to the distributed data store may be mapped from the common schema to the local data store schema. User identity may be verified to limit data synchronization to authorized users.

First claim

What is claimed is:

1. A system, comprising:
a memory to store program instructions which, if performed by at least one processor, cause the at least one processor to perform a method to at least:
receive, at an emulator, an invocation of an application programming interface (API) for a data storage service to access an item of a data set stored in a data store according to a first data schema from an application running on the at least one processor, wherein the API includes a reference to the item specified according to a second data schema for data stored in the storage service;
responsive to the invocation of the API:
translate, by the emulator, the reference to the item in the API to specify the item according to the first data schema;
access, by the emulator, the item in the data set at the data store using the translated reference to the item instead of the reference to the item included in the API; and
return, by the emulator, a result of the access to the application according to the API.

2. The system of claim 1 , wherein the access to the item updates the item, and wherein the program instructions further cause the at least one processor to perform the method to send, by the emulator, an indication of the update to the data storage service to synchronize a copy of the item stored at the data storage service.

3. The system of claim 1 , wherein the access to the item reads the item, and wherein the program instructions further cause the at least one processor to perform the method to:
prior to the return of the result, modify the result to include a prior version of the item than a version of the item stored in the data store when the invocation of the API is received in order to simulate an eventually consistent replication model for a distributed data store.

4. The system of claim 1 , wherein the program instructions further cause the at least one processor to perform the method to:
receive, at the emulator, another invocation of the API to access another item of the data set stored in the data store from the application, wherein the API includes a reference to the other item specified according to the second data schema;
responsive to the other invocation of the API:
determine, by the emulator, that a throughput rate for access to the data set from the application exceeds a provisioned throughput; and
in response to the determination that the throughput rate for the access to the data set from the application exceeds the provisioned throughput amount, return, by the emulator, a rejection response for the other invocation to the application according to the API.

5. The system of claim 1 , wherein the program instructions further cause the at least one processor to perform the method to:
prior to the return of the result, delay the return of the result in order to simulate a network latency between the application and the data storage service.

6. The system of claim 1 , wherein the program instructions further cause the at least one processor to perform the method to:
receive, at the emulator, another invocation of the API to access another item of the data set stored in the data store from the application, wherein the API includes a reference to the other item specified according to the second data schema;
responsive to the other invocation of the API:
determine, by the emulator, a data storage service error to inject into processing of the invocation of the API; and
return, by the emulator, the determined error in a response for the other invocation to the application according to the API.

7. The system of claim 1 , wherein the first data schema is a relational data schema, and wherein the second data schema is a key-value data schema.

8. A method, comprising:
receiving, at an emulator, an invocation of an application programming interface (API) for a data storage service to access an item of a data set stored in a data store according to a first data schema from an application running on the at least one processor, wherein the API includes a reference to the item specified according to a second data schema for data stored in the storage service;
responsive to the invocation of the API:
translating, by the emulator, the reference to the item in the API to specify the item according to the first data schema;
accessing, by the emulator, the item in the data set at the data store using the translated reference to the item instead of the reference to the item included in the API; and
returning, by the emulator, a result of the access to the application according to the API.

9. The method of claim 8 , wherein the access to the item updates the item, and wherein the method further comprises sending, by the emulator, an indication of the update to the data storage service to synchronize a copy of the item stored at the data storage service.

10. The method of claim 8 , wherein the access to the item reads the item, and wherein the method comprises:
prior to returning the result, modifying the result to include a prior version of the item than a version of the item stored in the data store when the invocation of the API is received in order to simulate an eventually consistent replication model for a distributed data store.

11. The method of claim 8 , further comprising:
receiving, at the emulator, another invocation of the API to access another item of the data set stored in the data store from the application, wherein the API includes a reference to the other item specified according to the second data schema;
responsive to the other invocation of the API:
determining, by the emulator, that a throughput rate for access to the data set from the application exceeds a provisioned throughput; and
in response to determining that the throughput rate for the access to the data set from the application exceeds the provisioned throughput amount, returning, by the emulator, a rejection response for the other invocation to the application according to the API.

12. The method of claim 8 , further comprising:
prior to the return of the result, delaying the return of the result in order to simulate a network latency between the application and the data storage service.

13. The method of claim 8 , further comprising:
receiving, at the emulator, another invocation of the API to access another item of the data set stored in the data store from the application, wherein the API includes a reference to the other item specified according to the second data schema;
responsive to the other invocation of the API:
determining, by the emulator, a data storage service error to inject into processing of the invocation of the API; and
returning, by the emulator, the determined error in a response for the other invocation to the application according to the API.

14. The method of claim 8 , wherein the first data schema is a relational data schema, and wherein the second data schema is a key-value data schema.

15. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices, causes the one or more computing devices to implement:
receiving, at an emulator, an invocation of an application programming interface (API) for a data storage service to access an item of a data set stored in a data store according to a first data schema from an application running on the at least one processor, wherein the API includes a reference to the item specified according to a second data schema for data stored in the storage service;
responsive to the invocation of the API:
translating, by the emulator, the reference to the item in the API

Assignees

• Amazon Tech Inc

Inventors

• Shams Khawaja Salman
• Pandey Prashant
• Sivasubramanian Swaminathan
• Zaki Omer Ahmed
• Yanacek David Craig
• Olson Johanna S
• De Kock Hendrik
• Wu Johnny Ying

Classifications

• G06F17/30578 Primary

  Physics · mapped topic

• G06F17/30569

  Physics · mapped topic

• G06F16/273 Primary

  Asynchronous replication or reconciliation · CPC title

• G06F16/23 Primary

  Updating · CPC title

• G06F16/258

  Data format conversion from or to a database · CPC title