Multi-layered key-value storage

Patent metadata

Field	Value
Publication number	US-11507555-B2
Application number	US-202017067629-A
Country	US
Kind code	B2
Filing date	Oct 9, 2020
Priority date	Oct 13, 2019
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

Systems and methods for multi-layered key-value storage are described. For example, methods may include receiving two or more put requests that each include a respective primary key and a corresponding respective value; storing the two or more put requests in a buffer in a first datastore; determining whether the buffer is storing put requests that collectively exceed a threshold; responsive to the determination that the threshold has been exceeded, transmitting a write request to a second datastore, including a subsidiary key and a corresponding data file that includes the respective values of the two or more put requests at respective offsets in the data file; for the two or more put requests, storing respective entries in an index in the first datastore that associate the respective primary keys with the subsidiary key and the respective offsets; and deleting the two or more put requests from the buffer.

First claim

What is claimed is:

1. A method comprising:
receiving a first put request, wherein the first put request includes a first primary key and a corresponding first value;
receiving a second put request, wherein the second put request includes a second primary key and a corresponding second value;
storing the first put request in a buffer in a first datastore;
storing the second put request in the buffer in the first datastore;
transmitting a write request to a second datastore, wherein the write request includes a subsidiary key and a corresponding data file, and wherein the data file includes the first value starting at a first offset and the second value starting at a second offset;
storing a first entry in an index in the first datastore, wherein the first entry includes the first primary key along with the subsidiary key and the first offset;
storing a second entry in the index in the first datastore, wherein the second entry includes the second primary key along with the subsidiary key and the second offset; and
deleting the first put request and the second put request from the buffer.

2. The method of claim 1 , comprising:
receiving a get request, wherein the get request includes the first primary key;
based on the first primary key, accessing the first entry in the index in the first datastore to retrieve the subsidiary key and the first offset;
transmitting a read request to the second datastore, wherein the read request includes the subsidiary key and the first offset;
responsive to the read request, receiving the first value from the second datastore; and
responsive to the get request, transmitting the first value.

3. The method of claim 1 , comprising:
receiving a get request, wherein the get request includes the first primary key;
determining whether a put request including the first primary key is currently stored in the buffer in the first datastore;
responsive to a determination that a put request including the first primary key is currently stored in the buffer in the first datastore, accessing the first put request to retrieve the first value; and
responsive to the get request, transmitting the first value.

4. The method of claim 1 , comprising:
determining whether the buffer in the first datastore is storing put requests that collectively exceed a threshold, wherein the write request is transmitted to the second datastore responsive to the determination that the threshold has been exceeded.

5. The method of claim 4 , wherein the threshold is a number of put requests.

6. The method of claim 4 , wherein the threshold is a data size.

7. A system comprising:
a network interface,
a processor, and
a memory, wherein the memory stores instructions executable by the processor to:
receive a first put request, wherein the first put request includes a first primary key and a corresponding first value;
receive a second put request, wherein the second put request includes a second primary key and a corresponding second value;
store the first put request in a buffer in a first datastore;
store the second put request in the buffer in the first datastore;
transmit, using the network interface, a write request to a second datastore, wherein the write request includes a subsidiary key and a corresponding data file, and wherein the data file includes the first value starting at a first offset and the second value starting at a second offset;
store a first entry in an index in the first datastore, wherein the first entry includes the first primary key along with the subsidiary key and the first offset;
store a second entry in the index in the first datastore, wherein the second entry includes the second primary key along with the subsidiary key and the second offset; and
delete the first put request and the second put request from the buffer.

8. The system of claim 7 , wherein the memory stores instructions executable by the processor to:
receive a get request, wherein the get request includes the first primary key;
based on the first primary key, access the first entry in the index in the first datastore to retrieve the subsidiary key and the first offset;
transmit, using the network interface, a read request to the second datastore, wherein the read request includes the subsidiary key and the first offset;
responsive to the read request, receive, using the network interface, the first value from the second datastore; and
responsive to the get request, transmitting the first value.

9. The system of claim 7 , wherein the memory stores instructions executable by the processor to:
receive a get request, wherein the get request includes the first primary key;
determine whether a put request including the first primary key is currently stored in the buffer in the first datastore;
responsive to a determination that a put request including the first primary key is currently stored in the buffer in the first datastore, access the first put request to retrieve the first value; and
responsive to the get request, transmit the first value.

10. The system of claim 7 , wherein the memory stores instructions executable by the processor to:
determine whether the buffer in the first datastore is storing put requests that collectively exceed a threshold, wherein the write request is transmitted to the second datastore responsive to the determination that the threshold has been exceeded.

11. The system of claim 10 , wherein the threshold is a number of put requests.

12. The system of claim 10 , wherein the threshold is a data size.

13. The system of claim 7 , wherein the first value is a trace blob.

14. The system of claim 7 , wherein the subsidiary key is a filename.

15. The system of claim 7 , wherein the first entry includes a size of the first value.

16. The system of claim 7 , wherein the write request is transmitted via a wide area network.

17. The system of claim 7 , wherein the first datastore is a disk-based storage.

18. A non-transitory computer-readable storage medium that includes instructions that, when executed by a processor, facilitate performance of operations comprising:
receiving a first put request, wherein the first put request includes a first primary key and a corresponding first value;
receiving a second put request, wherein the second put request includes a second primary key and a corresponding second value;
storing the first put request in a buffer in a first datastore;
storing the second put request in the buffer in the first datastore;
transmitting a write request to a second datastore, wherein the write request includes a subsidiary key and a corresponding data file, and wherein the data file includes the first value starting at a first offset and the second value starting at a second offset;
storing a first entry in an index in the first datastore, wherein the first entry includes the first primary key along with the subsidiary key and the first offset;
storing a second entry in the index in the first datastore, wherein the second entry includes the second primary key along with the subsidiary key and the second offset; and
deleting the first put request and the second put request from the buffer.

19. The non-transitory computer-readable storage medium of claim 18 , including instructions that, when executed by a processor, facilitate performance of operations comprising:
receiving a get request, wherein the get request includes the first primary key;
based on the first primary key, accessing the first entry in the index in the first datastore to retrieve

Assignees

• Thoughtspot Inc

Inventors

• Biswas Samprit
• Shekhar Satyam
• Anand Ashok
• Prakash Bhanu

Classifications

• G06F21/6227

  where protection concerns the structure of data, e.g. records, types, queries · CPC title

• G06F16/2445

  Data retrieval commands; View definitions · CPC title

• G06F21/78

  to assure secure storage of data (address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00) · CPC title

• G06F16/2272 Primary

  Management thereof · CPC title

• G06F16/325

  Hash tables · CPC title