Key-value store with internal key-value storage interface

Patent metadata

Field	Value
Publication number	US-9870168-B1
Application number	US-201414579323-A
Country	US
Kind code	B1
Filing date	Dec 22, 2014
Priority date	Dec 22, 2014
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

A key-value store is provided having one or more key-value storage interfaces. A key-value store on at least one compute node comprises a memory for storing a plurality of key-value pairs; and an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device providing a key-value interface for persistent storage of one or more of the plurality of key-value pairs, wherein the software interface module provides the one or more key-value pairs to the at least one persistent storage device in a key-value format. The abstract storage interface optionally processes one or more batch operations on the plurality of key-value pairs. A distributed embodiment for a partitioned key-value store is also provided.

First claim

What is claimed is:

1. A key-value store on at least one compute node, comprising:
a memory for storing a plurality of key-value pairs, wherein keys in said key-value pairs comprise semantic information; and
an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device providing a key-value interface for persistent storage of one or more of said plurality of key-value pairs, wherein said software interface module provides said one or more key-value pairs to said at least one persistent storage device in a key-value format that maintains said semantic information of said keys in said key-value pairs.

2. The key-value store of claim 1 , wherein said at least one persistent storage device comprises one or more of a flash storage device providing a key-value interface and a disk storage device providing a key-value interface.

3. The key-value store of claim 2 , wherein said flash storage device providing said key-value interface and said disk storage device providing said key-value interface comprise respective disjoint subsets of storage.

4. The key-value store of claim 1 , wherein said abstract storage interface processes one or more batch operations on said plurality of key-value pairs.

5. The key-value store of claim 1 , wherein said abstract storage interface provides a storage-specific interface for a plurality of storage destination types.

6. The key-value store of claim 1 , wherein said abstract storage interface provides a key-value interface between one or more local key-value stores and one or more additional storage devices having a key-value interface.

7. The key-value store of claim 1 , wherein a processor in said at least one persistent storage device performs one or more of index building, reorganization, resorting, deduplication and compression of said one or more key-value pairs.

8. A distributed key-value store, comprising:
a plurality of compute nodes connected by a network, wherein at least one of said compute nodes comprises a partitioned key-value store server;
wherein the partitioned key-value store server is configured to store key-value data using a local key-value store, wherein keys in said key-value data comprise semantic information, said local key-value store comprising an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device providing a key-value interface for persistent storage of one or more key-value pairs, wherein said software interface module provides said one or more key-value pairs to said at least one persistent storage device in a key-value format that maintains said semantic information of said keys in said key-value pairs.

9. The distributed key-value store of claim 8 , wherein the partitioned key-value store server is configured to communicate with a plurality of partitioned key-value store clients over a network and to process requests from said plurality of partitioned key-value store clients.

10. The distributed key-value store of claim 8 , wherein the partitioned key-value store server is further configured to communicate with one or more additional partitioned key-value store servers over a network and to process requests from said one or more additional partitioned key-value store servers.

11. The distributed key-value store of claim 8 , further comprising at least one additional partitioned key-value store server, wherein each partitioned key-value store server stores a globally sorted range of keys.

12. The distributed key-value store of claim 8 , further comprising at least one additional partitioned key-value store server, wherein each partitioned key-value store server stores a locally sorted range of keys within a set of string table files (SST).

13. The distributed key-value store of claim 8 , further comprising at least one additional partitioned key-value store server, wherein at least one of said partitioned key-value store servers processes one or more batch operations on one or more sorted ranges of keys.

14. A method comprising:
providing a key-value store server on at least one compute node of a distributed storage system for storing a plurality of key-value pairs, wherein keys in said key-value pairs comprise semantic information; and
communicating with at least one persistent storage device providing a key-value interface for persistent storage of one or more of said plurality of key-value pairs using an abstract storage interface comprising a software interface module, wherein said software interface module provides said one or more key-value pairs to said at least one persistent storage device in a key-value format that maintains said semantic information of said keys in said key-value pairs.

15. The method of claim 14 , wherein said at least one persistent storage device comprises one or more of a flash storage device providing a key-value interface and a disk storage device providing a key-value interface.

16. The method of claim 14 , wherein said abstract storage interface processes one or more batch operations on said plurality of key-value pairs.

17. The method of claim 14 , wherein said abstract storage interface provides a storage-specific interface for a plurality of storage destination types.

18. The method of claim 14 , wherein said abstract storage interface provides a key-value interface between one or more local key-value stores and one or more additional storage devices having a key-value interface.

19. The method of claim 14 , wherein a processor in said at least one persistent storage device performs one or more of index building, reorganization, resorting, deduplication and compression of said one or more key-value pairs.

20. The method of claim 14 , wherein said key-value store server comprises a partitioned key-value store server configured to communicate with one or more additional partitioned key-value store servers over a network and to process requests from said one or more additional partitioned key-value store servers.

21. The key-value store of claim 6 , wherein said one or more local key-value stores comprise a different hardware tier comprising respectively disjoint subsets of storage relative to said one or more additional storage devices.

22. The method of claim 18 , wherein said one or more local key-value stores comprise a different hardware tier comprising respectively disjoint subsets of storage relative to said one or more additional storage devices.

Assignees

• Emc Corp
• Los Alamos Nat Security Llc

Inventors

• Bent John M
• Faibish Sorin
• Ting Dennis P J
• Tzelnic Percy
• Gupta Uday
• Grider Gary
• Bonnie David J

Classifications

• G06F3/0607 Primary

  by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device · CPC title

• G06F3/0614

  Improving the reliability of storage systems · CPC title

• G06F3/0647 Primary

  Migration mechanisms · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0664

  at device level, e.g. emulation of a storage device or system · CPC title