Native key-value storage enabled distributed storage system

Patent metadata

Field	Value
Publication number	US-11287994-B2
Application number	US-202016746803-A
Country	US
Kind code	B2
Filing date	Jan 17, 2020
Priority date	Dec 13, 2019
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

Inventive aspects include a method, apparatus, and system for supporting a native key-value distributed storage system. The system includes a namenode having a KV-SSD and one or more datanodes each including one or more KV-SSDs. The system includes a client device that is communicatively coupled to the namenode and the one or more datanodes. The client device includes a native key-value storage and networking stack. Some embodiments include a hybrid block-based native key-value distributed storage system that supports both block-based files and native key-value tuples.

First claim

What is claimed is:

1. A native key-value distributed storage system, comprising:
a namenode comprising a key-value mapping section comprising an ordered list of one or more datanodes;
one or more datanodes comprising one or more key-value storage devices; and
a client device comprising an application layer communicatively coupled to a native key-value storage and networking stack.

2. The native key-value distributed storage system of claim 1 , wherein:
the client device is configured to cause a key-value tuple to be created based on a call to an operation,
wherein the call to the operation is made from within the application layer, and
the key-value tuple is stored on the one or more key-value storage devices of the one or more datanodes.

3. The native key-value distributed storage system of claim 1 , wherein a key-value tuple is striped across a first key-value storage device from among the one or more key-value storage devices and a second key-value storage device from among the one or more key-value storage devices.

4. The native key-value distributed storage system of claim 3 , wherein the key-value tuple comprises a first internal key-value tuple stored on the first key-value storage device and a second internal key-value tuple stored on the second key-value storage device.

5. The native key-value distributed storage system of claim 2 , wherein:
the call to the operation is a first call to a first operation;
the client device is configured to cause to be accessed the key-value tuple based on a second call to a second operation; and
the second call to the second operation is made from within the application layer.

6. The native key-value distributed storage system of claim 5 , wherein the second operation is at least one of a native key-value get( ) operation or a native key-value put( ) operation.

7. The native key-value distributed storage system of claim 5 , wherein the client device is configured to cause to be deleted the key-value tuple based on a third call to a third operation, wherein the third call to the third operation is made from within the application layer.

8. The native key-value distributed storage system of claim 1 , wherein:
the ordered list of datanodes comprises the one or more datanodes.

9. The native key-value distributed storage system of claim 8 , wherein:
the ordered list is a first ordered list;
the first ordered list is associated with a first key-value tuple;
the key-value mapping section comprises a second ordered list of datanodes; and
the second ordered list is associated with a second key-value tuple.

10. The native key-value distributed storage system of claim 1 , wherein the namenode comprises a key-value storage device, and the key-value storage device of the namenode is the same as the one or more key-value storage devices of the one or more datanodes.

11. The native key-value distributed storage system of claim 1 , wherein:
the client device comprises a remote direct memory access (RDMA) engine;
the one or more datanodes comprises a remote direct memory access (RDMA) engine; and
the client device is configured to cause a direct data transfer of data associated with a key-value tuple using the RDMA engine of the client device and the RDMA engine of the one or more datanodes.

12. The native key-value distributed storage system of claim 1 , wherein the namenode comprises a space management section configured to at least one of store or track a capacity of at least one of the one or more datanodes.

13. The native key-value distributed storage system of claim 12 , wherein the space management section is configured to track a used space of at least one of the one or more datanodes.

14. A native key-value distributed storage system, comprising:
a namenode comprising a key-value storage device;
one or more datanodes comprising one or more key-value storage devices; and
a client device that is communicatively coupled to the namenode and the one or more datanodes, wherein the client device comprises a native key-value storage and networking stack, wherein the client device is configured to natively perform at least one of a create operation, an access operation, or a delete operation associated with a plurality of key-value tuples stored on the one or more datanodes in coordination with the namenode.

15. A hybrid block-based native key-value distributed storage system, comprising:
a namenode comprising a key-value storage device;
one or more first datanodes comprising one or more key-value storage devices;
one or more second datanodes comprising one or more block-based storage devices; and
a client device that is communicatively coupled to the namenode, the one or more first datanodes, and the one or more second datanodes, wherein the client device comprises a hybrid block-based native key-value storage and networking stack,
wherein:
the client device comprises an application layer that is communicatively coupled to the hybrid block-based native key-value storage and networking stack;
the client device is configured to cause to be created a key-value tuple based on a call to a first operation;
the call to the first operation is made from within the application layer;
the key-value tuple is stored on the one or more key-value storage devices of the one or more first datanodes;
the client device is configured to cause to be created a block-based file based on a second call to a second operation;
the second call to the second operation is made from within the application layer; and
the block-based file is stored on the one or more storage devices of the one or more second datanodes.

16. The native key-value distributed storage system of claim 15 , wherein:
the client device is configured to cause to be accessed the key-value tuple based on a third call to a third operation;
the third call to the third operation is made from within the application layer;
the client device is configured to cause to be accessed the block-based file based on a fourth call to a fourth operation; and
the fourth call to the fourth operation is made from within the application layer.

17. The native key-value distributed storage system of claim 16 , wherein:
the client device is configured to cause to be deleted the key-value tuple based on a fifth call to a fifth operation;
the fifth call to the fifth operation is made from within the application layer;
the client device is configured to cause to be deleted the block-based file based on a sixth call to a sixth operation; and
the sixth call to the sixth operation is made from within the application layer.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Changho
• Bisson Timothy
• Chen Ke

Classifications

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0688 Primary

  Non-volatile semiconductor memory arrays · CPC title

• G06F16/182

  Distributed file systems · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title