Native key-value storage enabled distributed storage system

Patent metadata

Field	Value
Publication number	US-2021181963-A1
Application number	US-202016746803-A
Country	US
Kind code	A1
Filing date	Jan 17, 2020
Priority date	Dec 13, 2019
Publication date	Jun 17, 2021
Grant date	—

Abstract

Inventive aspects include a method, apparatus, and system for supporting a native key-value distributed storage system. The system includes a namenode having a KV-SSD and one or more datanodes each including one or more KV-SSDs. The system includes a client device that is communicatively coupled to the namenode and the one or more datanodes. The client device includes a native key-value storage and networking stack. Some embodiments include a hybrid block-based native key-value distributed storage system that supports both block-based files and native key-value tuples.

First claim

What is claimed is:

1 . A native key-value distributed storage system, comprising:
a namenode including a key-value solid state drive (KV-SSD); one or more datanodes each including one or more KV-SSDs; and a client device that is communicatively coupled to the namenode and the one or more datanodes, wherein the client device includes a native key-value storage and networking stack.

2 . The native key-value distributed storage system of claim 1 , wherein:
the client device includes an application layer that is communicatively coupled to the native key-value storage and networking stack; the client device is configured to cause a key-value tuple to be created responsive to a call to an operation, wherein the call to the operation is made from within the application layer, and the key-value tuple is stored on the one or more KV-SSDs of the one or more datanodes.

3 . The native key-value distributed storage system of claim 2 , wherein the key-value tuple is striped across a first KV-SSD from among the one or more KV-SSDs and a second KV-SSD from among the one or more KV-SSDs.

4 . The native key-value distributed storage system of claim 3 , wherein the striped key-value tuple includes a first internal key-value tuple stored on the first KV-SSD and a second internal key-value tuple stored on the second KV-SSD.

5 . The native key-value distributed storage system of claim 2 , wherein:
the call to the operation is a first call to a first operation; the client device is configured to cause to be accessed the key-value tuple responsive to a second call to a second operation; and the second call to the second operation is made from within the application layer.

6 . The native key-value distributed storage system of claim 5 , wherein the second operation is at least one of a native key-value get( ) operation or a native key-value put( ) operation.

7 . The native key-value distributed storage system of claim 5 , wherein the client device is configured to cause to be deleted the key-value tuple responsive to a third call to a third operation, wherein the third call to the third operation is made from within the application layer.

8 . The native key-value distributed storage system of claim 1 , wherein:
the namenode includes a key-value mapping section including an ordered list of datanodes; and the ordered list of datanodes includes the one or more datanodes.

9 . The native key-value distributed storage system of claim 8 , wherein:
the ordered list is a first ordered list; the first ordered list is associated with a first key-value tuple; the key-value mapping section includes a second ordered list of datanodes; and the second ordered list is associated with a second key-value tuple.

10 . The native key-value distributed storage system of claim 1 , wherein the namenode includes a space management section configured to at least one of store or track a total capacity of each of the one or more datanodes.

11 . The native key-value distributed storage system of claim 10 , wherein the space management section is configured to track a used space of each of the one or more datanodes.

12 . The native key-value distributed storage system of claim 1 , wherein the client device is configured to natively perform at least one of a create operation, an access operation, or a delete operation associated with a plurality of key-value tuples stored on the one or more datanodes in coordination with the namenode.

13 . The native key-value distributed storage system of claim 1 , wherein the KV-SSD of the namenode is the same as the one or more KV-SSDs of the one or more datanodes.

14 . The native key-value distributed storage system of claim 1 , wherein:
the client device includes a remote direct memory access (RDMA) engine; the one or more datanodes each includes a remote direct memory access (RDMA) engine; and the client device is configured to cause a direct data transfer of data associated with a key-value tuple using the RDMA engine of the client device and the RDMA engine of the one or more datanodes, without copying the data associated with the key-value tuple through the native key-value storage and networking stack.

15 . A hybrid block-based native key-value distributed storage system, comprising:
a namenode including a key-value solid state drive (KV-SSD); one or more first datanodes each including one or more KV-SSDs; one or more second datanodes each including one or more block-based solid state drives (SSDs); and a client device that is communicatively coupled to the namenode, the one or more first datanodes, and the one or more second datanodes, wherein the client device includes a hybrid block-based native key-value storage and networking stack.

16 . The native key-value distributed storage system of claim 15 , wherein:
the client device includes an application layer that is communicatively coupled to the hybrid block-based native key-value storage and networking stack; the client device is configured to cause to be created a key-value tuple responsive to a call to a first operation; the call to the first operation is made from within the application layer; the key-value tuple is stored on the one or more KV-SSDs of the one or more first datanodes; the client device is configured to cause to be created a block-based file responsive to a second call to a second operation; the second call to the second operation is made from within the application layer; and the block-based file is stored on the one or more SSDs of the one or more second datanodes.

17 . The native key-value distributed storage system of claim 16 , wherein:
the client device is configured to cause to be accessed the key-value tuple responsive to a third call to a third operation; the third call to the third operation is made from within the application layer; the client device is configured to cause to be accessed the block-based file responsive to a fourth call to a fourth operation; and the fourth call to the fourth operation is made from within the application layer.

18 . The native key-value distributed storage system of claim 17 , wherein:
the client device is configured to cause to be deleted the key-value tuple responsive to a fifth call to a fifth operation; the fifth call to the fifth operation is made from within the application layer; the client device is configured to cause to be deleted the block-based file responsive to a sixth call to a sixth operation; and the sixth call to the sixth operation is made from within the application layer.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Changho
• Bisson Timothy
• Chen Ke

Classifications

• G06F16/182

  Distributed file systems · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0688 Primary

  Non-volatile semiconductor memory arrays · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/284

  Relational databases · CPC title