Data processing method and data processing device

Patent metadata

Field	Value
Publication number	US-2023153006-A1
Application number	US-202217746719-A
Country	US
Kind code	A1
Filing date	May 17, 2022
Priority date	Nov 16, 2021
Publication date	May 18, 2023
Grant date	—

Abstract

A data processing method for a log structured merge (LSM) tree includes selecting SST files to be compressed and merged in a current layer and a next layer, sequentially reading the SST files to be compressed and merged in the current layer and the next layer from a first storage device and sequentially writing the SST files in a second storage device, randomly reading the SST files to be compressed and merged from the second storage device into a memory according to key sequence numbers of data blocks included in the SST files to be compressed and merged, and performing compression and merge processing on the SST files to be compressed and merged. Sequential and random read and write speed of the second storage device is higher than that of the first storage device

First claim

1 . An operation method of data processing device for a log structured merge (LSM) tree, the operation method comprising:
selecting SST files to be compressed and merged in a current layer and a next layer; sequentially reading the SST files to be compressed and merged in the current layer and the next layer from a first storage device, and sequentially writing the SST files in a second storage device; randomly reading the SST files to be compressed and merged from the second storage device into a memory according to key sequence numbers of data blocks included in the SST files to be compressed and merged; and performing compression and merge processing on the SST files to be compressed and merged, wherein sequential and random read and write speed of the second storage device is higher than that of the first storage device.

2 . The operation method of claim 1 , wherein after the SST files to be compressed and merged in the current layer and the next layer are sequentially read from the first storage device and sequentially written into the second storage device, the method further comprises:
updating storage paths of the SST files to be compressed and merged in the second storage device, in a mapping table, wherein randomly reading the SST files to be compressed and merged from the second storage device into the memory according to the key sequence numbers of the data blocks included in the SST files to be compressed and merged comprises: randomly reading the SST files to be compressed and merged from the second storage device into the memory according to the key sequence numbers of the data blocks included in the SST files to be compressed and merged, according to the storage paths of the SST files to be compressed and merged in the second storage device.

3 . The operation method of claim 2 , wherein the mapping table indicates a corresponding relationship between identification numbers of the SST files to be compressed and merged and the storage paths of the SST files to be compressed and merged in the second storage device.

4 . The operation method of claim 1 , wherein the first storage device is a magnetic disk and the second storage device is a solid state drive (SSD).

5 . The operation method of claim 1 , wherein sequentially reading the SST files to be compressed and merged in the current layer and the next layer from the first storage device and sequentially writing the SST files in the second storage device comprises:
in response to a pre-read data command, sequentially reading the SST files to be compressed and merged in the current layer and the next layer from the first storage device and sequentially writing the SST files into the second storage device.

6 . A data processing device for a log structured merge (LSM) tree, the data processing device comprising:
a selecting module configured to select SST files to be compressed and merged in a current layer and a next layer; a prefetching module configured to sequentially read the SST files to be compressed and merged in the current layer and the next layer from a first storage device and sequentially write the SST files in a second storage device; and a reading module configured to randomly read the SST files to be compressed and merged from the second storage device into a memory according to key sequence numbers of data blocks included in the SST files to be compressed and merged, and perform compression and merge processing on the SST files to be compressed and merged, wherein sequential and random read and write speed of the second storage device is higher than that of the first storage device.

7 . The data processing device of claim 6 , wherein the data processing device further comprises:
a mapping table module configured to update storage paths of the SST files to be compressed and merged in the second storage device in a mapping table, wherein the reading module is further configured to: randomly read the SST files to be compressed and merged from the second storage device into the memory according to the key sequence numbers of the data blocks included in the SST files to be compressed and merged, according to the storage paths of the SST files to be compressed and merged in the second storage device.

8 . The data processing device of claim 7 , wherein the mapping table module is further configured to store a corresponding relationship between identification numbers of the SST files to be compressed and merged and the storage paths of the SST files to be compressed and merged in the second storage device.

9 . The data processing device of claim 6 , wherein the first storage device is a magnetic disk and the second storage device is a solid state drive (SSD).

10 . The data processing device of claim 9 , wherein the prefetching module is further configured to:
in response to a pre-read data command, sequentially read the SST files to be compressed and merged in the current layer and the next layer from the first storage device and sequentially write the SST files into the second storage device.

11 - 18 . (canceled)

19 . The operation method of claim 1 , wherein the first storage device is a first NAND, and the second storage device is a second NAND faster than the first NAND.

20 . The operation device of claim 6 , wherein the first storage device is a first NAND, and the second storage device is a second NAND faster than the first NAND.

21 . A storage system comprising:
a first storage device storing a first file and a second file; a second storage device faster than the first storage device; a memory; and a data processing module configured to: perform sequentially read operation for the first and second files from the first storage device, perform sequentially write operation for the first and second files into the second storage device sequentially, and perform random read operation for the first and second files from the second storage device into the memory based on key sequence of the first and second files.

22 . A storage system of claim 21 , wherein:
the first file includes a first block and a second block; the second file includes a third block and a fourth block; and the data processing module further configured to read the third block into the memory between to read the first block and the second block.

23 . The storage system of claim 21 , wherein the first storage device is a magnetic disk and the second storage device is a solid state drive (SSD).

24 . The storage system of claim 21 , wherein the first storage device is a first NAND, and the second storage device is a second NAND faster than the first NAND.

25 . The storage system of claim 21 , wherein a first size of the first and second files read into the memory is smaller than the second size of the first and second files stored in the first storage device.

26 . The storage system of claim 21 , wherein the first and second files are included in a log structured merge (LSM) tree.

27 . The storage system of claim 21 , wherein a type of the first and second files is SST.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Zhang Kun
• Chen Zhao
• Dou Kun
• Qi Bei
• Liu Zhijun

Classifications

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/064

  Management of blocks · CPC title

• G06F3/0643 Primary

  Management of files · CPC title

• G06F3/0683

  Plurality of storage devices · CPC title

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title