Distributed storage system, data processing method, and storage node

Patent metadata

Field	Value
Publication number	US-11262916-B2
Application number	US-202016943279-A
Country	US
Kind code	B2
Filing date	Jul 30, 2020
Priority date	Jan 31, 2018
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

A distributed storage system including at least one storage node, and at least one object storage device (OSD) is deployed on the at least one storage node. At least one data processing device and at least one metadata processing device that has a mapping relationship with the at least one data processing device are deployed on the at least one OSD. A target data processing module of an input/output (IO) operation is configured to determine a target metadata processing device corresponding to the target data processing device and an identifier of an OSD in which the target metadata processing device is located, send an access operation to the target metadata processing device, and access data of the IO operation based on the IO operation. The target metadata processing device is configured to access metadata of the IO operation based on the access operation.

First claim

What is claimed is:

1. A distributed storage system comprising:
a plurality of storage nodes, wherein at least one storage node comprises at least one object storage device (OSD), wherein the at least one OSD comprises a data processing device and a metadata processing device that has a mapping relationship with the data processing device;
a target metadata processing device; and
a target data processing device, wherein the distributed storage system is allocated to —an input/output (IO) operation, having a first mapping relationship with and coupled to the target data processing device and configured to:
receive the IO operation;
access data of the IO operation;
determine the target metadata processing device and a first identifier of a first OSD in which the target metadata processing device is located; and
send an access operation to the target metadata processing device based on the first identifier,

wherein the target metadata processing device is configured to:
receive the access operation; and
access metadata of the IO operation based on the access operation.

2. The distributed storage system of claim 1 , wherein the target data processing device is further configured to:
determine, based on the first mapping relationship, the target metadata processing device; and
determine, based on a second mapping relationship between the target metadata processing device and the first OSD, the first identifier.

3. The distributed storage system of claim 1 , wherein the IO operation is a write operation, wherein the access operation is a write access operation that carries metadata of the write operation, wherein the target data processing device is further configured to:
write, based on the write operation, data of the write operation into a storage node in which the target data processing device is located; and
generate the metadata of the write operation, and
wherein the target metadata processing device is further configured to write the metadata of the write operation into at least two storage nodes based on the write access operation.

4. The distributed storage system of claim 1 , wherein the IO operation is a read operation, wherein the access operation is a read access operation, wherein the target metadata processing device is further configured to:
read metadata of the read operation based on the read access operation; and
send the metadata of the read operation to the target data processing device, and
wherein the target data processing device is further configured to read data of the read operation based on the read operation and the metadata of the read operation.

5. The distributed storage system of claim 1 , further comprising a second OSD in which the target data processing device is located, wherein the first OSD is different from the second OSD.

6. The distributed storage system of claim 1 , wherein each of the OSDs further comprises a management device, wherein the management device comprises a third mapping relationship between the management device and the data processing device, and wherein the management device is configured to:
receive the IO operation, wherein the IO operation is a write operation; and
calculate data protection information of the write operation.

7. The distributed storage system of claim 6 , further comprising a client configured to:
receive the IO operation;
determine the management device and a third identifier of a third OSD in which the management device is located; and
send the IO operation to the management device based on the third identifier.

8. The distributed storage system of claim 6 , wherein the management device is further configured to:
receive the IO operation, wherein the IO operation is the write operation;
determine the target data processing device and a second identifier of a second OSD in which the target data processing device is located; and
send the write operation to the target data processing device based on the second identifier.

9. The distributed storage system of claim 1 , further comprising a client configured to:
receive the IO operation, wherein the IO operation is a read operation;
determine the target data processing device and a second identifier of a second OSD in which the target data processing device is located; and
send the read operation to the target data processing device based on the second identifier.

10. The distributed storage system of claim 1 , wherein each of the OSDs further comprises a management device, wherein the management device comprises a third mapping relationship between the management device and the data processing device, and wherein the management device is configured to:
receive the IO operation, wherein the IO operation is a read operation;
obtain data protection information of the read operation;
read, based on the data protection information, data comprising a protection relationship with data of the read operation; and
reconstruct the data of the read operation based on the data comprising the protection relationship with the data of the read operation.

11. A data processing method implemented by a target data processing device of a distributed storage system, wherein the data processing method comprises:
receiving an input/output (IO) operation;
allocating the target data processing device to the IO operation;
determining a target metadata processing device and a first identifier of a first object storage device (OSD) in which the target metadata processing device is located, wherein the target metadata processing device comprises a first mapping relationship with the target data processing device;
sending an access operation to the target metadata processing device based on the first identifier to instruct the target metadata processing device to access metadata of the IO operation; and
accessing data of the IO operation based on the IO operation.

12. The data processing method of claim 11 , further comprising:
determining based on the first mapping relationship, the target metadata processing device that has the mapping relationship with the target data processing module; and
determining, based on a second mapping relationship between the target metadata processing device and the first OSD, the first identifier in which the target metadata processing module is located.

13. The data processing method of claim 11 , wherein the IO operation is a write operation, wherein the access operation is a write access operation comprising metadata of the write operation, and wherein the data processing method further comprises:
writing, based on the write operation, data of the write operation into a storage node in which the target data processing device is located; and
generating the metadata of the write operation.

14. The data processing method of claim 11 , wherein the IO operation is a read operation, wherein the access operation is a read access operation, and wherein the data processing method further comprises:
receiving metadata of the read operation from the target metadata processing device; and
reading data of the read operation based on the read operation and the metadata of the read operation.

15. A data processing method implemented by a target metadata processing device of a distributed storage system, wherein the data processing method comprises:
receiving an access operation from a target data processing device that is allocated by the distributed storage system to an input/output (IO) operation wherein the target metadata processing device comprises a first mapping relationship with the target data processing device, wherein the d

Assignees

• Huawei Tech Co Ltd

Inventors

• Wu Zhangling
• Wei Mingchang

Classifications

• G06F3/0613 Primary

  in relation to throughput · CPC title

• G06F3/0658

  Controller construction arrangements · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/061 Primary

  Improving I/O performance · CPC title