Facilitating access to data in distributed storage system

Patent metadata

Field	Value
Publication number	US-11314694-B2
Application number	US-201414577045-A
Country	US
Kind code	B2
Filing date	Dec 19, 2014
Priority date	Dec 31, 2013
Publication date	Apr 26, 2022
Grant date	Apr 26, 2022

Abstract

Embodiments of the present disclosure relate to an apparatus and method for facilitating access to data in a distributed storage system by using a processing unit configured to collect information related to running status of the distributed storage system; obtain metadata related to data stored in the distributed storage system; in response to a request from a client, provide to the client the information related to running status and metadata related to data to be accessed; and extend storage nodes in the distributed storage system so that the client can directly access the storage nodes by using the information related to running status and the metadata related to data to be accessed. Other embodiments are also disclosed.

First claim

What is claimed is:

1. An apparatus for facilitating access to data in a distributed storage system, the apparatus comprising:
a processing unit configured to:
collect information related to an active status of, and a statistical value associated with, one or more performance criteria of one or more nodes within a distributed storage system, wherein the active status of the one or more nodes within the distributed storage system includes one or more of storage node load at a certain time point, storage node performance, and node health status, wherein the processing unit is further configured to collect a real-time matrix from the distributed storage system;
generate, based upon the real-time matrix, a status overall view of at least one of the distributed storage system and a detailed status for each of the one or more nodes at the certain time point;
obtain metadata related to data stored in the distributed storage system, wherein the data has been partitioned and stored in a plurality of stripes within the data storage system;
receive a request from a client for obtaining information related to the data stored in the distributed storage system, wherein in response to receiving the request from the client:
provide to the client the information related to the active status, the statistical value, and metadata related to the data stored in the distributed storage system; and
send a first query request to the processing unit to obtain the information related to the active status and send a second query request to the processing unit to obtain the metadata, wherein the metadata related to the data to be obtained comprises a storage location of each partition of the data to be obtained on the one or more nodes and a storage location of duplication of each partition of the data to be obtained on the one or more nodes; and

extend storage nodes in the distributed storage system wherein the client can directly access the storage nodes using the information related to the active status, the statistical value, and the metadata related to the data stored in the distributed storage system.

2. The apparatus according to claim 1 , further configured to, in response to the first query request, collect the information related to the active status from the distributed storage system; and
in response to the second query request, obtain the metadata from the distributed storage system.

3. The apparatus according to claim 1 , is further configured to periodically obtain the metadata from the distributed storage system.

4. The apparatus according to claim 1 , is further configured to receive a notification indicating a storage status of data changes, and obtain the metadata from the distributed storage system in response to the notification.

5. The apparatus according to claim 1 , is further configured to monitor a message bus to obtain information related to change in a storage status of the data, the information comprising the metadata.

6. The apparatus according claim 1 , further comprises:
an information repository configured to store the information related to the active status and the metadata obtained by the metadata obtaining unit.

7. The apparatus according to claim 6 , wherein the processing unit is further configured to, in response to the request from the client, query the information repository to obtain the information related to the active status and the metadata.

8. The apparatus according to claim 1 , further configured to:
send a request to obtain information related to the active status of the distributed storage system and the metadata related to the data stored in the distributed storage system;
receive a response to the request, the response containing the information related to the active status and the metadata; and
directly access the storage nodes in the distributed storage system using the information related to the active status and the metadata.

9. A method for facilitating access to data in a distributed storage system, the method comprising:
collecting information related to an active status of, and a statistical value associated with, one or more performance criteria of one or more nodes within a distributed storage system, wherein the active status of the one or more nodes within the distributed storage system includes one or more of storage node load at a certain time point, storage node performance, and node health status, wherein collecting information related to an actual status includes collecting a real-time matrix from the distributed storage system;
generating, based upon the real-time matrix, a status overall view of at least one of the distributed storage system and a detailed status for each of the one or more nodes at the certain time point;
obtaining metadata related to data stored in the distributed storage system, wherein the data has been partitioned and stored in a plurality of stripes within the data storage system;
receiving a request from a client for obtaining information related to the data stored in the distributed storage system, wherein in response to receiving the request from the client:
providing to the client the information related to the active status and the metadata related to the data stored in the distributed storage system; and
sending a first query request to the processing unit to obtain the information related to the active status and sending a second query request to the processing unit to obtain the metadata, wherein the metadata related to the data to be obtained comprises a storage location of each partition of the data to be obtained on the one or more nodes and a storage location of duplication of each partition of the data to be obtained on the one or more nodes; and

extending storage nodes in the distributed storage system so that the client can directly access the storage nodes using the information related to the active status and the metadata related to the data stored in the distributed storage system.

10. The method according to claim 9 , further comprises
in response to the request from the client, collecting the information related to the active status from the distributed storage system and obtaining the metadata from the distributed storage system.

11. The method according to claim 9 , further comprises: periodically obtaining the metadata from the distributed storage system.

12. The method according to claim 9 , further comprises:
receiving a notification indicating a storage status of data changes; and
obtaining the metadata from the distributed storage system in response to the notification.

13. The method according to claim 9 , further comprises:
monitoring a message bus to obtain information related to change in a storage status of the data, the information comprising the metadata.

14. The method according to claim 9 , further comprising:
storing the information related to the active status and the metadata in an information repository in the distributed storage system.

15. The method according to claim 14 , further comprises:
in response to a request from the client, sending a query request to the information repository to obtain the information related to the active status and the metadata; and
providing to the client the information related to the active status and the metadata obtained from the information repository.

16. The method according to claim 9 , further comprising:
sending a request to obtain the information related to the status of the distributed storage system and the metadata related to the data stored in the distributed storage system;
receiving a response to the requ

Assignees

• Emc Ip Holding Co Llc

Inventors

• Guo Feng
• Wang Tianqing
• Yan Kai
• Chen Qiyan
• Zhang Yun
• Wu Yicang
• Peng Lin
• Lu Zhihao

Classifications

• G06F16/1748 Primary

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title