Methods and apparatus to organize an object store namespace

Patent metadata

Field	Value
Publication number	US-12373462-B2
Application number	US-202318535818-A
Country	US
Kind code	B2
Filing date	Dec 11, 2023
Priority date	Oct 5, 2023
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

Disclosed examples create at least first and second database shards in a leader node, the leader node located in a consensus ring; and cause replication of first namespace metadata in the at least the first and second database shards of the leader node and in at least first and second database shards in a follower node, the follower node located in the consensus ring.

First claim

What is claimed is:

1. An apparatus comprising:
interface circuitry;
machine-readable instructions; and
programmable circuitry to at least one of instantiate or execute the machine-readable instructions to:
create at least first and second database shards in a leader node, the leader node located in a consensus ring; and
cause replication of first namespace metadata to the at least the first and second database shards of the leader node and to at least first and second database shards in corresponding ones of first and second namespace databases in a follower node, the follower node located in the consensus ring.

2. The apparatus of claim 1 , comprising:
determine that the at least the first and second database shards are filled to a threshold capacity;
create a third database shard in the leader node and a third database shard in the follower node; and
cause replication of second namespace metadata in the third database shard of the leader node and in the third database shard of the follower node.

3. The apparatus of claim 2 , wherein the programmable circuitry is to create the third database shards in the leader node and in the follower node to scale up an object store namespace based on at least one additional object identified by a client device.

4. The apparatus of claim 1 , wherein the programmable circuitry is to cause replication of cross-shard metadata in a master shard of the leader node and in a master shard of the follower node.

5. The apparatus of claim 4 , wherein the cross-shard metadata includes volume information corresponding to volumes of the first namespace metadata stored in the at least the first and second database shards in the leader node.

6. The apparatus of claim 1 , wherein the programmable circuitry is to cause the replication of the first namespace metadata to the at least the first and second database shards of the leader node and to the at least the first and second database shards in the follower node by writing the first namespace metadata to the leader node and the follower node in parallel.

7. The apparatus of claim 1 , wherein the programmable circuitry is to cause the replication of the first namespace metadata to the at least the first and second database shards of the leader node and to the at least the first and second database shards in the follower node by causing transmission of transaction commands to command logs of the leader and follower nodes.

8. The apparatus of claim 1 , wherein the programmable circuitry is to create a third database shard in the leader node to increase input/output operations (IOPS), the IOPS to execute transaction commands in a command log of the leader node.

9. A non-transitory computer-readable medium comprising instructions to cause programmable circuitry to at least:
establish database shards for a leader node and a follower node of a consensus ring;
generate cross-shard metadata, the cross-shard metadata to describe the database shards; and
initiate transactions to command logs in the leader node and the follower node, the transactions to write namespace metadata and the cross-shard metadata in first ones of the database shards in the leader node and replicate the namespace metadata and the cross-shard metadata to second ones of the database shards in corresponding namespace databases in the follower node.

10. The non-transitory computer-readable medium of claim 9 , wherein the instructions are to cause the programmable circuitry to:
determine that the first ones of the database shards are filled to a threshold capacity;
create an additional database shard in the leader node and an additional database shard in the follower node; and
cause replication of second namespace metadata in the additional database shards of the leader node and the follower node.

11. The non-transitory computer-readable medium of claim 10 , wherein the instructions are to cause the programmable circuitry to create the additional database shards in the leader node and in the follower node to scale up a namespace based on a client device requesting to represent at least one additional object in the namespace.

12. The non-transitory computer-readable medium of claim 9 , wherein the instructions are to cause the programmable circuitry to cause replication of the cross-shard metadata in a master shard of the leader node and in a master shard of the follower node.

13. The non-transitory computer-readable medium of claim 12 , wherein the cross-shard metadata includes volume information corresponding to volumes of the namespace metadata stored in the database shards of the leader node.

14. The non-transitory computer-readable medium of claim 9 , wherein the instructions are to cause the programmable circuitry to initiate the transactions to the command logs by causing transmission of transaction commands to the command logs of the leader node and the follower node.

15. The non-transitory computer-readable medium of claim 9 , wherein the instructions are to cause the programmable circuitry to create an additional database shard in the leader node to increase input/output operations (IOPS), the IOPS to execute transaction commands in the leader node.

16. A method comprising:
creating at least first and second database shards in a leader node, the leader node located in a consensus ring; and
causing replication of first namespace metadata across the at least the first and second database shards of the leader node and at least first and second database shards in corresponding ones of first and second namespace databases in a follower node, the follower node located in the consensus ring.

17. The method of claim 16 , comprising:
determining that the at least the first and second database shards are filled to a threshold capacity;
creating a third database shard in the leader node and a third database shard in the follower node; and
causing replication of second namespace metadata in the third database shard of the leader node and in the third database shard of the follower node.

18. The method of claim 17 , wherein the creating of the third database shards in the leader node and in the follower node is to scale up a namespace based on a client device requesting to represent at least one additional object in the namespace.

19. The method of claim 16 , including causing replication of cross-shard metadata in a master shard of the leader node and in a master shard of the follower node.

20. The method of claim 19 , wherein the cross-shard metadata includes volume information corresponding to volumes of the first namespace metadata stored in the at least the first and second database shards in the leader node.

21. The method of claim 16 , wherein the replication of the first namespace metadata across the at least the first and second database shards of the leader node and the at least the first and second database shards in the follower node is in parallel.

22. The method of claim 16 , wherein the replication of the first namespace metadata across the at least the first and second database shards of the leader node and the at least the first and second database shards in the follower node is performed by causing transmission of transaction commands to command logs of the leader and follower nodes.

23. The method of claim 16 , including creating a third database shard in the leader node to increase input/output operations (IOPS), the IOPS to execute transaction commands in a command log of the leader node.

Assignees

• Cloudera Inc

Inventors

• Pogde Prashant
• Wagle Siddharth Jivan
• Gangumalla Uma Maheswara Rao
• Agarwal Arpit Ashok

Classifications

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F16/278 Primary

  Data partitioning, e.g. horizontal or vertical partitioning · CPC title