Method for replaying log on data node, data node, and system

Patent metadata

Field	Value
Publication number	US-12093241-B2
Application number	US-202218147507-A
Country	US
Kind code	B2
Filing date	Dec 28, 2022
Priority date	Jun 30, 2020
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

A method for replaying a log on a data node, a data node, and a database system are described. The method is used to improve a log replay speed, and includes: the data node obtaining a plurality of logs, wherein the plurality of logs includes at least one transaction commit log and at least one page operation log; and a first thread of a plurality of threads replaying the at least one page operation log, and a same second thread replaying all page operation logs including an operation on a same page. As such, log replay progress does not need to be synchronized between threads. A process in which the first thread replays the transaction commit log is independent of a process in which at least one second thread replays the at least one page operation log.

First claim

What is claimed is:

1. A method for replaying a log comprising:
obtaining, by a data node having a plurality of threads running therein, a plurality of logs including at least one transaction commit log and at least one page operation log, wherein each of the at least one transaction commit log comprises a transaction commit operation, and each of the at least one page operation log comprises one or more operations on one page;
replaying, by the data node, the at least one transaction commit log by using a first thread of the plurality of threads;
replaying, by the data node, the at least one page operation log by using at least one second thread of the plurality of threads, wherein all page operation logs that are in the at least one page operation log and that comprise an operation on a same page are replayed by using a same second thread; and
determining, by the data node, a maximum visible log sequence number (LSN), wherein the maximum visible LSN indicates a log replay progress;
wherein all transaction commit logs whose log sequence numbers (LSNs) are less than or equal to the maximum visible LSN in the at least one transaction commit log are replayed, and all page operation logs whose LSNs are less than or equal to the maximum visible LSN in the at least one page operation log are replayed; and
wherein a process in which the first thread replays the at least one transaction commit log is independent of a process in which the at least one second thread replays the at least one page operation log.

2. The method according to claim 1 , wherein obtaining the plurality of logs comprises:
parsing, by the data node, at least one operation log to obtain the at least one page operation log, wherein each of the at least one operation log comprises an operation on one or more pages;
wherein the at least one page operation log comprises a first page operation log, and an LSN of the first page operation log is the same as an LSN of an operation log which is parsed to obtain the first page operation log.

3. The method according to claim 1 , wherein replaying the at least one transaction commit log comprises:
replaying, by the data node by using the first thread of the plurality of threads, the at least one transaction commit log on an initial version of a first page in an LSN sequence, to obtain a plurality of versions of the first page, and each of the plurality of versions of the first page is associated with at least one of the at least one transaction commit log;
wherein the plurality of versions of the first page comprise a first version of the first page and a second version of the first page that is generated based on the first version of the first page, wherein the second version of the first page is obtained by replaying, on the first version of the first page in the LSN sequence, transaction commit logs associated with the first version of the first page.

4. The method according to claim 3 , wherein a quantity of transaction commit logs associated with each of the plurality of versions of the first page is the same, or replay time of transaction commit logs associated with each of the plurality of versions of a second page is the same.

5. The method according to claim 3 , further comprising:
replaying, by the data node, in response to a first indication, on a reference version of the first page in the LSN sequence, transaction commit logs associated with the reference version until a first transaction commit log is relayed, to obtain a target version of the first page, wherein the first indication is used to query log replay progress of the at least one transaction commit log,
wherein the reference version of the first page is a version that is associated with the first transaction commit log and that is in the plurality of versions of the first page, and the first transaction commit log is a log whose LSN is the largest and whose LSN is less than or equal to the maximum visible LSN in the at least one transaction commit log; and
determining the log replay progress of the at least one transaction commit log based on the target version of the first page.

6. The method according to claim 1 , wherein replaying the at least one page operation log comprises:
replaying, by the data node by using one second thread of the plurality of threads, on an initial version of a second page in an LSN sequence, all page operation logs that are in the at least one page operation log and that comprise an operation on the second page, to obtain a plurality of versions of the second page, and each of the plurality of versions of the second page is associated with at least one of all page operation logs comprising the operation on the second page;
wherein the plurality of versions of the second page comprise a first version of the second page and a second version of the second page that is generated based on the first version of the second page, and the second version of the second page is obtained by replaying, on the first version of the second page in the LSN sequence, page operation logs associated with the first version of the second page.

7. The method according to claim 6 , wherein a quantity of page operation logs associated with each of the plurality of versions of the second page is the same, or replay time of page operation logs associated with each of the plurality of versions of the second page is the same.

8. The method according to claim 6 , further comprising:
replaying, by the data node, in response to a second indication, on a reference version of the second page in the LSN sequence, transaction commit logs associated with the reference version until a first page operation log is replayed, to obtain a target version of the second page, wherein the second indication is used to query log replay progress of the at least one page operation log,
wherein the reference version of the second page is a version that is associated with the first page operation log and that is in the plurality of versions of the second page, and the first page operation log is a log whose LSN is the largest and whose LSN is less than or equal to the maximum visible LSN in all the page operation logs comprising the operation on the second page; and
determining the log replay progress of the at least one page operation log based on the target version of the second page.

9. A data node, comprising:
at least one processor executing a plurality of threads; and
one or more memories coupled to the at least one processor and storing programming instructions therein for execution by the at least one processor to cause the data node to:
obtain a plurality of logs having at least one transaction commit log and at least one page operation log, wherein each of the at least one transaction commit log comprises a transaction commit operation, and each of the at least one page operation log comprises one or more operations on one page;
replay the at least one transaction commit log by using a first thread of the plurality of threads;
replay the at least one page operation log by using at least one second thread of the plurality of threads, wherein all page operation logs that are in the at least one page operation log and that comprise an operation on a same page are replayed by using a same second thread; and
determine a maximum visible log sequence number (LSN), wherein the maximum visible LSN indicates a log replay progress;
wherein a process in which the first thread replays the at least one transaction commit log is independent of a process in which the at least one second thread replays the at least one page operation log;
wherein all transaction commit logs whose log sequence numbers (LSNs) are less than or equal to the maximum visible LSN in the at least

Assignees

• Huawei Tech Co Ltd

Inventors

• Zhang Qiong
• Li Xuli
• Li Heting
• Yao Geyong
• Hu Xiang

Classifications

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F16/2358 Primary

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F11/20

  using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements · CPC title