Method, apparatus and medium for data synchronization between cloud database nodes

Patent metadata

Field	Value
Publication number	US-11657066-B2
Application number	US-202017107588-A
Country	US
Kind code	B2
Filing date	Nov 30, 2020
Priority date	Nov 30, 2020
Publication date	May 23, 2023
Grant date	May 23, 2023

Abstract

Methods, processing units, and computer-readable media in a cloud-based database are described. Redo log records are applied to a page at a database replica only when an updated version of the page is requested at the database replica. A log cache may be used by a replica node of the database to track recent redo log records applicable to a given page. The recent redo log records stored in the log cache may be applied to update the page on-demand when an updated version of the page is requested. By applying only the redo log records applicable to pages that are currently being requested, processing resources may be used only to generate pages that are currently required. Methods for registering redo log records into the log cache by the master or replica server are also described.

First claim

The invention claimed is:

1. A method for serving database data of a cloud-based database comprising a compute layer and a storage layer, the compute layer comprising a database master server configured to generate redo log records and a database replica server configured to respond to client read requests, the storage layer comprising a log store configured to store the redo log records and a page store configured to store database data pages, update the database data pages based on the redo log records, and provide the updated database data pages to the database replica server, the method comprising:
receiving from a client, at the database replica server, a read request for a desired page of database data; and
in response to receiving the request for the desired page:
in response to determining that neither the database replica server nor the page store currently stores an up to date version of the desired page:
retrieving an out of date stored version of the desired page;
retrieving one or more redo log records associated with the desired page from a log cache operating in the compute layer; and
applying, at the database replica server, the one or more redo log records to the out of date stored version of the desired page to generate an updated desired page; and

sending, from the database replica server to the client, the updated desired page.

2. The method of claim 1 , wherein:
the cloud-based database is a multi-master cloud-based database; and
the database replica server is a database master server.

3. The method of claim 1 , wherein the log cache is part of the database replica server.

4. The method of claim 1 , wherein the log cache is part of the cloud-based database separate from the database replica server.

5. The method of claim 1 , wherein the database replica server retrieves the stored version of the desired page from a buffer pool of the database replica server.

6. The method of claim 1 , wherein the database replica server retrieves the stored version of the desired page from a page store of the cloud-based database.

7. The method of claim 1 , further comprising, prior to retrieving the one or more redo log records associated with the desired page:
storing a plurality of redo log records to the log cache, the plurality of redo log records comprising at least one of the one or more redo log records associated with the desired page.

8. The method of claim 7 , wherein the step of storing the plurality of redo log records to the log cache is performed by a database master server.

9. The method of claim 7 , wherein the step of storing the plurality of redo log records to the log cache is performed by the database replica server.

10. The method of claim 7 , further comprising, prior to storing the plurality of redo log records to the log cache:
determining that the log cache has insufficient memory available to store the plurality of redo log records; and
evicting one or more redo log records stored in the log cache.

11. The method of claim 7 , wherein:
each redo log record of the plurality of redo log records is associated with a page;
storing the plurality of redo log records to the log cache comprises storing each redo log record of the plurality of redo log records, in a lookup table of the log cache, in association with a lookup table entry corresponding to a page associated with the redo log record; and
retrieving the one or more redo log records from the log cache comprises retrieving one or more redo log records stored in association with a lookup table entry corresponding to the desired page.

12. The method of claim 11 , wherein:
the stored version of the desired page is associated with a log sequence number; and
retrieving the one or more redo log records comprises retrieving one or more redo log records stored in association with log sequence numbers higher than the log sequence number of the stored version of the desired page.

13. The method of claim 12 , further comprising, prior to retrieving the one or more redo log records associated with the desired page:
determining that the log cache has insufficient memory available to store a plurality of redo log records;
evicting one or more redo log records stored in the log cache;
storing the plurality of redo log records to the log cache, the plurality of redo log records comprising at least one of the one or more redo log records associated with the desired page; and
updating a visible log sequence number of the database replica server to a value equal to an ending log sequence number associated with the plurality of redo log records.

14. A processing unit, comprising:
a processing device; and
a memory storing instructions that, when executed by the processing device, cause the processing unit to perform the functions of a database replica server configured to respond to client read requests for database data of a cloud-based database, the cloud-based database comprising a compute layer and a storage layer, the compute layer comprising a database master server configured to generate redo log records and the database replica server, the storage layer comprising a log store configured to store the redo log records and a page store configured to store database data pages, update the database data pages based on the redo log records, and provide the updated database data pages to the database replica server, the database replica server being further configured to:
receive, from a client, a read request for a desired page of database data; and
in response to receiving the request for the desired page:
in response to determining that neither the database replica server nor the page store currently stores an up to date version of the desired page:
retrieve an out of date stored version of the desired page;
retrieve one or more redo log records associated with the desired page from a log cache operating in the compute layer; and
apply, at the database replica server, the one or more redo log records to the out of date stored version of the desired page to generate an updated desired page; and
send, to the client, the updated desired page.

15. The processing unit of claim 14 , wherein:
the processing unit comprises the log cache; and
the instructions, when executed by the processing device, further cause the processing unit to perform the functions of a log cache.

16. The processing unit of claim 15 , wherein:
the log cache comprises a plurality of stored redo log records, each stored redo log record of the plurality of stored redo log records being stored in a lookup table of the log cache in association with a lookup table entry corresponding to a page; and
retrieving the one or more redo log records from the log cache comprises retrieving one or more stored redo log records stored in association with the lookup table entry corresponding to the desired page.

17. The processing unit of claim 16 , wherein:
the instructions, when executed by the processing device, further cause the processing unit to perform the functions of a buffer pool; and
the stored version of the desired page is retrieved from the buffer pool.

18. The processing unit of claim 17 , wherein:
the stored version of the desired page is associated with a log sequence number; and
retrieving the one or more redo log records comprises retrieving one or more redo log records stored in association with log sequence numbers higher than the log sequence number of the stored version of the desired page.

19. The processing unit of claim 18 , wherein the

Assignees

• Huawei Cloud Computing Tech Co Ltd

Inventors

• Chen Chong
• CHEN Jin
• Lin Shu
• Sun Chunsheng

Classifications

• G06F16/27 Primary

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• H04L67/1095 Primary

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• G06F16/9017

  using directory or table look-up (use of a directory or look-up table in file systems G06F16/13) · CPC title

• H04L67/568

  Storing data temporarily at an intermediate stage, e.g. caching · CPC title