Database system with database engine and separate distributed storage service

Patent metadata

Field	Value
Publication number	US-12038906-B2
Application number	US-202218054870-A
Country	US
Kind code	B2
Filing date	Nov 11, 2022
Priority date	Mar 15, 2013
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

First claim

The invention claimed is:

1. A system, comprising:
one or more computing nodes, each of which comprises at least one processor and a memory, wherein the one or more computing nodes are configured to collectively implement a distributed data store that stores data blocks on one or more storage devices;
wherein one of the one or more computing nodes is configured to:
receive a read request targeting one of the data blocks;
in response to the receipt of the read request for the targeted data block:
determine that the computing node does not store a current version of the targeted data block;
in response to the determination that the computing node does not store a current version of the targeted data block:
retrieve a previously stored version of the targeted data block;
apply one or more update records that are stored on the computing node to the previously stored version of the targeted data block to generate the current version of the targeted data block, wherein the one or more update records specify a modification to be made to the targeted data block; and

return the current version of the targeted data block.

2. The system of claim 1 , wherein the one computing node is further configured to write the current version of the targeted data block to one of the storage devices.

3. The system of claim 1 , wherein the one computing node is further configured to:
receive the one or more update records from a client of the distributed data store; and
write the one or more update records to the one or more storage devices.

4. The system of claim 3 , wherein the one computing node is further configured to return an acknowledgement to the client of the distribute data store that the one or more update records are written to the one or more storage devices.

5. The system of claim 1 , wherein the one computing node is further configured to:
receive another read request targeting a different one of the data blocks;
determine that a current version of the different data block is stored on the computing node;
read the current version of the different data block; and
return the current version of the different data block in response to the other read request.

6. The system of claim 5 , wherein the one computing node is further configured to generate the current version of the different data block prior to the receipt of the other read request targeting the different data block.

7. The system of claim 1 , wherein the data blocks store data stored for a network-based database service, wherein the one or more update records are one or more redo log records received from a database head node of the database service that provides clients of the database with access to the database.

8. A method, comprising:
receiving, at a first node of a plurality of nodes implementing a distributed data store, a read request targeting one of a plurality of data blocks;
in response to the receipt of the read request for the targeted data block:
determining that the computing node does not store a current version of the targeted data block;
in response to determining that the computing node does not store a current version of the targeted data block:
retrieving a previously stored version of the targeted data block;
applying, by the first node, one or more update records that are stored on the first node to the previously stored version of the targeted data block to generate the current version of the targeted data block, wherein the one or more update records specify a modification to be made to the targeted data block; and

returning the current version of the targeted data.

9. The method of claim 8 , further comprising writing the current version of the targeted data block to one or more storage devices for the first node.

10. The method of claim 8 , further comprising:
receiving the one or more update records from a client of the distributed data store; and
writing the one or more update records to the one or more storage devices.

11. The method of claim 10 , further comprising returning an acknowledgement to the client of the distribute data store that the one or more update records are written to the one or more storage devices.

12. The method of claim 8 , further comprising:
receiving another read request targeting a different one of the data blocks;
determining that a current version of the different data block is stored on the computing node;
reading the current version of the different data block; and
returning the current version of the different data block in response to the other read request.

13. The method of claim 12 , further comprising generating the current version of the different data block prior to the receipt of the other read request targeting the different data block.

14. The method of claim 8 , wherein the data blocks store data stored for a network-based database service, wherein the one or more update records are one or more redo log records received from a database head node of the database service that provides clients of the database with access to the database.

15. A system, comprising:
one or more processors; and
one or more memories, the one or more memories storing program instructions, wherein the program instructions, when executed by the one or more processors implement a log-structured storage node, wherein the log- structured storage node is configured to:
store a plurality of log records, wherein each of the plurality of log records describes a respective change to a data page of a plurality of data pages stored at the log-structured storage; and
receive a read request targeting one of the data pages;

in response to the receipt of the read request for the targeted data page:
determine that the computing node does not store a current version of the targeted data page;
in response to the determination that the computing node does not store a current version of the targeted data page:
apply one or more of the log records that are stored on the log-structured storage node to a previously stored version of the targeted data page to generate the current version of the targeted data page; and

return the current version of the targeted data page.

Assignees

• Amazon Tech Inc

Inventors

• Gupta Anurag Windlass
• Fachan Neal
• Mckelvie Samuel James
• Burchall Laurion Darrell
• Newcombe Christopher Richard
• Madhavarapu Pradeep Jnana
• Tobler Benjamin
• Corey James Mcclellan

Classifications

• G06F11/1451

  by selection of backup contents · CPC title

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F2201/80

  Database-specific techniques · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F16/23

  Updating · CPC title