Handling large writes to distributed logs

Patent metadata

Field	Value
Publication number	US-10360145-B2
Application number	US-201615180816-A
Country	US
Kind code	B2
Filing date	Jun 13, 2016
Priority date	Jun 13, 2016
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

A system includes writing of log data in chunks over a first range of heap log sequence positions of a heap log stored in the non-volatile memory system, and writing of a heap anchor marker in a first anchor stream of a transaction log stored in the non-volatile memory system, at a first transaction log sequence position of the transaction log, wherein the first anchor stream is associated with a first stream of the transaction log, and the heap anchor marker identifies the first range of heap log sequence positions of the heap log.

First claim

What is claimed is:

1. A system comprising:
a non-volatile memory system; and
one or more processors to execute processor-executable process steps to cause the system to:
write log data in chunks, by a log writer, over a first range of heap log sequence positions of a heap log stored in the non-volatile memory system; and
write a heap anchor marker, by a heap log writer, in a first anchor stream of a transaction log stored in the non-volatile memory system, at a first transaction log sequence position of the transaction log,
wherein the first anchor stream is associated with a first stream of the transaction log, and the heap anchor marker identifies the first range of heap log sequence positions of the heap log.

2. The system according to claim 1 , the one or more processors to execute processor-executable process steps to cause the system to:
write a heap creation marker to a garbage collection stream at a second transaction log sequence position of the transaction log, the heap creation marker identifying the first stream of the first transaction log,
wherein writing the heap anchor marker in the first anchor stream comprises writing the heap anchor marker in the garbage collection stream at the first transaction log sequence position.

3. The system according to claim 1 , the one or more processors to execute processor-executable process steps to cause the system to:
write second log data in chunks over a second range of heap log sequence positions of the heap log,
wherein writing the heap anchor marker in the first anchor stream comprises writing a second heap anchor marker in a second anchor stream of the transaction log, at the first transaction log sequence position of the transaction log,
wherein the second anchor stream is associated with a second stream of the transaction log, and the second heap anchor marker identifies the second range of heap log sequence positions of the heap log.

4. The system according to claim 3 , the one or more processors to execute processor-executable process steps to cause the system to:
write a first heap creation marker and a second heap creation marker to a garbage collection stream at a second transaction log sequence position of the transaction log, the heap creation marker identifying the first stream of the first transaction log and the second heap creation marker identifying the second stream of the first transaction log,
wherein writing the heap anchor marker in the first anchor stream comprises writing the heap anchor marker in the garbage collection stream at the first transaction log sequence position, and
wherein writing the second heap anchor marker in the second anchor stream comprises writing the second heap anchor marker in the garbage collection stream at the first transaction log sequence position.

5. The system according to claim 1 , the one or more processors to execute processor-executable process steps to cause the system to:
write second log data in chunks over a second range of heap log sequence positions of the heap log; and
write a second heap anchor marker in a second anchor stream of the transaction log, at a second transaction log sequence position of the transaction log,
wherein the second anchor stream is associated with a second stream of the transaction log, and the second heap anchor marker identifies the second range of heap log sequence positions of the heap log.

6. The system according to claim 5 , the one or more processors to execute processor-executable process steps to cause the system to:
write a heap creation marker to a garbage collection stream at a second transaction log sequence position of the transaction log, the heap creation marker identifying the first stream of the first transaction log; and
write a second heap creation marker to the garbage collection stream at a third transaction log sequence position of the transaction log, the second heap creation marker identifying the second stream of the first transaction log,
wherein writing the heap anchor marker in the first anchor stream comprises writing the heap anchor marker in the garbage collection stream at the first transaction log sequence position, and
wherein writing the second heap anchor marker in the second anchor stream comprises writing the second heap anchor marker in the garbage collection stream at the second transaction log sequence position.

7. A method comprising:
writing, by a log writer, log data in chunks over a first range of heap log sequence positions of a heap log stored in a non-volatile memory system; and
writing, by a heap log writer, a heap anchor marker in a first anchor stream of a transaction log stored in the non-volatile memory system, at a first transaction log sequence position of the transaction log,
wherein the first anchor stream is associated with a first stream of the transaction log, and the heap anchor marker identifies the first range of heap log sequence positions of the heap log.

8. The method according to claim 7 , further comprising:
writing a heap creation marker to a garbage collection stream at a second transaction log sequence position of the transaction log, the heap creation marker identifying the first stream of the first transaction log,
wherein writing the heap anchor marker in the first anchor stream comprises writing the heap anchor marker in the garbage collection stream at the first transaction log sequence position.

9. The method according to claim 8 , further comprising:
writing second log data in chunks over a second range of heap log sequence positions of the heap log,
wherein writing the heap anchor marker in the first anchor stream comprises writing a second heap anchor marker in a second anchor stream of the transaction log, at the first transaction log sequence position of the transaction log,
wherein the second anchor stream is associated with a second stream of the transaction log, and the second heap anchor marker identifies the second range of heap log sequence positions of the heap log.

10. The method according to claim 9 , further comprising:
writing a first heap creation marker and a second heap creation marker to a garbage collection stream at a second transaction log sequence position of the transaction log, the heap creation marker identifying the first stream of the first transaction log and the second heap creation marker identifying the second stream of the first transaction log,
wherein writing the heap anchor marker in the first anchor stream comprises writing the heap anchor marker in the garbage collection stream at the first transaction log sequence position, and
wherein writing the second heap anchor marker in the second anchor stream comprises writing the second heap anchor marker in the garbage collection stream at the first transaction log sequence position.

11. The method according to claim 7 , further comprising:
writing second log data in chunks over a second range of heap log sequence positions of the heap log; and
writing a second heap anchor marker in a second anchor stream of the transaction log, at a second transaction log sequence position of the transaction log,
wherein the second anchor stream is associated with a second stream of the transaction log, and the second heap anchor marker identifies the second range of heap log sequence positions of the heap log.

12. The method according to claim 11 , further comprising:
writing a heap creation marker to a garbage collection stream at a second transaction log sequence position of the transaction log, the heap creation marker identifying the first stream of the first transaction log; and
writing a second heap creation marker to the garbage coll

Assignees

• Sap Se

Inventors

• Pound Jeffrey
• Auch Nathan
• Bumbulis Peter
• Goel Anil Kumar
• Maclean Scott
• Garber Eric

Classifications

• G06F12/0253 Primary

  Garbage collection, i.e. reclamation of unreferenced memory · CPC title

• G06F11/1474

  in transactions (updating of structured data in databases G06F16/23) · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title