Native copy mechanism for in-memory and paged primitive data structures

Patent metadata

Field	Value
Publication number	US-12524428-B2
Application number	US-202519044325-A
Country	US
Kind code	B2
Filing date	Feb 3, 2025
Priority date	Nov 8, 2023
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

A database system detects a query targeting a first database object stored in a unified persistence format in a persistence store. In response to detecting the query, the database system determines whether to load the first database object in column-loadable format or page-loadable format into an in-memory store. In response to determining to load the first database object in the column-loadable format, the database system determines if a first bit representation of one or more corresponding primitives in the persistence store is compatible with a second bit representation used for storing the primitives in the in-memory store. The database system performs a native copy operation to copy the primitives from the persistence store to the in-memory store if the bit representations are compatible, where the native copy operation involves copying the primitives on a page-by-page basis without decoding and encoding values of the one or more primitives.

First claim

What is claimed:

1 . A method performed by a database system, comprising:
detecting a load operation for loading a first database object into an in-memory store in a column-loadable format; responsive to detecting the load operation:
accessing page-based metadata associated with the first database object in a persistence store, and accessing column-based metadata associated with the in-memory store;
comparing the page-based metadata to the column-based metadata;
determining, based on the comparison of the page-based metadata to the column-based metadata, whether an in-memory representation of the first database object will be compatible with a persistence store representation of the first database object; and responsive to determining that the in-memory representation of the first database object will be compatible with the persistence store representation of the first database object, performing a native copy operation to copy one or more paged primitives representing the first database object into the in-memory store.

2 . The method of claim 1 , wherein the native copy operation comprises copying the one or more paged primitives on a page-by-page basis without decoding and encoding values of the one or more paged primitives.

3 . The method of claim 1 , wherein determining if the in-memory representation of the first database object will be compatible with the persistence store representation of the first database object is performed on a per-primitive basis.

4 . The method of claim 1 , further comprising determining if a first number of rows per column used for the in-memory representation are equal to a second number of rows per column used for the persistence store representation.

5 . The method of claim 1 , further comprising determining if a first number of bits per row element used for the in-memory representation are equal to a second number of bits per row element used for the persistence store representation.

6 . The method of claim 1 , wherein responsive to determining that the in-memory representation of the first database object will be incompatible with the persistence store representation of the first database object, the method further comprising performing a non-native copy operation to copy the one or more paged primitives representing the first database object into the in-memory store.

7 . The method of claim 1 , further comprising locating a contiguous memory buffer that stores the one or more paged primitives.

8 . The method of claim 1 , wherein the one or more paged primitives comprise a paged bit vector.

9 . The method of claim 1 , wherein the one or more paged primitives comprise a paged index vector.

10 . The method of claim 9 , wherein the paged index vector is a bit-compressed index vector.

11 . A system, comprising:
at least one processor; and at least one memory including program instructions which when executed by the at least one processor causes operations comprising:
detecting a load operation for loading a first database object into an in-memory store in a column-loadable format;
responsive to detecting the load operation:
accessing page-based metadata associated with the first database object in a persistence store, and accessing column-based metadata associated with the in-memory store;
comparing the page-based metadata to the column-based metadata;
determining, based on the comparison of the page-based metadata to the column-based metadata, whether an in-memory representation of the first database object will be compatible with a persistence store representation of the first database object; and
responsive to determining that the in-memory representation of the first database object will be compatible with the persistence store representation of the first database object, performing a native copy operation to copy one or more paged primitives representing the first database object into the in-memory store.

12 . The system of claim 11 , wherein the native copy operation comprises copying the one or more paged primitives on a page-by-page basis without decoding and encoding values of the one or more paged primitives.

13 . The system of claim 11 , wherein determining if the in-memory representation of the first database object will be compatible with the persistence store representation of the first database object is performed on a per-primitive basis.

14 . The system of claim 11 , wherein the operations further comprise determining if a first number of rows per column used for the in-memory representation are equal to a second number of rows per column used for the persistence store representation.

15 . The system of claim 11 , wherein the operations further comprise determining if a first number of bits per row element used for the in-memory representation are equal to a second number of bits per row element used for the persistence store representation.

16 . The system of claim 11 , wherein responsive to determining that the in-memory representation of the first database object will be incompatible with the persistence store representation of the first database object, the operations further comprise performing a non-native copy operation to copy the one or more paged primitives representing the first database object into the in-memory store.

17 . The system of claim 11 , wherein the operations further comprise locating a contiguous memory buffer that stores the one or more paged primitives.

18 . The system of claim 11 , wherein the one or more paged primitives comprise a paged bit vector.

19 . The system of claim 11 , wherein the one or more paged primitives comprise a paged index vector.

20 . A non-transitory computer readable storage medium storing instructions, which when executed by at least one data processor, cause operations comprising:
detecting a load operation for loading a first database object into an in-memory store in a column-loadable format; responsive to detecting the load operation:
accessing page-based metadata associated with the first database object in a persistence store, and accessing column-based metadata associated with the in-memory store;
comparing the page-based metadata to the column-based metadata;
determining, based on the comparison of the page-based metadata to the column-based metadata, whether an in-memory representation of the first database object will be compatible with a persistence store representation of the first database object; and responsive to determining that the in-memory representation of the first database object will be compatible with the persistence store representation of the first database object, performing a native copy operation to copy one or more paged primitives representing the first database object into the in-memory store.

Assignees

• Sap Se

Inventors

• Maniyar Anish
• Kumar Santhosh T
• Ye Guo Gang
• Pathak Amit
• Seifert Sebastian
• Pal Souvik

Classifications

• G06F16/221

  Column-oriented storage; Management thereof · CPC title

• G06F16/254 Primary

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title