Dual data storage using an in-memory array and an on-disk page structure

Patent metadata

Field	Value
Publication number	US-9891831-B2
Application number	US-201414553859-A
Country	US
Kind code	B2
Filing date	Nov 25, 2014
Priority date	Nov 25, 2014
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

The subject matter disclosed herein provides methods for the dual storage of data using an in-memory array and an on-disk page structure. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. Random access can be provided to a subset of the one or more rows by at least loading the subset of rows from the one or more pages to the in-memory array without loading all of the rows from the one or more pages. Related apparatus, systems, techniques, and articles are also described.

First claim

What is claimed is:

1. A method for implementation by one or more data processors forming part of at least one computing system, the method comprising:
storing an in-memory array to disk, the in-memory array including a portion of data from a column of data, and the in-memory array being stored to disk by at least:
determining, based at least on a content of the portion of data, a page on disk for storing the portion of data;
storing, in the page, the portion of data from the column of data, the portion of data being stored in a row in the page; and

providing random access to the portion of data from the column of data, the providing of the random access comprising:
identifying, based at least on the content of the portion of data, the page storing the portion of data; and
loading, from disk and into the in-memory array, the row from the page storing the portion of data, the row being loaded without loading another row from the page.

2. The method of claim 1 , wherein the in-memory array has a contiguous block of memory addresses, and wherein the contiguous block of memory addresses supports random access of the data when all of the data is loaded into the in-memory array.

3. The method of claim 1 , wherein the column of data in the in-memory array includes one or more value identifiers.

4. The method of claim 3 , wherein the one or more value identifiers are encoded in a native format, and wherein the row and/or the other row from the page store the one or more value identifiers in the native format.

5. The method of claim 4 , wherein the native format uses an N-bit encoding scheme.

6. The method of claim 3 , wherein the row and/or the other row from the page do not store any data for the column when the one or more value identifiers are null values.

7. The method of claim 3 , wherein the row and/or the other row from the page do not store any data for the column when an identity property applies to the column, and wherein the identity property applies to the column when each value identifier is equal in value to a row position associated with the value identifier.

8. The method of claim 1 , wherein the loading is performed when the computing system is being restored.

9. A non-transitory computer-readable medium containing instructions to configure a processor to perform operations comprising:
storing an in-memory array to disk, the in-memory array including a portion of data from a column of data, and the in-memory array being stored to disk by at least:
determining, based at least on a content of the portion of data, a page on disk for storing the portion of data;
storing, in the page, the portion of data from the column of data, the portion of data being stored in a row in the page; and

providing random access to the portion of data from the column of data, the providing of the random access comprising:
identifying, based at least on the content of the portion of data, the page storing the portion of data; and
loading, from disk and into the in-memory array, the row from the page storing the portion of data, the row being loaded without loading another row from the page.

10. The non-transitory computer-readable medium of claim 9 , wherein the in-memory array has a contiguous block of memory addresses, and wherein the contiguous block of memory addresses supports random access of the data when all of the data is loaded into the in-memory array.

11. The non-transitory computer-readable medium of claim 9 , wherein the column of data in the in-memory array includes one or more value identifiers.

12. The non-transitory computer-readable medium of claim 11 , wherein the one or more value identifiers are encoded in a native format, and wherein the row and/or the other row from the page store the one or more value identifiers in the native format.

13. The non-transitory computer-readable medium of claim 11 , the row and/or the other row from the page do not store any data for the column when the one or more value identifiers are null values.

14. The non-transitory computer-readable medium of claim 11 , wherein the row and/or the other row from the page do not store any data for the column when an identity property applies to the column, and wherein the identity property applies to the column when each value identifier is equal in value to a row position associated with the value identifier.

15. A system comprising:
at least one processor; and
at least one memory, wherein the at least one processor and the at least one memory are configured to perform operations comprising:
storing an in-memory array to a disk, the in-memory array including a portion of data from a column of data, and the in-memory array being stored to disk by at least:
determining, based at least on a content of the portion of data, a page on disk for storing the portion of data;
storing, in the page, the portion of data from the column of data, the portion of data being stored in a row in the page; and
providing random access to the portion of data from the column of data, the providing of the random access comprising:
identifying, based at least on the content of the portion of data, the page storing the portion of data; and
loading, from the disk and into the in-memory array, the row from the page storing the portion of data, the row being loaded without loading another row from the page.

16. The system of claim 15 , wherein the in-memory array has a contiguous block of memory addresses, and wherein the contiguous block of memory addresses supports random access of the data when all of the data is loaded into the in-memory array.

17. The system of claim 15 , wherein the column of data in the in-memory array includes one or more value identifiers.

18. The system of claim 17 , wherein the one or more value identifiers are encoded in a native format, and wherein the row and/or the other row from the page store the one or more value identifiers in the native format.

19. The system of claim 17 , wherein the row and/or the other row from the page do not store any data for the column when the one or more value identifiers are null values.

20. The system of claim 17 , wherein the row and/or the other row from the page do not store any data for the column when an identity property applies to the column, and wherein the identity property applies to the column when each value identifier is equal in value to a row position associated with the value identifier.

21. The method of claim 1 , wherein the page for storing the portion of data is determined and/or identified based at least on a hash of the content of the portion of data.

Assignees

• Legler Thomas
• Wein David
• Schreter Ivan
• Andrei Mihnea
• Goel Anil Kumar
• Radestock Guenter
• Sap Se

Inventors

• Legler Thomas
• Wein David
• Schreter Ivan
• Andrei Mihnea
• Goel Anil Kumar
• Radestock Guenter

Classifications

• G06F3/0605 Primary

  by facilitating the interaction with a user or administrator · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F16/90339

  by using parallel associative memories or content-addressable memories · CPC title

• G06F16/221 Primary

  Column-oriented storage; Management thereof · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title