Mirroring, in memory, data from disk to improve query performance

Patent metadata

Field	Value
Publication number	US-9292564-B2
Application number	US-201414337179-A
Country	US
Kind code	B2
Filing date	Jul 21, 2014
Priority date	Sep 21, 2013
Publication date	Mar 22, 2016
Grant date	Mar 22, 2016

Abstract

Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

First claim

What is claimed is:

1. A method comprising:
maintaining, on persistent storage, a database that is accessible to a database server;
wherein the database includes a set of persistent format data (PF data), stored on the persistent storage, in a persistent format;
converting the set of PF data to a mirror format to produce a set of mirror format data (MF data);
storing the set of MF data within volatile memory;
wherein the mirror format is different from and independent of the persistent format;
wherein, within the volatile memory, the set of MF data is accessible to the database server;
wherein each data item in the set of MF data is a copy of a corresponding data item in the set of PF data;
within the volatile memory, the database server maintaining the set of MF data transactionally consistent with the set of PF data as the database server executes database operations that make changes to the set of PF data;
receiving, by the database server, a request to execute a database command that requires both a first set of data items and a second set of data items;
in response to receiving the request, the database server determining whether obtaining the first set of data items, for execution of the database command, from the set of PF data would be more efficient than obtaining the first set of data items, for execution of the database command, from the set of MF data; and
during execution of the database command:
responsive to the database server determining that obtaining the first set of data items from the set of PF data would be more efficient, the database server obtaining the first set of data items from the set of PF data in the persistent format, and not from the set of MF data in volatile memory in the mirror format, and
the database server obtaining the second set of data items from the set of MF data in volatile memory in the mirror format, and not from the set of PF data in the persistent format.

2. The method of claim 1 further comprising:
in response to receiving the request, the database server determining whether obtaining the second set of data items, for execution of the database command, from the set of MF data would be more efficient than obtaining the second set of data items, for execution of the database command, from the set of PF data; and
wherein obtaining the second set of data items from the set of MF data in volatile memory in the mirror format, and not from the set of PF data in the persistent format, is responsive to the database server determining that obtaining the second set of data items from the set of MF data would be more efficient than obtaining the second set of data items, for execution of the database command, from the set of PF data second set of data items.

3. The method of claim 2 wherein:
within the database, data items that belong to the set of PF data are stored in a particular order; and
the method further comprises, within volatile memory, ordering data items that belong to the set of MF data in a manner that reflects said particular order.

4. The method of claim 1 wherein the mirror format is a column-major format.

5. The method of claim 4 wherein the persistent format is a row-major format.

6. The method of claim 2 further comprising: the database server determining that obtaining the first set of data items from the set of PF data would be more efficient, and that obtaining the second set of data items from the set of MF data would be more efficient.

7. The method of claim 1 further comprising:
maintaining, in the volatile memory, a cache of data items that are in the persistent format; and
wherein the database server determining whether obtaining the first set of data items from the set of PF data would be more efficient than obtaining the first set of data items from the set of MF data is based, at least in part, on whether the first set of data items currently reside in the cache.

8. The method of claim 1 wherein the step of maintaining the set of MF data transactionally consistent with the set of PF data includes implicitly making particular changes to data items that belong to the set of MF data, without directly making the particular changes to data items that belong to the set of MF data, by recording the particular changes in one or more journals stored in the volatile memory.

9. The method of claim 8 further comprising:
in response to a change, made as part of a transaction, to a data item that belongs to the set of PF data, performing the steps of:
before the transaction is committed, storing a record of the change in a private journal that is only visible to the transaction; and
in response to the transaction committing, moving the record of the change to a global journal that is visible to other transactions.

10. The method of claim 9 further comprising merging at least some changes indicated in the global journal into the set of MF data.

11. The method of claim 10 wherein the mirror format is a compressed format and the step of merging comprises:
decompressing at least a portion of the set of MF data to produce uncompressed data;
updating the uncompressed data to produce updated uncompressed data that reflects said at least some changes; and
compressing the updated uncompressed data.

12. The method of claim 1 wherein the mirror format is a compressed format.

13. The method of claim 1 wherein:
the database command requires data items, from the database, that satisfy certain criteria;
the method further comprises:
in response to receiving the request, the database server performing the steps of:
using the set of MF data to determine which data items, from the database, satisfy the certain criteria, and
obtaining the data items that satisfy the certain criteria from the database.

14. The method of claim 13 wherein:
the set of MF data includes a column vector for a particular column;
the certain criteria includes whether values from the particular column satisfy a condition; and
using the set of MF data to determine which data items, from the database, satisfy the certain criteria comprises identifying which values, within the column vector, satisfy the condition.

15. The method of claim 1 wherein:
a first portion of the set of MF data is compressed at a first level; and
a second portion of set of MF data is compressed at a second level that is different from the first level.

16. The method of claim 15 wherein the first level and the second level are specified by a user.

17. The method of claim 15 wherein the first level and the second level are automatically determined based on access patterns to the set of MF data.

18. The method of claim 1 , wherein the first and second sets of data items both reside in both the set of PF data and the set of MF data.

19. The method of claim 1 , wherein:
the database command requires a particular data item;
the set of PF data in persistent storage includes a first copy of the particular data item that is in the persistent format;
the set of MF data includes a second copy of the particular data item that is in the mirror format;
a data cache, residing in the volatile memory, includes a third copy of the particular data item that is in the persistent format; and
the method further comprises, during execution of the database command, the database server obtaining the third copy of the particular data item from the data cache in the persistent format, and not from the set of MF data or the set of PF data in persistent storage.

20. A system comprisi

Assignees

• Oracle Int Corp

Inventors

• Kamp Jesse
• Ganesh Amit
• Marwah Vineet
• Raja Vivekanandhan
• Lahiri Tirthankar
• Holloway Allison L
• Hase Sanket
• Chavan Shasank K
• Mukherjee Niloy
• Lee Teck Hua
• Gleeson Michael J
• Kunchithapadam Krishna

Classifications

• G06F3/0665

  at area level, e.g. provisioning of virtual or logical volumes · CPC title

• G06F16/2453

  Query optimisation · CPC title

• G06F16/27

  Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F2201/855

  Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes · CPC title