Combinators

Patent metadata

Field	Value
Publication number	US-11487735-B2
Application number	US-202117318725-A
Country	US
Kind code	B2
Filing date	May 12, 2021
Priority date	Jun 19, 2009
Publication date	Nov 1, 2022
Grant date	Nov 1, 2022

Abstract

A method, according to one embodiment, includes identifying data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node; assigning an identifier and a data storage hierarchical level to the data; mapping the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node; receiving a plurality of write operations; and combining a plurality of write tasks of the predetermined portion for a predetermined time period.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
identifying data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node;
assigning an identifier and a data storage hierarchical level to the data;
mapping the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node;
receiving a plurality of write operations; and
combining a plurality of write tasks of the predetermined portion for a predetermined time period.

2. The computer-implemented method of claim 1 , comprising delaying one or more of the plurality of write operations to collect additional write operations designated to the predetermined portion.

3. The computer-implemented method of claim 1 , wherein the predetermined time period includes a time period needed to collect a total amount of write operations that meet a predetermined file size.

4. The computer-implemented method of claim 1 , comprising:
designating a merging limit value to the one or more tables on the predetermined portion, wherein the merging limit value is indicative of one of a plurality of data storage hierarchical levels; and
limiting a merging of files in accordance with the merging limit value by refraining from merging data files in the one of the plurality of data storage hierarchical levels once the merging limit value is met.

5. The computer-implemented method of claim 1 , comprising:
storing a bloom filter for a plurality of row keys for each of a plurality of data storage hierarchical levels; and
consulting the bloom filter to determine whether one of the plurality of row keys is found in a given data storage hierarchical level of the plurality of data storage hierarchical levels.

6. The computer-implemented method of claim 1 , comprising metering the combined plurality of write tasks.

7. The computer-implemented method of claim 1 , wherein the data storage hierarchical level includes a first hierarchical level indicative of oldest data in the one or more tables on the predetermined portion.

8. The computer-implemented method of claim 7 , wherein the data storage hierarchical level includes a second hierarchical level indicative of newer stored data in the one or more tables on the predetermined portion.

9. The computer-implemented method of claim 8 , wherein the data storage hierarchical level includes a third hierarchical level indicative of most recently stored data in the one or more tables on the predetermined portion.

10. The computer-implemented method of claim 9 , wherein the plurality of write tasks are written to the third hierarchical level, data stored in the third hierarchical level is merged to the second hierarchical level after a first predetermined time period, and the data stored in the second hierarchical level is merged to the first hierarchical level after a second predetermined time period.

11. A computer program product for organizing data in a database system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:
identifying, utilizing the processor, data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node;
assigning, utilizing the processor, an identifier and a data storage hierarchical level to the data;
mapping, utilizing the processor, the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node;
receiving, utilizing the processor, a plurality of write operations; and
combining, utilizing the processor, a plurality of write tasks of the predetermined portion for a predetermined time period.

12. The computer program product of claim 11 , comprising delaying one or more of the plurality of write operations to collect additional write operations designated to the predetermined portion.

13. The computer program product of claim 11 , wherein the predetermined time period includes a time period needed to collect a total amount of write operations that meet a predetermined file size.

14. The computer program product of claim 11 , comprising:
designating a merging limit value to the one or more tables on the predetermined portion, wherein the merging limit value is indicative of one of a plurality of data storage hierarchical levels; and
limiting a merging of files in accordance with the merging limit value by refraining from merging data files in the one of the plurality of data storage hierarchical levels once the merging limit value is met.

15. The computer program product of claim 11 , comprising:
storing a bloom filter for a plurality of row keys for each of a plurality of data storage hierarchical levels; and
consulting the bloom filter to determine whether one of the plurality of row keys is found in a given data storage hierarchical level of the plurality of data storage hierarchical levels.

16. The computer program product of claim 11 , comprising metering the combined plurality of write tasks.

17. The computer program product of claim 11 , wherein the data storage hierarchical level includes a first hierarchical level indicative of oldest data in the one or more tables on the predetermined portion.

18. The computer program product of claim 17 , wherein the data storage hierarchical level includes a second hierarchical level indicative of newer stored data in the one or more tables on the predetermined portion.

19. The computer program product of claim 18 , wherein the data storage hierarchical level includes a third hierarchical level indicative of most recently stored data in the one or more tables on the predetermined portion.

20. A system, comprising:
a processor; and
logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to:
identify data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node;
assign an identifier and a data storage hierarchical level to the data;
map the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node;
receive a plurality of write operations; and

combine a plurality of write tasks of the predetermined portion for a predetermined time period.

Assignees

• IBM

Inventors

• Skrenta Rich
• Dole Bryn Robert
• Markson Michael
• Peters Keith
• Saliba Robert Michael
• Truel Robert N
• Lindahl Gregory B

Classifications

• G06F16/24554

  Unary operations; Data partitioning operations · CPC title

• G06F16/2282 Primary

  Tablespace storage structures; Management thereof · CPC title

• G06F16/273 Primary

  Asynchronous replication or reconciliation · CPC title

• G06F16/2462

  Approximate or statistical queries · CPC title

• G06F16/951

  Indexing; Web crawling techniques · CPC title