Loading data with complex relationships

Patent metadata

Field	Value
Publication number	US-9607021-B2
Application number	US-201414511164-A
Country	US
Kind code	B2
Filing date	Oct 9, 2014
Priority date	Oct 10, 2013
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

Data to be loaded into a target system includes entities having corresponding unique primary data value and secondary data values. The secondary data values have associated foreign key relationships, which links the secondary data value to the primary data value of another entity. In loading the data, the entities are split into insertion leaf groups by execution of an iteration process, each iteration involving generation of a respective set of insertion leaf groups containing entities whose secondary data values have no foreign key relationships or only foreign key relationships with the primary data values of entities of the insertion leaf groups generated during any prior iteration. The iteration process is executed until there are no such entities are left. A sequence of generation of the sets of insertion leaf groups is recorded, and the sets of insertion leaf groups are consecutively loaded into the target system according to the sequence.

First claim

What is claimed is:

1. A computer implemented method for loading data into a target system having a processor, wherein the data comprises entities, each of the entities having a corresponding unique primary data value and one or more corresponding secondary data values, at least a portion of the secondary data values having associated foreign key relationships, each foreign key relationship linking the corresponding secondary data value to the corresponding primary data value of another entity, the method comprising:
splitting the entities into insertion leaf groups by execution of an iteration process by the processor, comprising execution of consecutive iterations, wherein execution of each iteration involves generation of a respective set of the insertion leaf groups, the respective set of the insertion leaf groups containing entities whose secondary data values have either no foreign key relationships or only one or more foreign key relationships with the primary data values of the entities of the insertion leaf groups generated during execution of any prior iteration, wherein the iteration process is executed until there are no entities left whose secondary data values have either no foreign key relationships or only one or more foreign key relationships with the primary data values of the one or more entities of the insertion leaf groups generated during execution of any previous iteration, wherein a sequence of generation of the sets of the insertion leaf groups is recorded; and
consecutively loading the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system according to the sequence.

2. The method of claim 1 , wherein at least a portion of the foreign key relationships are cyclic foreign key relationships, wherein the cyclic foreign key relationships link entities comprising the corresponding primary and secondary data values linked by the cyclic foreign key relationships into one or more loops,
wherein the entities linked into the one or more loops comprise one or more cycle entities, wherein when all cycle entities and the foreign key relationships linking the primary data values of all cycle entities with the corresponding secondary data values are considered being deleted from the data, the data comprises no loops of the entities any more,
wherein the method further comprises:
generating a set of the insertion cycle groups, the set of the insertion cycle groups comprising all cycle entities, wherein all cycle entities and the foreign key relationships linking primary data values of the all cycle entities with the corresponding secondary data values are considered being deleted from the data during a splitting of the entities into insertion leaf groups;
before the consecutively loading of the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system loading the entities of the set of the insertion cycle groups into the target system; and
after the consecutively loading of the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system, loading the foreign key relationships linking the secondary data values of the entities of the set of the insertion cycle groups with the corresponding primary data values into the target system.

3. The method of claim 2 , wherein the generating of the set of insertion cycle groups comprising all cycle entities is performed before the splitting of the entities into the insertion leaf groups.

4. The method of claim 2 , wherein after the splitting of the entities into the insertion leaf groups and before the loading of the entities of the set of the insertion cycle groups into the target system the generating of the set of the insertion cycle groups using the entities not yet split into the insertion leaf groups is performed,
wherein after the generating of the set of the insertion cycle groups and before the consecutively loading of the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system, the splitting of the entities into the insertion leaf groups and the recoding of the sequence of generation of the sets of the insertion leaf groups are further performed.

5. The method of claim 2 , wherein after the splitting of the entities into the insertion leaf groups and before the loading of the entities of the set of the insertion cycle groups into the target system the generating of the set of the insertion cycle groups using the entities not yet split into the insertion leaf groups is performed,
wherein after the generating of the set of the insertion cycle groups and before the consecutively loading of the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system, the following is performed:
deleting the sequence;
after the deleting of the sequence restarting of the splitting of the entities into the insertion leaf groups by execution of the iteration process comprising execution of the consecutive iterations.

6. The method of claim 1 , wherein the consecutively loading of the sets of the insertion leaf groups of the entities and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system according to the sequence comprises:
splitting the entities of one of the sets of the insertion leaf groups in at least two leaf group portions; and
loading each of the at least two leaf group portions into the target system by concurrent execution of insertion of entities of the each of the at least two leaf group portions and the foreign key relationships linking their secondary data values with the corresponding primary data values into the target system.

7. The method claim 2 , wherein the loading of the entities of the set of the insertion cycle group into the target system comprises:
splitting the entities of the set of the insertion cycle groups in at least two cycle group portions; and
loading each of the at least two cycle group portions into the target database by concurrent execution of insertion of entities of the each of the at least two cycle group portions into the target system.

8. The method claim 2 , wherein the loading of the foreign key relationships linking the secondary data values of the entities of the set of the insertion cycle groups with the corresponding primary data values into the target system comprises:
splitting the foreign key relationships linking the secondary data values of the entities of the set of the insertion cycle groups with the corresponding primary data values in at least two relationship group portions; and
loading each of the at least two relationship group portions into the target system by concurrent execution of insertion of the foreign key relationships of the each of the at least two relationship group portions into the target system.

9. The method claim 1 further comprising:
generating a directional graph, wherein each entity corresponds to a unique node of the graph, each foreign key relationship corresponding to a unique edge of the graph, wherein for a given foreign key relationship linking the respective secondary data value with the respective primary data value the graph edge begins at the node corresponding to the entity comprising the resp

Assignees

• IBM

Inventors

• Gerstl Peter
• Grasselt Mike
• Maier Albert
• Schwarz Thomas
• Suhre Oliver

Classifications

• G06F16/214 Primary

  Database migration support · CPC title

• G06F17/303 Primary

  Physics · mapped topic