Data locality in data integration applications

Patent metadata

Field	Value
Publication number	US-9607062-B1
Application number	US-201514945476-A
Country	US
Kind code	B1
Filing date	Nov 19, 2015
Priority date	Nov 19, 2015
Publication date	Mar 28, 2017
Grant date	Mar 28, 2017

Abstract

A computer-implemented method includes identifying a logical node. The logical node is associated with one or more source stages. The computer-implemented method further includes identifying one or more file block components. The one or more file block components include a retrieval target for at least one of the one or more source stages. The computer-implemented method further includes identifying one or more physical nodes and determining, for each of the one or more physical nodes, a degree value. A corresponding computer program product and computer system are also disclosed.

First claim

What is claimed is:

1. A computer-implemented method comprising:
identifying a logical node, said logical node being associated with one or more source stages;
identifying one or more file block components, said one or more file block components comprising a retrieval target for at least one of said one or more source stages;
identifying one or more physical nodes;
determining, for each of said one or more physical nodes, a degree value;
identifying one or more qualified physical nodes from said one or more physical nodes having said degree value of one or more;
creating a preferred physical node table, said preferred physical node table comprising, for each of said one or more qualified physical nodes, an identifying indication and an indication of said degree value; and
sorting said preferred physical node table based on said degree value associated with each of said one or more qualified physical nodes.

2. A computer-implemented method comprising:
identifying a logical node, said logical node being associated with one or more source stages;
identifying one or more file block components, said one or more file block components comprising a retrieval target for at least one of said one or more source stages;
identifying one or more physical nodes;
determining, for each of said one or more physical nodes, a degree value;
determining a candidate preferred physical node based on each said degree value;
determining whether said candidate preferred physical node is available for allocation to said logical node; and
responsive to said candidate preferred physical node being available for allocation to said logical node, allocating said candidate preferred physical node to said logical node.

3. The computer-implemented method of claim 2 , further comprising:
responsive to said candidate preferred physical node not being available for allocation to said logical node:
marking said candidate preferred physical node as unavailable for allocation to said logical node; and
determining an alternative candidate preferred physical node based on each said degree value.

4. A computer program product comprising one or more computer readable storage media and program instructions stored on said one or more computer readable storage media, said program instructions comprising instructions to:
identify a logical node, said logical node being associated with one or more source stages;
identify one or more file block components, said one or more file block components comprising a retrieval target for at least one of said one or more source stages;
identify one or more physical nodes;
determine, for each of said one or more physical nodes, a degree value;
identify one or more qualified physical nodes from said one or more physical nodes having said degree value of one or more;
create a preferred physical node table, said preferred physical node table comprising, for each of said one or more qualified physical nodes, an identifying indication and an indication of said degree value; and
sort said preferred physical node table based on said degree value associated with each of said one or more qualified physical nodes.

5. The computer program product of claim 4 , wherein said program instructions further comprise instructions to:
determine a candidate preferred physical node based on each said degree value.

6. The computer program product of claim 5 , wherein said program instructions further comprise instructions to:
determine whether said candidate preferred physical node is available for allocation to said logical node; and
responsive to said candidate preferred physical node being available for allocation to said logical node, allocate said candidate preferred physical node to said logical node.

7. The computer program product of claim 6 , wherein said program instructions further comprise instructions to:
responsive to said candidate preferred physical node not being available for allocation to said logical node:
mark said candidate preferred physical node as unavailable for allocation to said logical node; and
determine an alternative candidate preferred physical node based on each said degree value.

Assignees

• IBM

Inventors

• Bonagiri Krishna K
• Jacobson Eric A

Classifications

• G06F16/182

  Distributed file systems · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/254 Primary

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F16/245

  Query processing · CPC title

• G06F16/183

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title