Parallel I/O read processing for use in clustered file systems having cache storage

Patent metadata

Field	Value
Publication number	US-9667736-B2
Application number	US-201414265176-A
Country	US
Kind code	B2
Filing date	Apr 29, 2014
Priority date	Apr 29, 2014
Publication date	May 30, 2017
Grant date	May 30, 2017

Abstract

In one embodiment, a method for providing parallel reading in a clustered file system having cache storage includes using an owner gateway node to read data for a fileset, determining whether to utilize other gateway nodes to handle a portion of read traffic for the fileset, selecting a set of eligible gateway nodes based on: a current internal workload, a network workload, and recent performance history data regarding workload distribution across the other gateway nodes, assigning and defining a size for read task items for each gateway node in the set based on a current dynamic profile of each gateway node in the set, providing in-memory and/or input/output resources at each gateway node in the set to handle assigned read task items, and distributing workload to the set of eligible gateway nodes according to the size for the assigned read task items for each gateway node in the set.

First claim

What is claimed is:

1. A computer program product for providing parallel reading in a clustered file system having cache storage, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by a processor to cause the processor to:
read, by the processor, data for a first fileset using an owner gateway (GW) node;
determine, by the processor, whether to utilize one or more other GW nodes to handle at least a portion of read traffic for the first fileset;
select, by the processor, a set of eligible GW nodes based on at least one of: a current internal workload, a network workload, and recent performance history data in regard to workload distribution across the one or more other GW nodes;
assign and define, by the processor, a size for one or more read task items for each GW node in the set of eligible GW nodes based on a current dynamic profile of each individual GW node in the set of eligible GW nodes;
provide, ensure, or provide and ensure availability to, by the processor, in-memory resources, input/output (I/O) resources, or in-memory and I/O resources at each GW node in the set of eligible GW nodes to handle one or more assigned read task items;
distribute, by the processor, workload to the set of eligible GW nodes according to the size for each of the one or more assigned read task items for each individual GW node in the set of eligible GW nodes;
receive, by the processor, indication of the one or more read task items to be split;
determine, by the processor, the one or more GW nodes to split each of the one or more read task items across individually;
remove, by the processor, any GW node from the set of eligible GW nodes when an error occurs at a GW node during read task processing; and
reassign, by the processor, any read task items that were assigned to a GW node that was removed from the set of eligible GW nodes to another GW node from the set of eligible GW nodes.

2. The computer program product as recited in claim 1 , wherein the program code configured to determine whether to utilize the one or more other GW nodes to handle the portion of the read traffic for the first fileset is performed as one of: a background process periodically, and a foreground process before initiating a large I/O request according to a predetermined I/O size threshold.

3. The computer program product as recited in claim 1 , wherein the program code is further configured to determine, by the processor, an amount of workload corresponding to the first fileset at the owner GW node based on information selected from a group comprising: an amount of data present in I/O requests that are queued for the first fileset, an amount of data in outstanding I/O requests for the first fileset that have been issued, an amount of data in outstanding I/O requests for the first fileset that have been received but not yet completed, an amount of data in outstanding I/O requests for the first fileset that have been issued and received but not yet completed, an I/O workload at one or more other GW nodes due to traffic from one or more other filesets, and an I/O workload at the one or more other GW nodes that are functioning as an application node serving local applications.

4. The computer program product as recited in claim 3 , wherein the program code is further configured to determine, by the processor, the amount of workload corresponding to the first fileset for other GW nodes which have connectivity with the owner GW node.

5. The computer program product as recited in claim 1 , wherein the program code is further configured to:
receive, by the processor, a remote block read request for an uncached file from an application node;
determine, by the processor, whether the remote block read request is sent in response to a foreground read request or initiation of a manual file prefetch;
determine, by the processor, whether a size of the requested file exceeds a parallel read threshold when the remote block read request is sent in response to the manual file prefetch and invoke a parallel I/O code path with handling of sparse blocks and small read scenarios when the size of the requested file exceeds the parallel read threshold; and
determine, by the processor, whether at least two blocks of the requested file have been cached when the remote block read request is sent in response to the foreground read request and invoke the parallel I/O code path with handling of sparse blocks and small read scenarios when at least two blocks of the requested file have been cached and a percentage of cached blocks of the requested file versus all blocks of the requested file exceeds a predetermined prefetch threshold.

6. The computer program product as recited in claim 1 , wherein the program code is further configured to:
determine, by the processor, each GW node eligible to process a read task and including the eligible GW nodes in a GW node list;
determine, by the processor, a home node that corresponds to each GW node in the GW node list;
create, by the processor, individual home node GW lists for each home node, wherein each GW node list comprises a set of GW nodes which share a same home node;
determine, by the processor, a peer GW eligibility value for each GW node in the GW node list;
determine, by the processor, a GW node having a highest peer GW eligibility value for each home node; and
remove, by the processor, all other GW nodes which do not have the highest peer GW eligibility value for each home node from the GW node list.

7. The computer program product as recited in claim 1 , wherein the program code is further configured to:
receive, by the processor, a task response from a slave GW node corresponding to the first fileset, the task response indicating at least a read start value and an end offset value;
retrieve, by the processor, a cached block bitmap of the first fileset, the cached block bitmap being an indication of which portions of the first fileset are stored to the cache storage;
determine, by the processor, whether, for each read request waiting for remote read on the first fileset, all blocks of the first fileset are stored in the cache storage;
send, by the processor, a response to the slave GW node indicating that caching has been performed for the first fileset when all blocks of the first fileset are stored in the cache storage; and
send, by the processor, a request to cache portions of the first fileset which are not currently in the cache storage when all blocks of the first fileset are not stored in the cache storage.

8. A method for providing parallel reading in a clustered file system having cache storage, the method comprising:
using an owner gateway (GW) node to read data for a first fileset;
determining whether to utilize one or more other GW nodes to handle at least a portion of read traffic for the first fileset;
selecting a set of eligible GW nodes based on at least one of: a current internal workload, a network workload, and recent performance history data in regard to workload distribution across the one or more other GW nodes;
assigning and defining a size for one or more read task items for each GW node in the set of eligible GW nodes based on a current dynamic profile of each individual GW node in the set of eligible GW nodes;
providing, ensuring, or providing and ensuring availability to in-memory resources, input/output (I/O) resources, or in-memory and I/O resources at each GW node in the set of eligible GW nodes to handle one or more assigned read task items;
distributing workload to the set of eligible GW nodes according to the size for each of the one or more assigned read task items for each individual GW node in the set of eligible GW node

Assignees

• IBM

Inventors

• Gunda Kalyan C
• Hildebrand Dean
• Naik Manoj P
• Shiraguppi Riyazahamad M

Classifications

• G06F17/30203

  Physics · mapped topic

• H04L67/2842 Primary

  Electricity · mapped topic

• H04L67/101

  based on network conditions · CPC title

• H04L67/1008

  based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) · CPC title

• H04L67/568 Primary

  Storing data temporarily at an intermediate stage, e.g. caching · CPC title