Distributed jobs handling

Patent metadata

Field	Value
Publication number	US-2016154681-A1
Application number	US-201514951630-A
Country	US
Kind code	A1
Filing date	Nov 25, 2015
Priority date	Nov 28, 2014
Publication date	Jun 2, 2016
Grant date	—

Abstract

Embodiments include methods, systems and computer program products for handling a distributed job by a FPGA. Aspects include obtaining a demand for performance in the distributed job and determining, according to the demand for performance, whether to reconfigure the FPGA. Aspects also include dynamically reconfiguring at least a part of the FPGA in response to determination of reconfiguring the FPGA. With the method and corresponding system, the performance of the distributed job can be effectively improved.

First claim

What is claimed is:

1 . A method for handling a distributed job, wherein the distributed job is handled by a FPGA, the method comprising:
obtaining a demand for performance in the distributed job; determining, according to the demand for performance, whether to reconfigure the FPGA; and dynamically reconfiguring at least a part of the FPGA in response to determination of reconfiguring the FPGA.

2 . The method according to claim 1 , wherein the distributed job is a MapReduce job.

3 . The method according to claim 1 , wherein the obtaining the demand for performance in the distributed job comprises detecting load imbalance in the distributed job.

4 . The method according to claim 3 , wherein the detecting the demand for load imbalance in the distributed job further comprises:
monitoring progress of each task of the distributed job; and determining whether load imbalance exists according to the progress of each task.

5 . The method according to claim 1 , wherein the determining, according to the demand for performance, whether to reconfigure the FPGA comprises:
estimating an overhead and a predicted benefit for reconfiguring at least a part of the FPGA according to the demand for performance; and determining to reconfigure at least the part of the FPGA in response to the predicted benefit being greater than the overhead.

6 . The method according to claim 5 , wherein the dynamically reconfiguring at least a part of the FPGA in response to determination of reconfiguring the FPGA comprises:
reallocating a resource for a task of the distributed job according to the estimated overhead and the predicted benefit; and dynamically reconfiguring at least the part of the FPGA according to the reallocation of the resource.

7 . The method according to claim 1 , wherein the estimating an overhead and a predicted benefit for reconfiguring at least a part of the FPGA according to the demand for performance further comprises estimating the predicted benefit by using a performance model based on expert knowledge.

8 . The method according to claim 7 , wherein the expert knowledge includes one or more of: a type of a task executed at current stage of the distributed job, acceleration performance of the FPGA for the type of the task, and a demand for performance at the current stage of the distributed job.

9 . The method according to claim 1 , wherein the FPGA includes a plurality of FPGA cards distributed on a plurality of devices.

10 . A system for handling a distributed job, wherein the distributed job is handled by a FPGA, the system comprising:
a performance demand obtaining module configured to obtain a demand for performance in the distributed job; a reconfiguration determining module configured to determine, according to the demand for performance, whether to reconfigure the FPGA; and a reconfiguration executing module configured to dynamically reconfigure at least a part of the FPGA in response to determination of reconfiguring the FPGA.

11 . The system according to claim 10 , wherein the distributed job is a MapReduce job.

12 . The system according to claim 10 , wherein the performance demand obtaining module is configured to detect load imbalance in the distributed job.

13 . The system according to claim 12 , wherein the performance demand obtaining module is further configured to:
monitor progress of each task of the distributed job; and determine whether load imbalance exists according to the progress of each task.

14 . The system according to claim 10 , wherein the reconfiguration determining module is further configured to:
estimate an overhead and a predicted benefit for reconfiguring at least a part of the FPGA according to the demand for performance; and determine to reconfigure at least the part of the FPGA in response to the predicted benefit being greater than the overhead.

15 . The system according to claim 14 , wherein the reconfiguration executing module is further configured to:
reallocate a resource for a task of the distributed job according to the estimated overhead and the predicted benefit; and dynamically reconfigure at least the part of the FPGA according to the reallocation of the resource.

16 . The system according to claim 10 , wherein the reconfiguration determining module is further configured to estimate the predicted benefit by using a performance model based on expert knowledge.

17 . The system according to claim 16 , wherein the expert knowledge includes one or more of: a type of a task executed at current stage of the distributed job, acceleration performance of the FPGA for the type of the task, and a demand for performance at the current stage of the distributed job.

18 . The system according to claim 10 , wherein the FPGA includes a plurality of FPGA cards distributed on a plurality of devices.

19 . A computer program product for handling a distributed job, the computer program product comprising:
a non-transitory storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising: obtaining a demand for performance in the distributed job; determining, according to the demand for performance, whether to reconfigure the FPGA; and dynamically reconfiguring at least a part of the FPGA in response to determination of reconfiguring the FPGA.

20 . The computer program product according to claim 19 , wherein the determining, according to the demand for performance, whether to reconfigure the FPGA comprises:
estimating an overhead and a predicted benefit for reconfiguring at least a part of the FPGA according to the demand for performance; and determining to reconfigure at least the part of the FPGA in response to the predicted benefit being greater than the overhead.

Assignees

• IBM

Inventors

• Chen Fei
• Chen Guan Cheng
• Hofstee H Peter
• Tao Liu
• Wang Kun
• Zhang Yu

Classifications

• G06F9/5083 Primary

  Techniques for rebalancing the load in a distributed system · CPC title

• G06F15/7867

  with reconfigurable architecture · CPC title