FPGA acceleration for serverless computing

Patent metadata

Field	Value
Publication number	US-11709704-B2
Application number	US-202117408259-A
Country	US
Kind code	B2
Filing date	Aug 20, 2021
Priority date	Jul 20, 2017
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

In one embodiment, a method for FPGA accelerated serverless computing comprises receiving, from a user, a definition of a serverless computing task comprising one or more functions to be executed. A task scheduler performs an initial placement of the serverless computing task to a first host determined to be a first optimal host for executing the serverless computing task. The task scheduler determines a supplemental placement of a first function to a second host determined to be a second optimal host for accelerating execution of the first function, wherein the first function is not able to accelerated by one or more FPGAs in the first host. The serverless computing task is executed on the first host and the second host according to the initial placement and the supplemental placement.

First claim

What is claimed is:

1. A method for hardware-accelerated serverless computing, the method comprising:
receiving a definition of a serverless computing task comprising a first portion and a second portion, at least the second portion able to be accelerated by a hardware accelerator;
placing the serverless computing task entirely with a first host of a plurality of hosts;
executing on the first host at least the first portion of the serverless computing task;
identifying, during the executing of the first portion, a second host of the plurality of hosts having a hardware accelerator to execute the second portion;
placing the second portion with the identified second host; and
executing the second portion on the hardware accelerator on the identified second host.

2. The method of claim 1 , wherein the hardware accelerator comprises a graphics processing unit.

3. The method of claim 1 , wherein the hardware accelerator comprises a field programmable gate array.

4. The method of claim 1 , wherein the second portion of the serverless computing task comprises a machine learning script.

5. The method of claim 1 , wherein the first host lacks a hardware accelerator suitable for accelerating the second portion of the serverless computing task.

6. The method of claim 1 , further comprising:
identifying, prior to the placing with the first host, the first host by estimating execution time of the serverless computing task for each of the plurality of hosts and selecting as the first host one of the plurality of hosts having less execution time than other ones of the plurality of hosts.

7. The method of claim 1 , wherein the placing with the identified second host is in response to input of an actual input data size to a function during run-time exceeding a predicted input data size to the function.

8. The method of claim 1 , wherein the identifying of the second host comprises estimating network communication overhead of data being redirected from the one of the plurality of hosts to others of the plurality of hosts, estimating execution time of a function on each of the others of the plurality of hosts, and selecting a second host having less network communication overhead and execution time than the others of the plurality of hosts.

9. A non-transitory computer readable media storing instructions programmed to cooperate with a system including electronic computer hardware to cause the system to perform operations comprising:
receiving a definition of a serverless computing task comprising a first portion and a second portion, at least the second portion able to be accelerated by a hardware accelerator;
placing the serverless computing task entirely with a first host of a plurality of hosts;
executing on the first host at least the first portion of the serverless computing task;
identifying, during the executing of the first portion, a second host of the plurality of hosts having a hardware accelerator to execute the second portion;
placing the second portion with the identified second host; and
executing the second portion on the hardware accelerator on the identified second host.

10. The non-transitory computer readable media of claim 9 , wherein the hardware accelerator comprises a graphics processing unit.

11. The non-transitory computer readable media of claim 9 , wherein the hardware accelerator comprises a field programmable gate array.

12. The non-transitory computer readable media of claim 9 , wherein the second portion of the serverless computing task comprises a machine learning script.

13. The non-transitory computer readable media of claim 9 , wherein the first host lacks a hardware accelerator suitable for accelerating the second portion of the serverless computing task.

14. The non-transitory computer readable media of claim 9 , the operations further comprising:
identifying, prior to the placing with the first host, the first host by estimating execution time of the serverless computing task for each of the plurality of hosts and selecting as the first host one of the plurality of hosts having less execution time than other ones of the plurality of hosts.

15. The non-transitory computer readable media of claim 9 , wherein the placing with the identified second host is in response to input of an actual input data size to a function during run-time exceeding a predicted input data size to the function.

16. The non-transitory computer readable media of claim 9 , wherein the identifying of the second host comprises estimating network communication overhead of data being redirected from the one of the plurality of hosts to others of the plurality of hosts, estimating execution time of a function on each of the others of the plurality of hosts, and selecting a second host having less network communication overhead and execution time than the others of the plurality of hosts.

17. A system, comprising:
a processor including electronic computer hardware; and
a non-transitory computer readable memory storing instructions programmed to cooperate with the processor to cause the system to perform operations comprising:
receiving a definition of a serverless computing task comprising a first portion and a second portion, at least the second portion able to be accelerated by a hardware accelerator;
placing the serverless computing task entirely with a first host of a plurality of hosts;
executing on the first host at least the first portion of the serverless computing task;
identifying, during the executing of the first portion, a second host of the plurality of hosts having a hardware accelerator to execute the second portion;
placing the second portion with the identified second host; and

executing the second portion on the hardware accelerator on the identified second host.

18. The system of claim 17 , wherein the hardware accelerator comprises a graphics processing unit.

19. The system of claim 17 , wherein the hardware accelerator comprises a field programmable gate array.

20. The system of claim 17 , wherein the second portion of the serverless computing task comprises a machine learning script.

21. The system of claim 17 , wherein the first host lacks a hardware accelerator suitable for accelerating the second portion of the serverless computing task.

22. The system of claim 17 , the operations further comprising:
identifying, prior to the placing with the first host, the first host by estimating execution time of the serverless computing task for each of the plurality of hosts and selecting as the first host one of the plurality of hosts having less execution time than other ones of the plurality of hosts.

23. The system of claim 17 , wherein the placing with the identified second host is in response to input of an actual input data size to a function during run-time exceeding a predicted input data size to the function.

24. The system of claim 17 , wherein the identifying of the second host comprises estimating network communication overhead of data being redirected from the one of the plurality of hosts to others of the plurality of hosts, estimating execution time of a function on each of the others of the plurality of hosts, and selecting a second host having less network communication overhead and execution time than the others of the plurality of hosts.

Assignees

• Cisco Tech Inc

Inventors

• Shimamura Komei
• Huang Xinyuan
• Saha Amit Kumar
• Dutta Debojyoti

Classifications

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/5038

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F9/5066 Primary

  Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451) · CPC title

• G06F9/5088

  involving task migration · CPC title