FPGA acceleration for serverless computing

Patent metadata

Field	Value
Publication number	US-11740935-B2
Application number	US-202117519395-A
Country	US
Kind code	B2
Filing date	Nov 4, 2021
Priority date	Jul 20, 2017
Publication date	Aug 29, 2023
Grant date	Aug 29, 2023

Abstract

In one embodiment, a method for FPGA accelerated serverless computing comprises receiving, from a user, a definition of a serverless computing task comprising one or more functions to be executed. A task scheduler performs an initial placement of the serverless computing task to a first host determined to be a first optimal host for executing the serverless computing task. The task scheduler determines a supplemental placement of a first function to a second host determined to be a second optimal host for accelerating execution of the first function, wherein the first function is not able to accelerated by one or more FPGAs in the first host. The serverless computing task is executed on the first host and the second host according to the initial placement and the supplemental placement.

First claim

What is claimed is:

1. A method for hardware-accelerated serverless computing, the method comprising:
receiving, from a user, a serverless computing task associated with one or more functions including at least one accelerable function able to be accelerated by hardware accelerators provided in at least one host of a plurality of hosts of a serverless computing environment;
performing a placement of the serverless computing task to a first host of the plurality of hosts, the first host determined to be a host where a first function of the serverless computing task can be executed;
executing on the first host at least the first function of the serverless computing task;
identifying a second host of the plurality of hosts, the identified second host having a shared hardware accelerator and determined to be capable of accelerating execution of the accelerable function, the accelerable function not able to be accelerated by the first host; and
executing the serverless computing task on the first host and the identified second host, the executing of the serverless computing task on the first host and the identified second host including provisioning a first data forwarder for forwarding data to be processed by the accelerable function from the first host to the identified second host.

2. The method of claim 1 , wherein the shared hardware accelerators comprise graphics processing units.

3. The method of claim 1 , wherein the shared hardware accelerators comprise field programmable gate arrays.

4. The method of claim 1 , wherein the accelerable function comprises a machine learning function.

5. The method of claim 1 , wherein the accelerable function comprises an artificial intelligence function.

6. The method of claim 1 , further comprising:
determining an initial placement of the serverless computing task by estimating execution time of the serverless computing task for each of the plurality of hosts and selecting the identified second host for having less execution time for the accelerable function than other hosts of the plurality of hosts.

7. A system comprising:
a first host including at least one processor;
a second host comprising a hardware accelerator;
an interface connected to the first host and the second host and configured to receive, from a user, a serverless computing task with one or more functions to be executed, the one or more functions including at least one function able to be accelerated by one or more hardware accelerators provided in the first host or the second host;
a task scheduler configured to perform a placement of the serverless computing task to the first host, the first host determined to be a host where the serverless computing task can be executed, and to determine a placement of a hardware accelerable function of the serverless computing task to the second host, the second host determined to be capable of accelerating execution of the hardware accelerable function, the hardware accelerable function not able to be accelerated by the first host; and
one or more networked hardware resources configured to execute the serverless computing task on the first host and the second host, executing the serverless computing task on the first host and the second host including provisioning a first data forwarder for forwarding data to be processed by the accelerable function from the first host to the second host.

8. The system of claim 7 , wherein the hardware accelerators comprise graphics processing units (GPUs).

9. The system of claim 7 , wherein the hardware accelerators comprise field-programmable gate arrays (FPGAs).

10. The system of claim 7 , wherein the accelerable function comprises a machine learning algorithm.

11. The system of claim 7 , wherein the accelerable function comprises an artificial intelligence function.

12. The system of claim 7 , wherein the placement is triggered based on an actual input data size to the hardware accelerable function.

13. One or more computer-readable non-transitory media comprising instructions, that when executed on one or more processors configure the one or more processors to perform operations comprising:
receiving, from a user, a serverless computing task having at least two functions able to be accelerated by hardware accelerators provided in at least one of a plurality of hosts of a serverless computing environment;
performing a placement of the serverless computing task to a first host of the plurality of hosts, the first host determined to be capable of executing a first accelerable function of the serverless computing task;
performing a supplemental placement of a second accelerable function of the serverless computing task to a second host of the plurality of hosts, the second host determined to be capable of accelerating execution of the second accelerable function, the second accelerable function not able to be accelerated by one or more of the hardware accelerators in the first host; and
executing the serverless computing task on the first host and the second host according to the placement and the supplemental placement, the executing of the serverless computing task on the first host and the second host including provisioning a first data forwarder for forwarding data to be processed by the second accelerable function from the first host to the second host.

14. The one or more computer-readable non-transitory media of claim 13 , wherein the hardware accelerators comprise graphics processing units (GPUs).

15. The one or more computer-readable non-transitory media of claim 13 , wherein the hardware accelerators comprise field-programmable gate arrays (FPGAs).

16. The one or more computer-readable non-transitory media of claim 13 , wherein the accelerable functions comprise a machine learning algorithm.

17. The one or more computer-readable non-transitory media of claim 13 , wherein the accelerable functions comprise an artificial intelligence function.

18. The one or more computer-readable non-transitory media of claim 13 , wherein the operations include triggering the supplemental placement based on the input data size to the second accelerable function.

19. The method of claim 1 , wherein the first host comprises at least one of a virtual machine or a software container, and wherein the second host comprises at least one of a different virtual machine or a different software container.

20. The method of claim 1 , wherein the first host comprises a first computer system comprising at least one memory and at least one processor coupled to the at least one memory, and wherein the second host comprises a second computer system comprising at least one additional memory and at least one additional processor coupled to the at least one additional memory.

21. The method of claim 1 , wherein the first host lacks a hardware accelerator suitable for accelerating the accelerable function of the serverless computing task.

Assignees

• Cisco Tech Inc

Inventors

• Shimamura Komei
• Huang Xinyuan
• Saha Amit Kumar
• Dutta Debojyoti

Classifications

• G06F9/4881 Primary

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/5038

  considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration (scheduling strategies G06F9/4881 and subgroups) · CPC title

• G06F9/5066 Primary

  Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451) · CPC title

• G06F9/5088

  involving task migration · CPC title