Distributed load balancing systems

Patent metadata

Field	Value
Publication number	US-10135737-B2
Application number	US-201414557290-A
Country	US
Kind code	B2
Filing date	Dec 1, 2014
Priority date	Sep 30, 2014
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

Some embodiments provide a novel method for load balancing data messages that are sent by a source compute node (SCN) to one or more different groups of destination compute nodes (DCNs). In some embodiments, the method deploys a load balancer in the source compute node's egress datapath. This load balancer receives each data message sent from the source compute node, and determines whether the data message is addressed to one of the DCN groups for which the load balancer spreads the data traffic to balance the load across (e.g., data traffic directed to) the DCNs in the group. When the received data message is not addressed to one of the load balanced DCN groups, the load balancer forwards the received data message to its addressed destination. On the other hand, when the received data message is addressed to one of load balancer's DCN groups, the load balancer identifies a DCN in the addressed DCN group that should receive the data message, and directs the data message to the identified DCN. To direct the data message to the identified DCN, the load balancer in some embodiments changes the destination address (e.g., the destination IP address, destination port, destination MAC address, etc.) in the data message from the address of the identified DCN group to the address (e.g., the destination IP address) of the identified DCN.

First claim

We claim:

1. A distributed load balancing (DLB) system comprising:
a plurality of host computers on which a plurality of source compute nodes (SCNs) execute;
a software forwarding element (SFE) comprising a plurality of SFE ports executing on each host computer;
a plurality of load balancers executing on each host computer, each load balancer associated with one SFE port connected to an SCN executing on the host computer, each load balancer (i) receiving data messages sent by the SCN connected to the SFE port from the associated SFE port along the egress path of the data messages from the SCN to the SFE on the load balancer's host computer, (ii) based on a set of load balancing criteria, distributing the data messages that are addressed to a virtual Internet Protocol (IP) address associated with a destination compute node (DCN) group to the DCNs in the group by changing a destination virtual IP address of the DCN group in the data messages to an IP address of one of the DCNs in the group, and (iii) supplying the data messages back to the SFE port to supply the data messages to the SFE on the load balancer's host computer to forward the data messages to the DCNs, wherein each SFE port in the plurality of SFE ports has its own load balancer; and
a set of controllers for configuring the load balancers on the host computers, wherein the set of controllers configures the load balancers by collecting message traffic statistics from the load balancers, aggregating the message traffic statistics, computing adjusted load balancing criteria from the aggregated message traffic statistics, and distributing the adjusted load balancing criteria to the load balancers for the load balancers to adjust how they distribute the data messages among the DCNs.

2. The DLB system of claim 1 , wherein the set of controllers includes at least one controller for deploying at least one SCN on the host computers.

3. The DLB system of claim 1 , wherein the message traffic statistics from a load balancer comprises data regarding a number of data messages that the load balancer forwarded to each DCN in the DCN group.

4. The DLB system of claim 1 , wherein the message traffic statistics from a load balancer comprises data regarding a number of data message flows that the load balancer forwarded to each DCN in the DCN group.

5. The DLB system of claim 1 , wherein the distributed adjusted load balancing criteria comprises adjusted weight values that affect how the load balancer distributes the data messages among the DCN.

6. The DLB of claim 5 , wherein the weight values include one weight value for each DCN, wherein based on the weight values, the load balancer performs a weighted round robin selection of the DCNs for new data message flows.

7. The DLB system of claim 1 , wherein the set of controllers further configures the load balancers by distributing addresses of the DCNs that are part of the DCN group.

8. The DLB system of claim 1 , wherein the set of controllers further configures the load balancers by (i) distributing a list of the DCNs that are part of the DCN group, and (ii) when a DCN is to be added or removed from the DCN group, distributing an updated list that adds or removes the added or removed DCN from the DCN group.

9. The DLB system of claim 1 , wherein
the controller set is further for deploying the SCNs on the host computer, and
the controller set configures each load balancer as part of the deployment of the load balancer's SCN.

10. The DLB system of claim 1 , wherein:
the DCN group is a first DCN group; and
at least one load balancer is further for intercepting, on the load balancer's host computer, data messages that are sent by at least one SCN to a second DCN group and distributing the data messages among the DCNs of the second group.

11. The DLB system of claim 1 , wherein
the DCNs are data compute end nodes or middle box service nodes; and
the SCNs are virtual machines.

12. The DLB system of claim 1 , wherein
each load balancer enforces the load balancing criteria needed to distribute the data traffic from the load balancer's associated SCN.

13. The DLB system of claim 1 , wherein each SFE port is associated with only one load balancer executing on the host computer.

14. A method for a host computing device that belongs to a group of a plurality of host computers, the method comprising:
executing a set of one or more source compute nodes (SCN) on the host computing device;
executing a plurality of load balancers that implement a distributed load balancer (DLB) with other load balancers on host computers of the group, each load balancer associated with one software forwarding element (SFE) port connected to an SCN executing on the host computer, each load balancer in the set (i) receiving data messages sent by the SCN from the associated SFE port along the egress path of data message from the SCN to an SFE on the load balancer's host computing device, (ii) based on a set of load balancing criteria, distributing the data messages that are addressed to a virtual internet protocol (IP) address associated with a destination compute node (DCN) group to the DCNs in the group by changing a destination virtual IP address of the DCN group in the data messages to an IP address of one of the DCNs in the group, and (iii) supplying the data messages back to the SFE port to supply the data messages to the SFE on the load balancer's host computer to forward the data messages to the DCNs, wherein each SFE port has its own load balancer; and
receiving configuration data from a set of controllers, the configuration data for configuring each load balancer on the host computing device, wherein the set of controllers collects message traffic statistics from load balancers on the plurality of host computers, aggregates the message traffic statistics, computes adjusted load balancing criteria from the aggregated message traffic statistics, and distributes configuration data comprising adjusted load balancing criteria to the load balancers for the load balancers to adjust how they distribute the data messages among the DCNs.

15. The method of claim 14 ,
wherein the message traffic statistics collected from load balancers on the plurality of host computers comprises one or more attributes of flows distributed by the load balancers to the DCNs in the DCN group, the flow attributes comprising at least one of a number of data messages distributed to each DCN, a number of data message flows distributed to each DCN, and a size of the data message flows distributed to each DCN.

16. The method of claim 14 , wherein the configuration data further comprises addresses of the DCNs that are part of the DCN group.

17. The method of claim 14 , wherein the SCNs send the data messages to the virtual IP address associated with the DCN group.

18. A non-transitory machine readable medium storing sets of instructions for execution by at least one processing unit of a host computer that belongs to a group of a plurality of host computers, the sets of instructions for:
executing a set of source compute nodes (SCN);
executing a plurality of load balancers that implement a distributed load balancer (DLB) with other load balancers executing on host computers of the group, each load balancer associated with one software forwarding element (SFE) port connected to an SCN executing on the host computer, each load balancer in the set (i) receiving data messages sent by the SCN from the associated SFE port along the egress path of the data messages from the SCN to an SFE on the load balancer's host computing d

Assignees

• Nicira Inc

Inventors

• Jain Jayant
• Sengupta Anirban
• Parthasarathy Mohan
• Sequeira Allwyn
• Maskalik Serge
• Lund Rick

Classifications

• G06F2009/4557

  Distribution of virtual machine instances; Migration and load balancing · CPC title

• G06F9/45558

  Hypervisor-specific management and integration aspects · CPC title

• H04L67/1025

  Dynamic adaptation of the criteria on which the server selection is based · CPC title

• H04L67/1029

  using data related to the state of servers by a load balancer · CPC title

• H04L47/125 Primary

  by balancing the load, e.g. traffic engineering · CPC title