Connection redistribution in load-balanced systems

Patent metadata

Field	Value
Publication number	US-2016373373-A1
Application number	US-201615250440-A
Country	US
Kind code	A1
Filing date	Aug 29, 2016
Priority date	Jun 26, 2013
Publication date	Dec 22, 2016
Grant date	—

Abstract

Methods and apparatus for connection redistribution in load-balanced systems that include multiple load balancers each serving multiple nodes. In the connection redistribution method, each node estimates a connection close rate, which may be based on an estimation of the percentage of the overall client traffic received by the respective load balancer that is being handled by the node. The node generates close requests for connections between the respective load balancer and clients according to the connection close rate. The node sends the close requests to its load balancer, which forwards the close requests to the appropriate clients. Upon receiving a close request, a client may close the connection(s) indicated by the request, obtain a public IP address for a load balancer, and initiate new connection(s) to the respective load balancer via the public IP address.

First claim

1 .- 20 . (canceled)

21 . A network-based system, comprising:
a plurality of computing devices implementing a plurality of nodes each configured to provide one or more functionalities of the network-based system, wherein the plurality of nodes are grouped into a plurality of node groups each comprising two or more of the nodes; and a plurality of load balancers each fronting at least one of the plurality of node groups, each load balancer configured to:
receive client requests from a plurality of client devices via connections over a network between the client devices and the load balancer; and
distribute the received client requests among the nodes in a node group corresponding to the load balancer according to a load balancing technique;
wherein a node of the plurality of nodes is configured to:
generate connection close requests for connections between the client devices and a corresponding one of the plurality of load balancers, each connection close request indicating one or more connections; and
send the connection close requests to the load balancer;
wherein the load balancer is configured to forward the connection close requests received from the node in its corresponding node group to appropriate ones of the plurality of client devices via respective connections.

22 . The network-based system as recited in claim 21 , further comprising:
an endpoint discovery service configured to:
receive the client requests from the plurality of client devices over the network; and
distribute the received client requests among the plurality of load balancers;
and wherein the load balancer is configured to direct the endpoint discovery service to stop distributing client requests to the load balancer for a specified period of time subsequent to the forwarding of the connection close requests.

23 . The network-based system as recited in claim 21 , wherein the connection close request is configured to cause the client device that receives the connection close request to close one or more indicated connections to the load balancer and to cause the client device to open one or more new connections to at least one of the plurality of load balancers.

24 . The network-based system as recited in claim 22 , wherein the endpoint discovery service is a Domain Name Server (DNS) service.

25 . The network-based system as recited in claim 21 , wherein, to generate connection close requests for connections, the node is configured to:
receive information about the client devices; select a client request received from the load balancer for which to generate a connection close request based at least in part on the information about the client devices; and generate a response message to the client request with a header that includes the connection close request.

26 . The network-based system as recited in claim 25 , wherein the client requests and the response messages are Hypertext Transport Protocol (HTTP) messages.

27 . The network-based system as recited in claim 21 , wherein the connections between the client devices and the plurality of load balancers are Transmission Control Protocol (TCP) connections.

28 . The network-based system as recited in claim 21 , wherein the network-based system is configured to implement a distributed database service, wherein each of the plurality of nodes is a request router of the distributed database service, wherein each request router is configured to route client requests received from a corresponding load balancer to appropriate ones of a plurality of storage nodes of the distributed database service.

29 . A method, comprising:
receiving, at a node in a node group from a load balancer, a plurality of client requests, wherein the load balancer distributes client requests received via connections between the load balancer and clients among a plurality of nodes in the node group; generating, at the node, a connection close request for a connection between a client and the load balancer, the connection close request indicating the connection; and sending the connection close request to the client via the load balancer.

30 . The method as recited in claim 29 , wherein generating the connection close request for a connection between the load balancer and the client comprises modifying a header of a response message to the client request to include a connection close request.

31 . The method as recited in claim 29 , further comprising:
collecting information about the client, and selecting the connection for which to generate the connection close request based at least in part on the information about the client.

32 . The method as recited in claim 31 , further comprising:
sending the information about the client to a second node in the node group via a gossip protocol.

33 . The method as recited in claim 32 , further comprising:
sending presence information and health information about the node to the second node via the gossip protocol.

34 . The method as recited in claim 33 , wherein the second node belong to a second node group that is distinct from the node group.

35 . A non-transitory computer-accessible storage medium storing program instructions computer-executable to implement:
operating a node group comprising a plurality of nodes, the node group configured to receive client requests from clients via a load balancer; receiving, by a node in the node group, a plurality of client requests via a plurality of connections between a plurality of clients and the load balancer; generating, by the node, one or more connection close requests for one or more of the plurality of connections; and sending the connection close requests to the respective client via the load balancer.

36 . The non-transitory computer-accessible storage medium as recited in claim 35 , wherein, in generating a connection close request of the one or more connection close requests, the program instructions are computer-executable to implement modifying a header of a response message to a client request to include the connection close request.

37 . The non-transitory computer-accessible storage medium as recited in claim 35 , wherein, in generating a connection close request of the one or more connection close requests, the program instructions are computer-executable to implement selecting a connection for which to generate the connection close request; and
wherein the selecting is based at least in part on information collected about a client associated with the connect.

38 . The non-transitory computer-accessible storage medium as recited in claim 37 , wherein the selecting is based at least in part on the busyness of the client relative to other clients.

39 . The non-transitory computer-accessible storage medium as recited in claim 37 , wherein the information collected about the client is shared with the plurality of nodes of the node group.

40 . The non-transitory computer-accessible storage medium as recited in claim 35 , wherein the connections between the clients and the plurality of load balancers are Transmission Control Protocol (TCP) connections that carry Hypertext Transport Protocol (HTTP) messages between the clients and the plurality of load balancers.

Assignees

• Amazon Tech Inc

Inventors

• Das Pejus Manoj
• Yanacek David Craig

Classifications

• H04L67/02

  based on web technology, e.g. hypertext transfer protocol [HTTP] · CPC title

• H04L67/143

  Termination or inactivation of sessions, e.g. event-controlled end of session · CPC title

• H04L69/16

  Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] · CPC title

• H04L67/1002

  Electricity · mapped topic

• H04L61/2007

  Electricity · mapped topic