Using CRDs to create externally routable addresses and route records for pods

Patent metadata

Field	Value
Publication number	US-12177124-B2
Application number	US-202217960126-A
Country	US
Kind code	B2
Filing date	Oct 4, 2022
Priority date	Oct 4, 2022
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

Some embodiments provide an automated method for defining externally routable Pods within a Kubernetes cluster. In some embodiments, the Pod operates in a guest cluster has its own VPC (virtual private cloud) network in a datacenter with several other guest clusters that have their own VPC networks and their own set of managers. In some embodiments, a Pod within a GC can be made externally routable so that it can be directly addressable from an external client outside of the Pod's network by using two new Kubernetes CRDs (custom resource definitions), which are an IPPool CRD and a RouteSet CRD. Examples of such external clients include VMs or Pods in another GC or a supervisor cluster connected to the particular GC through a gateway, or from a machine outside of the network of all of the GCs or SC.

First claim

The invention claimed is:

1. A method of deploying an externally routable Pod on a worker node in a first cluster deployed in a first virtual private cloud, the method comprising:
at a control plane of a supervisor second cluster (SC):
receiving, from a server, notification that an IP Pool custom resource instance has been created in response to an addition of the worker node to the first cluster;
creating a set of one or more externally routable IP subnets for the IP pool; and
providing, to the server, the set of externally routable IP subnets for the server to use to update IP Pool customer resource instance.

2. The method of claim 1 further comprising:
providing a status update to a manager of the first cluster for the manager to direct a Kubernetes manager to update a routing record for the Pod on the worker node.

3. The method of claim 2 , wherein the routing record is a CIDR (Classless Inter-Domain Routing) record.

4. The method of claim 2 further comprising before the notification is received:
from the Kubernetes manager, receiving, at the first cluster manager, a notification that the worker node has been added; and
from the first cluster manager, receiving, at the server, a request to create an IP Pool custom resource based on an IP Pool CRD (custom resource definition).

5. The method of claim 4 further comprising:
from the first cluster manager, receiving, at the server, a request to create a RouteSet custom resource based on a RouteSet CRD;
from the first cluster manager, receiving, at the SC control plane, a notification that a RouteSet CR has been added; and
creating, at the SC control plane, one or more static routes to deploy at one or more intervening gateway routers between the Pod in the first cluster of the VPC and a network external to a first network for the first VPC.

6. The method of claim 5 , wherein the external network is the network of the supervisor cluster.

7. The method of claim 5 , wherein the external network is the network of a second VPC that connects to the first VPC and the SC through one or more gateway routers.

8. The method of claim 7 , wherein each VPC has a set of managers to manage compute and network resources in each VPC, and the SC has a set of managers to manage compute and network resources in the SC as well as configure the gateway routers to allow traffic to reach said Pod from outside of the first network of the first VPC.

9. The method of claim 5 further comprising:
specifying a middlebox service rule to perform a middlebox service operation on packets sent to the externally routable Pod from outside of the Pod's network; and
distributing the middlebox service rule to a service node associated with an intervening gateway, said service node enforcing the service rule on said packets.

10. The method of claim 5 further comprising:
specifying a middlebox service rule to perform a middlebox service operation on packets sent by the externally routable Pod to a machine outside of the Pod's network; and
distributing the middlebox service rule to a service node associated with an intervening gateway, said service node enforcing the service rule on said packets.

11. A non-transitory machine readable medium storing a program for execution by a set of processing units, the program for deploying an externally routable Pod on a worker node in a first cluster deployed in a first virtual private cloud, the program comprising sets of instructions for:
at a control plane of a supervisor second cluster (SC):
receiving, from a server, notification that an IP Pool custom resource instance has been created in response to an addition of the worker node to the first cluster;
creating a set of one or more externally routable IP subnets for the IP pool; and
providing, to the server, the set of externally routable IP subnets for the server to use to update IP Pool customer resource instance.

12. The non-transitory machine readable medium of claim 11 , the program further comprising a set of instructions for:
providing a status update to a manager of the first cluster for the manager to direct a Kubernetes manager to update a routing record for the Pod on the worker node.

13. The non-transitory machine readable medium of claim 12 , wherein the routing record is a CIDR (Classless Inter-Domain Routing) record.

14. The non-transitory machine readable medium of claim 12 , the program further comprising, before the notification is received, sets of instructions for:
from the Kubernetes manager, receiving, at the first cluster manager, a notification that the worker node has been added; and
from the first cluster manager, receiving, at the server, a request to create an IP Pool custom resource based on an IP Pool CRD (custom resource definition).

15. The non-transitory machine readable medium of claim 14 , the program further comprising sets of instructions for:
from the first cluster manager, receiving, at the server, a request to create a RouteSet custom resource based on a RouteSet CRD;
from the first cluster manager, receiving, at the SC control plane, a notification that a RouteSet CR has been added; and
creating, at the SC control plane, one or more static routes to deploy at one or more intervening gateway routers between the Pod in the first cluster of the VPC and a network external to a first network for the first VPC.

16. The non-transitory machine readable medium of claim 15 , wherein the external network is the network of the supervisor cluster.

17. The non-transitory machine readable medium of claim 15 , wherein the external network is the network of a second VPC that connects to the first VPC and the SC through one or more gateway routers.

18. The non-transitory machine readable medium of claim 7 , wherein each VPC has a set of managers to manage compute and network resources in each VPC, and the SC has a set of managers to manage compute and network resources in the SC as well as configure the gateway routers to allow traffic to reach said Pod from outside of the first network of the first VPC.

19. The non-transitory machine readable medium of claim 15 , the program further comprising sets of instructions for:
specifying a middlebox service rule to perform a middlebox service operation on packets sent to the externally routable Pod from outside of the Pod's network; and
distributing the middlebox service rule to a service node associated with an intervening gateway, said service node enforcing the service rule on said packets.

20. The non-transitory machine readable medium of claim 15 , the program further comprising sets of instructions for:
specifying a middlebox service rule to perform a middlebox service operation on packets sent by the externally routable Pod to a machine outside of the Pod's network; and
distributing the middlebox service rule to a service node associated with an intervening gateway, said service node enforcing the service rule on said packets.

Assignees

• VMware LLC

Inventors

• Liu Danting
• Sun Qian
• Shen Jianjun
• Liu Wenfeng
• Han Donghai

Classifications

• H04L45/02

  Topology update or discovery · CPC title

• H04L45/586 Primary

  of virtual routers · CPC title

• H04L45/745 Primary

  Address table lookup; Address filtering · CPC title