Networking Plugin
...
One of the requirements for VNF's is to support multiple virtual network interfaces and multiple IP addresses. Multus acts as a Multi plugin in Kubernetes and provides the multiple network interface support in a pod. https://github.com/intel/multus-cni. It'll be used in this project to provide a default management port based on Flannel to all VNF's. The other interfaces will all be based on ovn-kubernetes as discussed in the next section.
CNI configuration
Multus is configured to have default network as Flannel:
{
"type": "multus",
"kubeconfig": "/etc/kubernetes/admin.conf",
"delegates": [
{
"type": "flannel",
"masterplugin": true,
"delegate": {
"isDefaultGateway": true
}
}
]
}
Create following network resource for OVN.
OVN network resources in Kubernetes:
apiVersion: "kubernetes.cni.cncf.io/v1"
kind: Network
metadata:
name: ovn-network
spec:
config: '{
"name": "ovn-kubernetes",
"type": "ovn-k8s-cni-overlay"
}'
With this setup a Pod can be annotated as below for OVN:
apiVersion: v1
kind: Pod
metadata:
name: pod-ovn
annotations:
kubernetes.v1.cni.cncf.io/networks: '[
{ "name": "ovn-network", "interfaceRequest": "eth1"}
]'
spec: # specification of the pod's contents
containers:
- name: pod-ovn
image: "busybox"
command: ["top"]
stdin: true
tty: true
With this setup and with above pod definition a Pod with two interfaces is created. This has a limitation though. The OVN interfaces always belongs to the subnet configured during initialization.
OPEN: To get above pod configuration to work ovn-kubernetes had to be modified and AddRoute for the Pod had to be removed.
Ovn-kubernetes design and changes for multiple interface support
- ovn-kubernetes current design doesn't support multiple interfaces and also a single default network is created at the time initialization time and all pods are connected to that.
- In the current design of ovn-kubernetes Currently if a Pod is annotated with "ovn" then static IP, MAC can be assigned for the single interface that is created. And if the pod is not annotated with "ovn" then the ovn watcher annotates the pod with the "ovn" with dynamic address allocation. This information is then used by OVN CNI plugin to assign address for the interface. For backward compatibility this behavior will not change and pods can be created with single interface as before. The one interface is attached to a default network created at the time initialization time.
- To create multiple interface the pods or to attach single interface to a different virtual network a new annotation is added to the Pods - "ovnNetwork". If "ovnNetwork" annotation is present then the "ovn " annotation is ignored.
- Assumption CreateVL call is already made prior pod getting created with that the virtual network needed for new Pod initialization.
- To create a pod with multiple interfaces annotate the pod like below:
...
{ "name": "ovn-ls-26", "interface": "net1", "ip_address":"172.16.2526.3", "mac_address":"0a:00:00:00:00:15" }
In this example net0 interface will be connected to a "ovn-ls-25" network and dynamic address allocation will be done for the addresses. net1 is connected to the ovn-ls-26 network and static address is allocated for itused as specified.
- "name": Virtual network name, used to connect interface to network
- "interface": Name of the interface. Assumption: Current 10 interfaces are supported. All interfaces should be named with alphanumeric characters with the last character being a number between 0-9. This is required as this number is used to distinguish between interfaces within the pod namespace.
- Ip_address: static ip. Optional if not given dynamically assigned
- mac_address: static mac. Optional if not given dynamically assigned
- ovn-kubernetes watcher based on the "ovnNetwork" annotation connects the interfaces to the right virtual network and creates a list of interfaces and corresponding addresses for the CNI to assign addresses to the newly created Pods. It annotates the pod with the list "ovnIfaceList".
For example: ovnIfaceList=[{"ip_address":"172.16.25.3", "mac_address":"0a:00:00:00:00:1524", "gateway_ip": "172.16.25.1"}, {"ip_address":"172.16.26.3", "mac_address":"0a:00:00:00:00:15", "gateway_ip": "172.16.26.1"}]