Work in Progress - 9/9/2020
Kubernetes clusters using ONAP Multicloud K8s Plugin Project for R4/R5 do not report hardware features to A&AI. Consequently during the CNF/VNF life cycle CNFs/VNFs that require or recommend specific hardware during instantiation cannot dynamically reach the correct cluster and node that provides the needed hardware capabilities.
This epic ( MULTICLOUD-729 - Getting issue details... STATUS ) adds the ability to discover, report and use hardware features during CNF/VNF life cycle to accelerate use cases using ONAP .
- Discovery of HPA features will be done using the node feature discovery [NFD] addon (https://github.com/kubernetes-sigs/node-feature-discovery) for kubernetes. NFD will be deployed as a daemonSet, with pods on each node and will discover and label hardware features on each kubernetes node with features labels (https://github.com/kubernetes-sigs/node-feature-discovery#feature-labels).
- Reporting of HPA features to AA&I will be done in Multicloud K8s Plugin registrationHandler. Node labels will be discovered on each node in a cluster. This date will be munged to cloud-region/tenant/flavors schema for on that cluster and reported to AA&I.
- CNF and VNF helm charts will be updated with nodeSelector requirements & recommendations in line with node labels created by NFD during feature discovery.
Example Node Features:
# Hardware Feature Labels: feature.node.kubernetes.io/cpu-cpuid.AESNI=true, feature.node.kubernetes.io/cpu-cpuid.AVX2=true, feature.node.kubernetes.io/cpu-cpuid.AVX=true, feature.node.kubernetes.io/cpu-cpuid.FMA3=true, feature.node.kubernetes.io/cpu-cpuid.IBPB=true, feature.node.kubernetes.io/cpu-cpuid.STIBP=true, feature.node.kubernetes.io/cpu-hardware_multithreading=true, feature.node.kubernetes.io/cpu-pstate.turbo=true, feature.node.kubernetes.io/cpu-rdt.RDTCMT=true, feature.node.kubernetes.io/cpu-rdt.RDTMON=true, feature.node.kubernetes.io/memory-numa=true, feature.node.kubernetes.io/network-sriov.capable=true, feature.node.kubernetes.io/pci-0300_102b.present=true, feature.node.kubernetes.io/storage-nonrotationaldisk=true # Software Feature Labels: feature.node.kubernetes.io/kernel-config.NO_HZ=true, feature.node.kubernetes.io/kernel-config.NO_HZ_FULL=true, feature.node.kubernetes.io/kernel-version.full=3.10.0-957.el7.x86_64, feature.node.kubernetes.io/kernel-version.major=3, feature.node.kubernetes.io/kernel-version.minor=10, feature.node.kubernetes.io/kernel-version.revision=0, feature.node.kubernetes.io/system-os_release.ID=centos, feature.node.kubernetes.io/system-os_release.VERSION_ID.major=7, feature.node.kubernetes.io/system-os_release.VERSION_ID.minor=, feature.node.kubernetes.io/system-os_release.VERSION_ID=7
Policy and OOF can then discover capabilities of K8s nodes and use that information to home particular CNF workloads to nodes which can accelerate their function. Further Multicloud will use labels when instantiating CNF workload to ensure kubernetes places the workloads on the correct node.
High Level Task Overview
- Ensure NFD (Node feature discovery - K8S CNCF project) deploy to Multicloud + K8s Plugin is working and labeling Kubernetes nodes
- Add registrationHandler to K8s Plugin and expand K8s Plugin API to work with ESR VIM registration of K8s Clusters
- Use POST/DELETE/GET of http://{{MSB_IP}}:{{MSB_PORT}}/api/multicloud-k8s/v1/{{cloud-owner}}/{{cloud-region-id}}/registry
- Ensure registrationHandler can get K8s feature labels - (labels discovered and placed by NFD representing hardware features)
- Ensure registrationHandler can populate A&AI cloud-region/tenant/compute flavors based on cluster under registration
- Register K8s cluster internally with the K8s Plugin
- Add Node App in K8s Plugin to retrieve node labels from a cluster
- Identify and Address any gaps in policy and OOF to do match making of CNFs/VNFs using K8s Plugin