Orchestration of workloads in K8s Clusters with SmartNICs

Owned by Srinivasa Addepalli
Last updated: Feb 12, 2020
2 min read

Background:

Community started to think about SmartNICs, especially in Edges

  • To increase the density of network functions and application workloads in the CPUs :  as Edges are space/power/cost constrained, thinking is that offloading infrastructure networking to smarter network processing entities would save cores and thereby increase the workload density.

  • To protect workloads from DDOS attacks.

In many deployments, just foundation NICs are used.  Foundation NICs are part of the servers. Thety are not exposed as nodes in K8s cluster. Only servers are shown as K8s nodes in the cluster.  

SmartNICs, in addition to having purpose built logic (ASIC/NPU/FPGA) for offloads, they are increasingly bundled with general purpose cores.  Community thinking is that these general purpose cores can be leveraged for control plane  services of infrastructure networking and also used for network monitoring & network analytics services.  Essentially, offloading not only data plane portion of infrastructure networking, but also control plane processing of infrastructure networking. For example, purpose built logic can be used to offload OVS Data Path (Or TC flower data path) and general purpose cores offloading corresponding control plane services (ovs-vswitchd, ovsdb,  ovn-southdb etc..) and monitoring services such as IPFIX etc..  

To provide similar CI/CD mechanisms, it is thought that K8s is used for deploying infrastructure control-plane/monitoring services.

There are few orchestration challenges that need to be addressed as a single physical server entity is now exposed as two different K8s nodes.  Few challenges that need to be addressed are:

  • How do we connect server node with corresponding smartnic node?

  • How does one ensure that only some specific services are instantiated in SmartNIC nodes?

  • How does one ensure to connect server software with SmartNIC for offloading flows?

  • ???

In this working group, intention is to discuss and come out with

  • Where is SmartNIC market going?

  • What are different types and their capabilities?

  • What additional software entities to be developed for K8s networking offloads

  • How does 5G slicing work in SmartNIC enviornments?