Introduction:

In today, many complex applications are consisting in a mixed, complex workload, that is described in many Kubernetes resources, e.g. to be run on a certain cluster, etc.  In order to deploy the application, the orchestration task would be requiring dealing with different abstract layers of resources, different templates system mapping, and application packaging.  Another challenge is to keep up with changes in the cloud infrastructures features enhancement mapping into abstract resource template.

 The proposal focuses on two main parts:

1. Packaging, in a single, well defined bundle, cloud applications, to enable distribution, provisioning and installation.
2. Application metadata and cloud-native tooling.

An overview of the presentation is available and presented at CNF Task Force in May 2021.

https://wiki.onap.org/download/attachments/93004634/ONAP_Nokia_Ericsson_K8S_application_descriptor_and_packaging_r2.pdf?version=1&modificationDate=1621622424000&api=v2

Application Packaging:

In order to facilitate compatibility with ETSI, ONAP and other telco standards, the CSAR (NFV SOL 004ed351 or ed421) packaging format is used. The only modification is allowing the proposed Application Service Descriptor (ASD) to be a top-level descriptor for the package, instead of an NSD or VNFD as defined in SOL001.

Additionally, the following directories will exist inside the CSAR:

• deployment_artifacts: where all deployment files go, like Helm charts.
• images: holds container images referenced from the main application and dependencies, in OCI format.(ref: https://github.com/opencontainers/image-spec)

Note that the “images” directory may be empty, or only contain a part of the images required for the whole application. This might happen for example in application update packages, where the container images may have already been onboarded onto the associated registry. In case images are present, they must be referenced in the CSAR manifest.

This application packaging format is designed to support a single, container-based deployment “flavor” or “type”. If an application has multiple such deployment types, there should be multiple packages, with their own appropriate descriptors.

An orchestrator is expected to load any container images present in the package onto the correct registry for the target cluster(s) before attempting to deploy the application. Since the images are in OCI format, they should follow the OCI image layout specification, and MUST contain a “name” annotation in their index.json layout descriptor. This name should be used as the “tag” value when the orchestrator provisions the image in the corresponding registry.

Application metadata and cloud-native tooling:

In order to describe the containerized application to an orchestrator, there is a need for some metadata to accompany the bundled cloud-native deployment artifacts (e.g. Helm files) and images.

A basic decision that will affect this metadata is what exactly the cloud-native deployment artifacts are, and what implications that has for the way the orchestrator communicates with the cluster.

The primary decision is to require that applications are deployed using Helm v3 or later (ref: https://helm.sh/docs/helm/helm/), and therefore the deployment artifacts are one or more Helm Charts. Furthermore, in order not to limit future choices of tooling, or tooling choices creating dependencies between the orchestrator and the underlying Kubernetes cluster, it is assumed that the interface between the orchestrator and the cluster is the Kubernetes API (ref: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.21/).

Figure xx: Interaction between orchestrator and cluster

This implies that the orchestrator

• Has an embedded Helm v3 client and is able to use it to deploy the artifacts embedded in the package.
• If it requires any information present in the K8S resource descriptions, it is ably to pre-render the Helm Charts and extract such information.

The most widely used format for describing telco virtualized applications is the ETSI MANO Virtualized Network Function Descriptor (VNFD), ETSI NFV SOL001 specification. This descriptor was created as a vendor neutral way to fully describe a virtualized application, with highly detailed requirements ranging from placement rules, the various virtual components like virtual NICs, CPU and memory requirements, scaling policy, input parameters, and monitoring parameters, etc.

ETSI sought to extend the existing VNFD to cover containerized workloads. It has been pointed out that such efforts have noticeable drawbacks:

• The overall proposed model is essentially duplicating the workload description that K8S (and Helm) provide, but so far with less features.
• The ETSI NFV VNFD (VM and containerized based) definitions could conflict with the Helm Chart definitions, which can cause orchestration confusion and/or failure.
• Non-ETSI-based CNF model and orchestration desire a simplified CNF descriptor

Therefore, this proposal is proposing a new, simple descriptor, the Application Service Descriptor (ASD) with the minimum information for the orchestrator, and pointers to cloud-native artifacts and code (including configuration) required for the LCM implementation. An ASD can describe a complete application / NF, or parts of application / NF.

The ASD allows a clean separation between high-level orchestration, focused on service and resource models, and cloud-native application deployment, implemented via Helm Charts.

Application Service Descriptor model

The tables below summarize the Application Service Descriptor contents.

The overall objective is to keep the items in the descriptor to the bare minimum information, and not duplicate any attributes that might be instead extracted from the Helm Charts. This helps maintain the principle that Helm Charts are the primary deployment artifact for a containerized application and avoids any possible source of error or confusion that such duplication would cause.

Application Service Descriptor (ASD) Information Element (top level)

The initial attributes essentially describe the application – a unique identifier, a schema version (that enables versioning the data model of the descriptor itself), and basic metadata, like application name and version and human-readable descriptive fields.

The next two attributes “extraServiceRequirement” and “enhancedClusterCapabilities” will be used for multi-app or multi-cloud orchestration – a description of exposed endpoints (to enable orchestrators to string together or optimally place linked applications), and two fields that list extra capabilities required from clusters.

Finally, “deploymentItems” is a list of deployment items, i.e. Helm Charts, that together can deploy an application. The table below shows the information element of these deployment item descriptors.

Deployment Item Information Element

asdExtCpd Information Element

NOTE 1: Corresponds more or less to a virtual_link requirement in ETSI NFV SOL001.
NOTE 2: Applies only for ExtCpds representing secondary network interfaces in a pod.
NOTE 3: Several ExtCpd may refer to same additional network interface requirements.

asdExtCpd, NetworkInterfaceRequirements Information Element

datatype.ExtCpd.ParamMappings Information Element

Note 1: When the asdExt Cpd represent a networkRedundant/mated-pair of sriov interfaces, there are references to 2 or 3 related NADs needed to be passed, lwhile for other interface types only one NAD reference is needed to be passed.

Note 2: The format of the Content strings is specific for each different orchestration templating technology used (Helm, Teraform, etc.). Currently only a format for use with Helm charts is suggested: ”helmchartname:[subchartname.]0..N[parentparamname.] 0..Nparametername”. Whether the optional parts of the format are present depends on how the parameter is declared in the helm chart.

enhancedClusterCapabilities Information Element