Page Comparison

Table of Contents

Note: Work in Progress

...

• Support for provisioning ASD-based CNFs using an external K8s Manager
  • Instantiate and optionally overwrite the values of an application
  • Terminate Delete an application and optionally remove all relevant Persistent Volume Claims and Persistent Volumes
  • Change application details using the change package function, which allows to change the deployment version of the CNF thru an upgrade or rollback (stretch goal)
• Leverage and enhance SO capabilities and adding new capabilities for ASD- and Helm-based CNF orchestration
  • Orchestrator shall support the capability to use the deployment parameters from ASD for the application or CNF deployment. These deployment parameter values shall correspond to the parameters defined in the “lifecycleParameters” section(s) of the ASD.
  • Orchestrator shall support the capability to construct a values file from instance specific parameter values provided at deployment time, and default values supplied in the chart.
  • Orchestrator shall support the capability to perform a chart render into concrete K8S resource descriptions.
  • Container resource management for determining placement for CNF application on certain K8S cluster(s), orchestrator shall support the capability to parse the workload descriptors and extract those values.

...

JIRA:

Epic

...

SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration (deployment and configuration)

...

...

User Story

...

SDC Controller,

CatalogDB Adapter,

CatalogDB

...

SO shall get the ASD-based CNF package (SDC Service CSAR) from SDC and store its metadata to SO Catalog DB.

Pre Condition:

• SO  SDC Controller, CatalogDB Adapter and CatalogDB component instances are running and ready to get SDC notifications

Post Condition:

• SO Catalog DB contains the ASD-based CNF package metadata and artifacts

...

Concepts

• K8S CISM Cluster: Container Infrastructure Service Manager Cluster performs the same function as VIM Zone but operates on containerized application level.
• CNF: Cloud Native Network Function. Containerized VNF is designed to be deployed in the cloud as a container. CNF is a better fit for microservice architecture due to the deployment size.
• Kubernetes: Kubernetes (K8s) is an open-source system for automating deployment, scaling and management of containerized applications
• POD: A Pod is a group of one or more containers (such as Docker containers), with shared storage/network, and a specification for how to run the containers.

Scope

In Scope

See the Epic and User Stories.

Out of Scope

TBD

Epic

User Story

Overall Process

• Pre-Conditions
  • SDC accepts onboarding App packages, including ASD and DeploymentItems models, Helm Charts, Images and other artifacts, what allows to keep decomposition of Service instance
  • SO subscribes and receives package notifications from SDC
• ASD-Based CNF LCM Orchestration
  • Based on the notifications, SO ASDC Controller queries for the App packages from SDC, and stores models and artifacts to SO Catalog Database
  • MACRO workflow in SO is used for orchestration
  • ASD supports multiple Helm Charts in  CNF packages. 
  • ASD instance will be decomposed to find its associated deployment item(s). 

ASD-Based App Package Type

• The Resource VF(s) in the onboarded Service CSAR will have the following resource main manifest file
  
  • application_name
  • application_provider
  • release_date_time
  • entry_definition_type [ "asd"]
• SO distinguishes the App package based on the "entry_defintion_type" metadata.
  • If it is "asd", SO will process the package as the ASD-based CNF.
  • SO delegates the ASD-based CNF orchestration to the CNF Manager

LCM Architecture Overview

The following diagram depicts LCM Architecture for the ASD-based CNF.

SO Internal Architecture

The following diagram depicts SO internal architecture for the ASD-based orchestration:

AS LCM Restful API

Please refer to the section, ASD LCM RESTful Protocols for SO CNF Manager .

• Swagger file (ASD LCM RESTful Protocols for SO CNF Manager)

ASD LCM Operations

<as_create_instantiate.bpmn example>

Image Removed

Process Sequence

Create and Instantiate AS

• <Preconditions>:
  • Registar K8S Cluster(s) to ONAP
  • During the registration, SO CNFM creates .kube/{target K8S cluster name}.config file(s)

• <SO BPMN Infra>
  • BPMN Infra workflow gets VF Resource TOSCA.meta. 
  • In there, the entity_defintion_type is found and it is set to "asd", BPMN infra launches the AS LCM Workflow as the above.
  • The workflows sends a request for Create AS Identifier to SO CNFM

...

CreateAsRequest:

...

asdId

...

asInstanceDescription

...

asInstanceName

...

The client passes "asdId" to SO CNFM, so CNFM can query an ASD with the asdId from the ASD Repository

...

• The returned AsInstance contains the asInstanceId
• Subsequent operations uses the asInstanceId; i.e., the Client use the asIstanceId in the REST Api path

...

Attribute Name

...

Data Type

...

Cardinality

...

Description

...

asdExtCpdInputParams

...

ExtCpdParams

...

deploymentItems

...

• caches the retrieved Helm Charts in the memory during the Instantiate operations

...

DeploymentItems

...

lifecycleParameterKeyValues

...

• Currently, use of OOF is out of the scope from the initial PoC
• In the initial PoC, a simplified placement function will be used

...

• set .kube/{target K8S cluster name}.config

...

• SO CNFM will have a few South-Bound plugin (helm client, CNF Adapter, others)
• in the initial PoC, the helm client will be used

...

• <Postconditions>:
  • each POD is a Ready status. This ensures that each POD can serve requests.
  • Some of the PODs will not reach the Ready status until a day 1 configuration is applied
  • The PODs still need verification of containers with a specific ContainerState

AS LCM APIs

Create an AS Identifier

In order for an application (CNF) to be eligible for instantiation, a CNF identifier must be created to identify the application service as a unique instance.

An AS instance supports multiple deployment items (helm charts). So, the AS instance id could be used for helm release names as the prefix.

Query AS Instance Identifiers

To list all AS instances, make a GET request to SO CNFM.

GET .../aslcm/v1/as_instances

Query an AS instance by Instance ID

To query a specific AS instance by its instance ID, make a GET request to SO CNFM.

GET .../aslcm/v1/as_instances/{asInstanceId}

Instantiate an AS

• To instantiate an AS, make a poST request to SO CNFM. The target cluster name is included in the requrest parameter.

POST .../as_instances/{asInstanceId}/instantiate

• if some of the parameters required for instantiation are complex and cannot fit into the additionalParams section of the JSON request body (TBD).

Upgrading an AS

This operation is referred to as the changing of an AS package.

POST .../as_instances/{asInstanceId}/change_aspkg

Terminate an AS Instance

This operation terminates an AS instance.

POST .../as_instances/{asInstnaceId}/terminate (TerminateAsRequest)

Note: for PoC, GRACEFUL termination type is not supported.

Delete an AS Identifier

This operation deletes an AS Identifier.

DELETE .../as_instances/{as_instanceId}

Note: optionally, Delete an AS Identifier could clean up resources of an AS Instance, e.g., Persistent Volumes (PVs)

===========

Register K8S Clusters (checking if AAI can be used)

Overall Process

• Pre-Conditions
  • SDC accepts onboarding App packages, including ASD and DeploymentItems models, Helm Charts, Images and other artifacts, what allows to keep decomposition of Service instance
  • SO subscribes and receives package notifications from SDC
• ASD-Based CNF LCM Orchestration
  • Based on the notifications, SO ASDC Controller queries for the App packages from SDC, and stores models and artifacts to SO Catalog Database
  • MACRO workflow in SO is used for orchestration
  • ASD supports multiple Helm Charts in  CNF packages. 
  • ASD instance will be decomposed to find its associated deployment item(s). 

ASD-Based App Package Type

• The Resource VF(s) in the onboarded Service CSAR will have the following resource main manifest file
  
  • application_name
  • application_provider
  • release_date_time
  • entry_definition_type [ "asd"]
• SO distinguishes the App package based on the "entry_defintion_type" metadata.
  • If it is "asd", SO will process the package as the ASD-based CNF.
  • SO delegates the ASD-based CNF orchestration to the CNF Manager

LCM Architecture Overview

The following diagram depicts LCM Architecture for the ASD-based CNF.

SO Internal Architecture

The following diagram depicts SO internal architecture for the ASD-based orchestration:

Future NS Consideration

The following diagram depict the NS connection of ASDs.

Image Added

AS LCM Restful API

Please refer to the section, AS LCM RESTful Protocols for SO CNF Manager .

• Swagger file (AS LCM RESTful Protocols for SO CNF Manager#SwaggerFile)

ASD LCM Operations

<as_create_instantiate.bpmn example>

Image Added

Process Sequence

Create and Instantiate AS

• <Preconditions>:
  • Registar K8S Cluster(s) to ONAP
  • During the registration, SO CNFM creates .kube/{target K8S cluster name}.config file(s)

• <SO BPMN Infra>
  • BPMN Infra workflow gets VF Resource TOSCA.meta. 
  • In there, the entity_defintion_type is found and it is set to "asd", BPMN infra launches the AS LCM Workflow as the above.
  • The workflows sends a request for Create AS Identifier to SO CNFM

• <Create an AS Identifier>
  • SO CNFM receives the create request (CreateAsRequest) and processes the request and return a response with AsInstance

    • CreateAsRequest:

      asdId	Identifier (UUID)	1	Identifier that identifies the ASD which defines the AS instance to be created.
      asInstanceDescription	String	0..1	Human-readable description of the AS instance to be created.
      asInstanceName	String	0..1	Human-readable name of the AS instance to be created.

      
      

    • The client passes "asdId" to SO CNFM, so CNFM can query an ASD with the asdId from the ASD Repository

    • SO CNFM copies the incoming CreateAsRequest attributes into an AsInstance structure, and persists the AsInstance, which has a connection between the asInstanceId and asdId
    • note: SO CNFM will uses the asdId when it queries an ASD from the ASD Repository during the instantiation operation

      
      

  • When the Create AS instance is successful, SO CNFM returns 200 OK with the AsInstance
    • The returned AsInstance contains the asInstanceId
    • Subsequent operations uses the asInstanceId; i.e., the Client use the asIstanceId in the REST Api path

• <Instantiate an AS>
  • BPMN Infra sends an AS instantiate request to SO CNFM with the asInstanceId as follows:
    • POST .../as_instances/{asInstanceId}/instantiate (InstantiateAsRequest)
    • InstantiateAsRequest

      
      

      Attribute Name	Data Type	Cardinality	Description
      asdExtCpdInputParams	ExtCpdParams	0..N	contains ext cpd parameter instance-level value
      deploymentItems	DeploymentItems	1..N	contains lifecycle parameters for deploymentItems
      additionalParams	KeyValuesPairs	0..1	Additional input parameters for the instantiation process (this is a pace holder to hold any additional parameters which are needed by SO CNFM)

      
      

  • SO CNFM retrieves the corresponding AsInstance from its DB; it is retrieve the "asdId" for the ASD query
  • SO CNFM queries an ASD with the asdId from the ASD Repository; caches the retrieved ASD in the memory during the Instantiate operations
  • SO CNFM reads thru the ASD DeploymentItems, and per deploymentItems, SO CNFM queries for the associated Helm Chart (1:1) from the Helm Chart Repository
    • caches the retrieved Helm Charts in the memory during the Instantiate operations
  • SO CNFM reads the deploymentItems.deploymentOrder. Based on the order sequence, SO CNFM processes the deploymentItems one by one
  • For each, deployment item,
    • SO CNFm creates vf-modules in AAI
    • SO CNFM assignes vf-modules in SDNC
    • From the InstantiateAsRequest, SO CNFM retrieves the deploymentItems

      • DeploymentItems

        deploymentItemId	Identifier	1	Identifies which deploymentItem
        lifecycleParameterKeyValues	KeyValuesPairs	0..N	provides lifecycle parameter keys and values

        
        

      • The lifecycleParameterKeyValues contains a list of customizable attributes (key) in the values.yaml with instance-level values
    • From the associated Helm Chart, SO CNFM gets the values.yaml
    • SO CNFM creates a new values.yaml, based on the retrieved values.yaml + lifecycleParameterKeyValues
    • ==== SO CNFM processes the asdExtCpdInputParam ==== TBD
    • SO CNFM performs "helm template " to render K8S resource template(s)
    • With the rendered k8S resource template(s), SO CNFM gets a placement decision from the Placement component (e.g., OOF)
      • Currently, use of OOF is out of the scope from the initial PoC
      • In the initial PoC, a simplified placement function will be used
    • Based on the placement decision, SO CNFM determines the target K8S cluster
    • Set the Helm command environment to connect to the target K8S cluster
      • set .kube/{target K8S cluster name}.config
    • SO CNFM invokes "helm install" command with the corresponding Helm Chart and a new values.yaml
      • SO CNFM will have a few South-Bound plugin (helm client, CNF Adapter, others)
      • in the initial PoC, the helm client will be used
      • SO CNFM Helm Client will select a target Kubernetes cluster
        • e.g., helm install <release> <chart> --kubeconfig ~/.kubeconfigs/<cluster_name>.kubeconfig
        • in the initial PoC, the target cluster is selected by the user and its name will be passed thru the user params (in SO) and additionalParams (in SO-CNFM)
    • If successful, SO CNFM update the corresponding vf-module in the AAI

• <Postconditions>:
  • each POD is a Ready status. This ensures that each POD can serve requests.
  • Some of the PODs will not reach the Ready status until a day 1 configuration is applied
  • The PODs still need verification of containers with a specific ContainerState

AS LCM APIs

Create an AS Identifier

In order for an application (CNF) to be eligible for instantiation, a CNF identifier must be created to identify the application service as a unique instance.

An AS instance supports multiple deployment items (helm charts). So, the AS instance id could be used for helm release names as the prefix.

Query AS Instance Identifiers

To list all AS instances, make a GET request to SO CNFM.

GET .../aslcm/v1/as_instances

Query an AS instance by Instance ID

To query a specific AS instance by its instance ID, make a GET request to SO CNFM.

GET .../aslcm/v1/as_instances/{asInstanceId}

Instantiate an AS

• To instantiate an AS, make a poST request to SO CNFM. The target cluster name is included in the requrest parameter.

POST .../as_instances/{asInstanceId}/instantiate

• if some of the parameters required for instantiation are complex and cannot fit into the additionalParams section of the JSON request body (TBD).

Upgrading an AS

This operation is referred to as the changing of an AS package.

POST .../as_instances/{asInstanceId}/change_aspkg

Terminate an AS Instance

This operation terminates an AS instance.

POST .../as_instances/{asInstnaceId}/terminate (TerminateAsRequest)

Note: for PoC, GRACEFUL termination type is not supported.

Delete an AS Identifier

This operation deletes an AS Identifier.

DELETE .../as_instances/{as_instanceId}

Note: optionally, Delete an AS Identifier could clean up resources of an AS Instance, e.g., Persistent Volumes (PVs)

The Delete operation may need to update AAI.

===========

Register K8S Clusters 

This adds an ability for CNFM to register container managers like Kubernetes. that will allow to instantiate and deploy containerized CNFs. 

Note: Kubernetes is advanced and commonly used CISM as of now, but in the future, adding other CISM types could be considered. for now Kubernetes is a reference implementation.

In order to make CNFM to be able to operate K8S Clusters we need introduce K8S Cluster registration, deregistration and querying features.

• As a user I want to be able to register and list CISM Cluster, which is already deployed.
• As a user I want to name the CISM Cluster and select it’s type.
• As for main information source of k8s cluster type configuration I want to use a kubectl configuration file to feed authentication information and connectivity point information.
• As a user I want to modify and transform information upon uploading (upload certificates, change or select username and credentials).
• As a user I want to be able to query all stored information for clusters both individual cluster information and filtered list by criteria(ie name, type, status, etc).

Information needed for registration

• CISM Cluster Name
• CISM Cluster type (Kubernetes)
• CISM Cluster Server URL
• Username to access API
• Option to populate connectivity details by uploading kubeconfig file

  • Kubeconfig contains connectivity details for the K8s cluster(hostname, port, apiRoot), which includes authentication details, too

• Certificate Authority

========

To instantiate an AS on an non-ONAP K8S cluster, a cluster configuration file that is specific to the cluster must be uploaded.

...

• to verify the connectdion to the target cluster, run the following command from the ONAP K8S cluster
  • kubectl --kubeconfig ${PATH_TO_TARGET_CLUSTER_CONFIGURATION_FILE} get namespaces

Deregister K8S Clusters 

...

To remove a cluster configruation file, create a DELETE request. .../aslcm/v1/clusterconfig/{configName}

...

• Helm Install
• Helm Uninstall
• Helm Upgrade

Note: Helm is a package manager. Managing how instances (helm release) of packages are run in an environment is a separate concern. For Helm release handling, we are considering additional tools and mechanisms. 

Component Communication Security

...