...
- Support for provisioning ASD-based CNFs using an external K8s Manager
- Instantiate and optionally overwrite the values of an application
- Terminate Delete an application and optionally remove all relevant Persistent Volume Claims and Persistent Volumes
- Change application details using the change package function, which allows to change the deployment version of the CNF thru an upgrade or rollback (stretch goal)
- Leverage and enhance SO capabilities and adding new capabilities for ASD- and Helm-based CNF orchestration
- Orchestrator shall support the capability to use the deployment parameters from ASD for the application or CNF deployment. These deployment parameter values shall correspond to the parameters defined in the “lifecycleParameters” section(s) of the ASD.
- Orchestrator shall support the capability to construct a values file from instance specific parameter values provided at deployment time, and default values supplied in the chart.
- Orchestrator shall support the capability to perform a chart render into concrete K8S resource descriptions.
- Container resource management for determining placement for CNF application on certain K8S cluster(s), orchestrator shall support the capability to parse the workload descriptors and extract those values.
...
Epic | Description | JIRA | priority | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration | SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration (deployment and configuration) |
| 1 | ||||||||
User Story
Function | Affected Component | User Story & Description | JIRA | Priority | PoC Scope Y/N |
---|---|---|---|---|---|
SO shall get the ASD-based CNF package from SDC and store its metadata to SO Catalog DB |
ASDC Controller, CatalogDB Adapter, CatalogDB | SO shall get the ASD-based CNF package (SDC Service CSAR) from SDC and store its metadata to SO Catalog DB. |
Pre Condition:
- SO SDC Controller, CatalogDB Adapter and CatalogDB component instances are running and ready to get SDC notifications
Post Condition:
- SO Catalog DB contains the ASD-based CNF package metadata and artifacts
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
|
|
Pre Condition:
Post Condition:
|
| 1 | |||||||||
For ASD-based CNF provisioning, SO shall process model info |
, decide flows and invoke SO CNFM for AS LCM | API Handler, RequestDB Adapter, RequestDB, SO BPMN Infra, AAI SO CNFM |
For ASD-based CNF provisioning, SO shall process model info and decide flows
Pre Condition:
- SO CNFM is running and ready to receive requests
- SO CatalogDB contains ASD Model metadata
- Resource VF TOSCA.metadata "ASD" entry_definition_type
- SO Client provides parameters based on the ASD lifecycleParameter list
- Existing SO requests will be used to invoke SO
- The SO Request should include the parameters required for the ASD LCM Restful Protocols. See ASD LCM RESTful Protocols for SO CNF Manager for the required parameters
- Instantiation time parameters such as network parameters and helm chart values overrides will be passed thru the above ASD LCM RESTful Protocols to invoke SO CNFM
- note: use of CDA and CDS is a future consideration - out of scope
- The selection of the cloud-region (which represents K8S cluster) comes from the SO Service creation request
Post Condition:
- SO CNF Manager (SO CNFM) receives requests from the SO BPMN Infra and gets ready to process the requests
- After the CNF Manager process, SO shall update CNF to AAI
Enhance SO API Handler and BPMN Infra workflow(s) for AS LCM
Post Condition:
|
|
| 2 |
SO BPMN Infra shall trigger Create AS Instance workflow(s) |
- ONAP Admin creates new cloud region(s) and tenant(s) in AAI
- Each K8S cluster is registered as a cloud-region in AAI, but K8S cluster connectivity info will not be stored in AAI
- In CNFO, MultiCloud K8S Plugin will hold the K8S connectivity info
- In our case, SO CNFM will hold the K8S connectivity info
- See the SO CNFM register/deregister K8S clusters operations for more details
- note: CNFO AAI Model K8S Resource Object Relations
- Create new BPMN workflows for ASD-based CNF workflows
- Design to invoke SO CNF Manager
- Configure SO MacroFlows configurations for invoking ASD-based CNF workflows
Task: SO API Handler Enhancement
- SO API Handler receives a service (or a la carte: make a design decision) request for ASD-based CNF and stores the request information into the Request DB
- Note: expect no change or minimum changes
Task: SO Create Service Instance to AAI
- SO shall create service instance to AAI, leveraging existing AAI schema service
- expect no code impact
Task: SO BPMN Infra processes for ASD-based VF
- SO BPMN Infra decomposes Service into VF Resource(s), and per VF resource, SO BPMN Infra process AS resource handling
- If the VF resource metadata indicates the ASD-based VF (e.g., entity_definition_type='ASD' or 'asd'), SO shall process ASD-based CNF workflows
Task: SO BPMN Infra Invocation for SO CNFM for AS LCM
- SO BPMN Infra shall delegate ASD-based CNF orchestration to SO CNFM
- Extract required parameters from the Service request body
- With the Service request input parameters, formulate request messages by conforming to ASD LCM Restful Protocols (ASD LCM RESTful Protocols for SO CNF Manager ) and its swagger file (ASD LCM RESTful Protocols for SO CNF Manager)
- for CreateASRequest, get:
- asdId
- asInstanceName
- asInstanceDescription
- for InstantiateAsRequest, get:
- asdExtCpdInputParams
- deploymentItems
- additionalparams
- for TerminateAsRequest, get:
- terminationType
- gracefulTerminationTimeout
- addiontalParams
- for Delete AS, get:
- asInstanceId as a parameter
- for CreateASRequest, get:
- As a SO CNFM client, use the following client operations to invoke SO CNFM
SO CNFM,
ASD Repository,
Helm Artifact Repository,
OOF,
AAI,
SDNC Adapter,
SDNC
SO CNFM shall process ASD-based CNF Lifecycle orchestration
Pre Condition:
- ASD and App onboarding packages are stored in the Catalog Registry
- Helm Chart(s) are stored in the Helm Artifact Registry
- Image(s) are stored in the Image Registry
- Target K8S Cluster(s) are ready
- Cloud-Region and Tenants are defined in AAI
AAI shall support CRUDQ of ASD-based CNF Resources
- Leverage existing AAI APIs
- Leverage AAI K8S Resource schema which was defined for CNFO
- Take a look at the CNFO logic how to interface with AAI
- Investigate any new AAI APIs for ASD-based CNF Resources (future consideration)
Post Condition:
- ASD-based CNF is provisioned by SO CNFM
Note: in the initial PoC, ASD external CPDs parameters for the primary network will be used (e.g., loadbalancerIP, externalIPs) to configure the K8s service or ingress controller that the ExtCpd represents
Note: ASD Registry, Helm Artifact Registry, Image Artifact Registry will be handled by another Epic, which is defined in Application Package Distribution
Jira Legacy | ||||||
---|---|---|---|---|---|---|
|
Task: Create SO CNFM and make it available in ONAP
- Create SO CNFM as an SO sub-component, with NBI, Business Logic and SouthBound Plugin for 1) Helm Client or 2) CNF Adapter
- In this PoC, the Helm Client SouthBound Plugin will be used
- use of the CNF Adapter is a future consideration.
- Make SO CNFM POD is deployable in OOM
- Register SO CNFM POD to AAI automatically to be recognized
Task: Create ASD LCM REST API Swagger
- Create SO CNFM REST API swagger, based on the ASD LCM Restful API, ASD LCM RESTful Protocols for SO CNF Manager and Swagger file (ASD LCM RESTful Protocols for SO CNF Manager)
- Note: the swagger file is created and it needs to be refined further
Task: Support for SO CNFM NBI API Handler
- SO CNFM shall support its NBI REST Apis to handle requests from SO.
- Create SO CNFM NBI API Handler sub-component based on ASD LCM Restful Protocols swagger (ASD LCM RESTful Protocols for SO CNF Manager)
- Support the following service operations:
- For multiple operator K8S Cluster support, the following operations are supported:
Note: for the initial PoC, registration and deregistration could be out of scope (stretch goal)
Task: SO CNFM Input Parameter Handling
- SO CNFM shall process input parameters that came thru SO BPMN Infra, by conforming to ASD LCM Restful Protocols, such as:
- DeploymentItem parameters
- The key value pairs, which is to override the default values.yaml
- ExtCpdParams parameters
- The parameters will be used to resolve input in the Helm Charts
- For the initial PoC, only loadbalancerIP or externalIPs will be used
- DeploymentItem parameters
Task: SO CNFM Accesses ASD Registry for Getting ASD
- SO CNFM shall communicate with the ASD Registry to get ASD (descriptor) and artifacts from the ASD Registry Manager
- Use the asdId (which is received during the Create AS operation) to retrieve an ASD
- Get ASD from the ASD Registry thru the ASD Registry Manager APIs
- e.g., GET /api/asds/<name>
Note: for initial PoC, it is possible ASD is received from SDC (tbd)
Task: SO CNFM Processes ASD
- SO CNFM shall decompose the received ASD and get the associated DeploymentItems lists
- Get HelmChart references from the DeploymentItems
- use the 'artifactId' attribute for Helm chart references
- Get the corresponding Helm Charts from the Helm Registry
e.g., helm chart pull <artifactId>
- Get HelmChart references from the DeploymentItems
Note: for initial PoC, Helm Charts are received from SDC (tbd)
Task: Create SO CNFM Instance Database Management
- Create SO CNFM Database tables (not necessarily, RDBMS: make a design design) to store:
- incoming requests
- custom values.yaml files
- Provides Database Access Objects (DAO) for CRUD for SO CNFM
Task: SO CNFM Processes Instance-Level Helm Charts
- SO CNFM shall support the capability to construct custom values file(s), by:
- applying Instance-level input parameter parameter values which came thru Task #3.4
- enhancing helm charts with resolved input parameter values which came thru Task #3.4
- generating new custom values file(s), based on the DeploymentItem parameter values which came thru Task #3.4
- SO CNFM shall store the generated helm charts and/or values files to the SO CNFM database
Task: SO CNFM Transforms Enhanced Helm Charts to K8S Resource Description
- SO CNFM shall transform enhanced Helm charts to K8S resource description (e.g., helm template or helm install --dry-run) to verify Helm charts and examine potential K8S resource requirements
Task: SO CNFM Gets an AS Placement Information for a Target K8S Cluster
- SO CNFM shall get placement information by passing K8 resources + ASD' + additional data to the Placement component such as OOF.
- Note: Use of OOF could be out-of-PoC-Scope (TBD)
- for PoC, it is allowed to choose predefined K8S Clusters
Task: SO CNFM Makes a AS Placement Decision for a Target K8S Cluster
- With the generated K8S resources description(s), for each K8S resource description (per Helm chart), SO CNFM shall make a placement decision based on the decision from the Placement component.
- In the initial PoC, a pre-selected K8S cluster (selected by the client) will be used.
- In the future, ASD enhancedClusterCapabilities attribute will be used to choose a proper/capable K8S cluster. Use of OOF is also a future consideration.
Task: SO CNFM Supports Target K8S Clusters Registration
- SO CNFM shall support Registration of K8S cluster(s)
- To instantiate an AS on an non-ONAP K8S cluster, a cluster configuration file that is specific to the cluster must be uploaded.
- To add a cluster configuration file of a cluster, create a POST request .../aslcm/v1/clusterconfigs will be performed.
- SO CNFM receives the clusterconfigs info and creates a cluster configuration file (cluster name + "." + "config" to the ./kube directory.
- Note:
- The cluster configuration file for a particular cluster must be retrieved from the cluster administrator.
- The Cluster configuration file pre-requisites are:
- It must start with an alphanumeric character
- It can only contain alphanumeric characters, dashes (-_, or underscores (_)
- It must end with .config
- Should the cluster configuration file change for any reason, e.g., CA certificate rotation on the target cluster or client key expires, then the cluster file registered in SO CNFM/AAI shall need to be updated.
- The target cluster server and port must be reachable from the SO CNFM.
- Verify the connection to the target cluster
Task: SO CNFM Supports Target K8S Clusters Deregistration
- To remove a cluster configruation file, create a DELETE request. .../aslcm/v1/clusterconfig/{configName}
- The configName would include the K8S Cluster name as the file prefix
- CNFM will remove the "configName" + "." + "config" file from the .kube directory.
Task: SO CNFM Provides a List of Registered K8S Clusters
- To get details about registered clusters, create a GET request .../aslcm/v1/clusterconfigs
- The API returns a paginated response, but if a customized response is needed, additional parameters for page, size, or and filtering could be applied.
Task: SO CNFM Helm Client Process for AS Deployment
- For the initial PoC, the Helm Client will be used as a SO CNF Southbound plugin for interfacing with K8S Cluster(s)
- An ASD can contain multiple DeploymentItems (which are corresponding Helm Charts). To determine the order of Helm Chart executions, the deploymentOrder from the ASD>DeploymentItems will be used.
- Based on the deploymentOrder, the SO CNFM Helm Client does:
- From the registered K8S Clusters, choose a target K8S cluster name
- With the helm command with --kubeconfig or kube-contenxt option, set the target K8S cluster to work with.
- Invoke the helm install command towards the target K8S cluster, passing the enhanced helm chart and custom values file.
Task:
Per DeploymentItem, SO CNFM shall send (with cluster id, parameter, cloud artifacts) to the SO CNF Adapter for the connection to K8S pluginSupport Helm Template / Dry-run, Helm Install, Helm Uninstall and Helm UpgradeSO CNF Adapter APIs are being studied.
Note: for the initial PoC, the SO CNF Adapter will not be used.
Task: SO CNFM Updates the AS CNF Instance to AAI
- SO CNFM shall update the AS CNF instance to AAI, by leveraging AAI APIs
SO Client shall send requests for ASD-based CNF orchestration
(note: E2E support is out of PoC scope)
SO Client,
SO,
SO BPMN,
SO CNFM
SO Client shall send requests for ASD-based CNF orchestration.
Post Condition:
|
| 2.1 | |||||||||||
SO BPMN Infra shall trigger Instantiate AS Instance workflow(s).
Pre Condition:
Post Condition:
|
| 2.2 | |||||||||||
SO BPMN Infra shall trigger Delete AS Instance workflow(s).
Pre Condition:
Post Condition:
|
| 2.3 | |||||||||||
ONAP Admin Creates Cloud Region(s) and Tenant(s) in AAI Note: use existing AAI admin interfaces (no SO code impact)
|
| 2.4 | |||||||||||
In BPMN Infra, create the Create AS Workflow(s) to launch SO CNFM for Create AS
Post Condition:
|
| 2.5 | |||||||||||
Enhance Instantiate AS Business Logic (Java code) to launch SO CNFM for Instantiate AS
Post Condition:
|
| 2.6 | |||||||||||
Enhance Delete AS Workflow(s) to launch SO CNFM for Delete AS
Post Condition:
|
| 2.7 | |||||||||||
SO CNFM shall process ASD-based CNF Lifecycle orchestration | SO CNFM, ASD Repository, Helm Artifact Repository, OOF, AAI, SDNC Adapter, SDNC | Create SO CNFM and make it available in ONAP
Post Condition:
|
| 3 | |||||||||
Create ASD LCM REST API Swagger
Post Condition:
|
| 3.1 | |||||||||||
Create SO CNFM NBI API Handler based on ASD LCM Restful Protocol swagger, no-ops
Post Condition:
|
| 3.2 | |||||||||||
Implement Create AS Business Logic in SO CNFM NBI Handler to invoke the Create AS workflows(s) Post Condition:
|
| 3.3 | |||||||||||
Implement Instantiate AS Business Logic in SO CNFM NBI Handler to invoke the Instantiate AS workflows(s) Post Condition:
|
| 3.4 | |||||||||||
Implement Delete AS Business Logic in SO CNFM NBI Handler to invoke Delete AS workflows(s) Post Condition:
|
| 3.5 | |||||||||||
Create SO CNFM Workflow(s) for Create AS
Post Condition:
|
| 3.6 | |||||||||||
Create SO CNFM Workflow(s) for Instantiate AS
Post Condition:
|
| 3.7 | |||||||||||
Create SO CNFM Workflow(s) for Delete AS
Post Condition:
|
| 3.8 | |||||||||||
SO CNFM accesses ASD Registry for getting ASD for Create AS
Post Condition:
|
(moved to London release) | 3.9 | |||||||||||
SO CNFM Processes ASD & Retrieves DeploymentItems
Note: for initial PoC, Helm Charts are received from SDC (tbd) Post Condition:
|
| 3.10 | |||||||||||
SO CNFM Input Parameter Handling and Instance-Level Helm Charts
Note: enhancedClusterCapbilities input parameter handling is a stretch goal. Post Condition:
|
| 3.11 | |||||||||||
Generate and replace values file based on instance variable
Post Condition:
|
(moved to London release) | 3.12 | |||||||||||
SO CNFM Transforms Enhanced Helm Charts to K8S Resource Description
Note: this transformation is for a placement decision. If the PoC placement logic is simple or predefined, this could be skipped. Post Condition:
|
(could be not part of the initial PoC) | 3.13 | |||||||||||
SO CNFM Gets an AS Placement Decision for a Target K8S Cluster
Post Condition:
|
(moved to London release) | 3.14 | |||||||||||
SO CNFM Supports Target K8S Clusters Registration
| TBD (registration is supported, but more to be enhanced in London; moved to London+) | 3.15 | |||||||||||
SO CNFM Supports Target K8S Clusters Deregistration
| TBD (moved to London) | 3.16 | |||||||||||
SO CNFM Provides a List of Registered K8S Clusters
| TBD (moved to London) | 3.17 | |||||||||||
SO CNFM Helm Client Process for AS Deployment
Post Condition:
|
| 3.18 | |||||||||||
Note: for the initial PoC, the SO CNF Adapter will not be used. | N/A | ||||||||||||
SO CNFM Updates the AS CNF Instance to AAI
Post Condition:
|
| 3.19 | |||||||||||
SO Client shall send requests for ASD-based CNF orchestration (note: E2E support is out of PoC scope) | SO Client, SO, SO BPMN, SO CNFM | SO Client shall send requests for ASD-based CNF orchestration to SO
Pre Condition:
Post Condition:
|
|
Task: SO Client Invokes SO for AS LCM orchestartion
- Provide required CNF instance input parameters, which are defined in the ASD LCM RESTful Protocols to the SO request
- Send AS LCM orchestration requests to SO
| 4 |
Overall Process
- Pre-Conditions
- SDC accepts onboarding App packages, including ASD and DeploymentItems models, Helm Charts, Images and other artifacts, what allows to keep decomposition of Service instance
- SO subscribes and receives package notifications from SDC
- ASD-Based CNF LCM Orchestration
- Based on the notifications, SO ASDC Controller queries for the App packages from SDC, and stores models and artifacts to SO Catalog Database
- MACRO workflow in SO is used for orchestration
- ASD supports multiple Helm Charts in CNF packages.
- ASD instance will be decomposed to find its associated deployment item(s).
...
Gliffy | ||||||||
---|---|---|---|---|---|---|---|---|
|
Future NS Consideration
The following diagram depict the NS connection of ASDs.
Gliffy | ||||
---|---|---|---|---|
|
AS LCM Restful API
Please refer to the section, ASD AS LCM RESTful Protocols for SO CNF Manager .
- Swagger file (ASD AS LCM RESTful Protocols for SO CNF ManagerManager#SwaggerFile)
Gliffy | ||||||||
---|---|---|---|---|---|---|---|---|
|
...
- <Instantiate an AS>
- BPMN Infra sends an AS instantiate request to SO CNFM with the asInstanceId as follows:
- POST .../as_instances/{asInstanceId}/instantiate (InstantiateAsRequest)
- InstantiateAsRequest
Attribute Name
Data Type
Cardinality
Description
asdExtCpdInputParams
ExtCpdParams
0..N contains ext cpd parameter instance-level value deploymentItems
DeploymentItems 1..N contains lifecycle parameters for deploymentItems additionalParams KeyValuesPairs 0..1 Additional input parameters for the instantiation process (this is a pace holder to hold any additional parameters which are needed by SO CNFM)
- SO CNFM retrieves the corresponding AsInstance from its DB; it is retrieve the "asdId" for the ASD query
- SO CNFM queries an ASD with the asdId from the ASD Repository; caches the retrieved ASD in the memory during the Instantiate operations
- SO CNFM reads thru the ASD DeploymentItems, and per deploymentItems, SO CNFM queries for the associated Helm Chart (1:1) from the Helm Chart Repository
- caches the retrieved Helm Charts in the memory during the Instantiate operations
- SO CNFM reads the deploymentItems.deploymentOrder. Based on the order sequence, SO CNFM processes the deploymentItems one by one
- For each, deployment item,
- SO CNFm creates vf-modules in AAI
- SO CNFM assignes vf-modules in SDNC
- From the InstantiateAsRequest, SO CNFM retrieves the deploymentItems
DeploymentItems
deploymentItemId Identifier 1 Identifies which deploymentItem lifecycleParameterKeyValues
KeyValuesPairs 0..N provides lifecycle parameter keys and values - The lifecycleParameterKeyValues contains a list of customizable attributes (key) in the values.yaml with instance-level values
- From the associated Helm Chart, SO CNFM gets the values.yaml
- SO CNFM creates a new values.yaml, based on the retrieved values.yaml + lifecycleParameterKeyValues
- ==== SO CNFM processes the asdExtCpdInputParam ==== TBD
- SO CNFM performs "helm template " to render K8S resource template(s)
- With the rendered k8S resource template(s), SO CNFM gets a placement decision from the Placement component (e.g., OOF)
- Currently, use of OOF is out of the scope from the initial PoC
- In the initial PoC, a simplified placement function will be used
- Based on the placement decision, SO CNFM determines the target K8S cluster
- Set the Helm command environment to connect to the target K8S cluster
- set .kube/{target K8S cluster name}.config
- SO CNFM invokes "helm install" command with the corresponding Helm Chart and a new values.yaml
- SO CNFM will have a few South-Bound plugin (helm client, CNF Adapter, others)Adapter, others)
- in the initial PoC, the helm client will be used
- SO CNFM Helm Client will select a target Kubernetes cluster
- e.g., helm install <release> <chart> --kubeconfig ~/.kubeconfigs/<cluster_name>.kubeconfig
- in the initial PoC, the
- target cluster is selected by the user and its name will be passed thru the user params (in SO) and additionalParams (in SO-CNFM)
- If successful, SO CNFM update the corresponding vf-module in the AAI
- BPMN Infra sends an AS instantiate request to SO CNFM with the asInstanceId as follows:
...
POST .../as_instances/{asInstanceId}/change_aspkg
Terminate an AS Instance
This operation terminates an AS instance.
POST .../as_instances/{asInstnaceId}/terminate (TerminateAsRequest)
Note: for PoC, GRACEFUL termination type is not supported.
...
Note: optionally, Delete an AS Identifier could clean up resources of an AS Instance, e.g., Persistent Volumes (PVs)
The Delete operation may need to update AAI.
===========
Register K8S Clusters
...
- Helm Install
- Helm Uninstall
- Helm Upgrade
Note: Helm is a package manager. Managing how instances (helm release) of packages are run in an environment is a separate concern. For Helm release handling, we are considering additional tools and mechanisms.
Component Communication Security
...