Lifecycle Management Interface

This interface allows the CNF Manager (CNFM) client to invoke AS-CNF lifecycle management operations of AS-CNF instances towards the CNFM.

The operations provide through this interface are:

Create AS Identifier
Instantiate AS
~~Terminate AS~~
Delete AS Identifier
Update AS

note: register and unregister of K8S cluster(s) are under consideration. May use existing AAI interface??? TBD.

Create AS Instance Resource

The creation of an "individual" AS instance resource.

REST Interfaces

POST .../as_instances (CreateAsRequest)
201 Created (AsInstance)
// Send AsIdentifierCreationNotification

post condition: upon successful completion, AS instance is created in NOT_INSTANTIATED state.

For the response AsInstance structure, see the AsInstance section below, AsInstanceforCreateandQuery

The caller of SO CNFM (in this case, BPMN Infra) need to use the return asInstanceId for invoking an AS Instance Instantiate.

Note: KeyValuePairs data type is typically realized as Hashmap or Hashtable

CreateAsRequest

Attribute Name	Data Type	Cardinality	Description
asdId	Identifier (UUID)	1	Identifier that identifies the ASD which defines the AS instance to be created.
asInstanceName	String	0..1	Human-readable name of the AS instance to be created.
asInstanceDescription	String	0..1	Human-readable description of the AS instance to be created.
additionalParams^{1, 2}	KeyValuePairs	0..1	Additional input parameters for the Create AS process (this is a pace holder to hold any additional parameters for the orchestrator, such as CNFM)

Note:

The additional parameters can be passed to define custom values. All keys that are separated by dots are handled as separate values. Passing a value file content is under consideration. Special characters are allowed to represent \n, \., etc.
The target cluster name could be passed thru additionalParams if the client wants to select the target cluster.

AsInstance (response)

see the AsInstance section below, AsInstanceforCreateandQuery

Instantiate AS Instance Resource

Before SO CNFM invokes As Instance Instantiate, SO CNFM may need to query for an asInstance based on the asInstanceId parameter.

REST Interfaces

POST .../as_instances/{asInstanceId}/instantiate (InstantiateAsRequest)
202 Accepted ()
Send asLcmOperationOccurrenceNotification (STARTING/PROCESSING/COMPLETED)
200 OK (AsLcmOpOcc:operationState=COMPLETED)

InstantiateAsRequest

Attribute Name	Data Type	Cardinality	Description
asdExtCpdInputParams	ExtCpdParams	0..N	contains ext cpd parameter instance-level values
deploymentItems	DeploymentItems	1..N	contains lifecycle parameters for deploymentItems
additionalParams^{1, 2}	KeyValuePairs	0..1	Additional input parameters for the Instantiate AS process (this is a pace holder to hold any additional parameters for the orchestrator, such as CNFM)

Note:

The additional parameters can be passed to define custom values. All keys that are separated by dots are handled as separate values. Passing a value file content is under consideration. Special characters are allowed to represent \n, \., etc.
The target cluster name could be passed thru additionalParams if the client wants to select the target cluster.

ExtCpdParams

extCpdId	UUID	1	identifier
loadbalancerIP	String	0..1	contains the IP address to configure the loadBalancer of the K8s service or ingress controller that the ExtCpd represents
externalIPs	String	0..N	contains external IPs
nadNames	String	0..N	contains a list of nad names
nadNamespace	String	0..1	contains a nad namespace

DeploymentItems

Attribute Name	Data Type	Cardinality	Description
deploymentItemId	Identifier	1	Identifies which deploymentItem
lifecycleParameterKeyValues¹	KeyValuesPairs	0..N	provides lifecycle parameter keys and values

Note:

provides instance-level key-value sets for the Helm Charts values file(s).

Terminate AS Instance Resource

The termination of an individual AS instance resource.

REST Interfaces

POST .../as_instances/{asInstanceId}/terminate (TerminateAsRequest)
202 Accepted ()
Send asLcmOperationOccurrenceNotification (STARTING/PROCESSING/COMPLETED)
~~200 OK (AsLcmOpOcc:operationState=COMPLETED)~~

TerminateAsRequest

Attribute Name	Data Type	Cardinality	Description
terminationType	Enum	1	FORCEFUL GRACEFUL
gracefulTerminationTimeout	Integer	0..1	The unit is seconds
additionalParams	KeyValuePairs	0..1	Additional input parameters for the Terminate AS process (this is a pace holder to hold any additional parameters for the orchestrator, such as CNFM)

Delete AS Instance Resource

The deletion of an individual AS instance resource.

REST Interfaces

DELETE .../as_instances/{asInstanceId}
204 No Content
// Send AsIdentifierDeletionNotification to Client

Update AS

TBD

Query Individual AS

REST Interfaces

GET .../as_instances/{asInstanceId}
200 OK (AsInstance)

AsInstance for Create and Query

Attribute Name	Data Type	Cardinality	Description
asInstanceId	Identifier	1	Identifier of the AS instance
asInstanceName	String	0..1	Name of the AS instance. This attribute can be modified with the PATCH (i.e., update) method.
asInstanceDescription	String	0..1	Human-readable description of the AS instance. This attribute can be modified with the PATCH method.
asdId	Identifier	1	Identifier of the ASD on which the CNF instance is based.
asVersion	Version	1	Specifies the version of the Application.
asSchemaVersion	Version	1	Specifies the version of the ASD’s schema. The value is copied from the ASD.
asProvider	String	1	Provider of the AS instance. The value is copied from the ASD.
asApplicationName	String	1	Name to identify the AS instance. The value is copied from the ASD.
asApplicationVersion	String	1	Specifies the version of the Application. The value is copied from the ASD.
asApplicationInfoName	String	0..1	Human readable name for the Application service instance. The value is copied from the ASD.
asInfoDescription	String	0..1	Human readable description of the AS instance. The value is copied from the ASD.
asdExtCpd	datatype.ExtCpd	0..N	Contains the externally exposed “instance-level” connection points of the application.
enhancedClusterCapabilities	datatype. enhancedClusterCapabilities	0..N	Contains a list of “instance-level” expected capabilities of the target Kubernetes cluster to aid placement of the application service on a suitable cluster.
deploymentItems	DeploymentItems	1..N	Contains Deployment artifacts with “instance-level” lifecylceparmaeterKeyValues
instantiationState	String of Enum; NON_INSTANTIATION, INSTANTIATED	0..1	Indicates the current Instantiation State
instantiationAsInfo	datatype.instantiationAsInfo	0..1	Information specific to an instantiated AS Instance, such as STARTED, STOPPED
metadata	object (key value pair)	0..1	represents a list of “instance-level” metadata key-value pairs
extensions	object (key value pair)	0..1	Additional AS-specific “instance-level” attributes that affect the lifecycle management of this AS instance
_links	datatype._links	0..1	Links to resources related to this resource, such as self, indicators, instantiate, terminate and operate URIs
asDeploymentName	String	0..1	name of the deployment in the namespace (e.g., helm release name) Note: once an app is deployed, the asDeploymentName is required
targetCluster	String	0..1	target cluster name

datatype.instantiationAsInfo

Attribute Name	Data Type	Cardinality	Description
asState	String of Enum; STARTED STOPPED	1	Identifier of the AS instance
localizationLanguage	String	0..1	Information about localization language of the AS

Attribute Name

Data Type

Cardinality

Description

asState

String of Enum;

STARTED
STOPPED

Identifier of the AS instance

localizationLanguage

String

0..1

Information about localization language of the AS

datatype._links

Attribute Name	Data Type	Cardinality	Description
self	object	0..1	represents a link to a resource using an absolute URI
>href	String	1	string formatted according to IETF RFC 3986
indicators	object	0..1	represents a link to a resource using an absolute URI
>href	String	1	string formatted according to IETF RFC 3986
instantiate	object	0..1	represents a link to a resource using an absolute URI
>href	String	1	string formatted according to IETF RFC 3986
terminate	object	0..1	represents a link to a resource using an absolute URI
>href	String	1	string formatted according to IETF RFC 3986
operate	object	0..1	represents a link to a resource using an absolute URI
>href	String	1	string formatted according to IETF RFC 3986

Register K8S Clusters

To instantiate an AS on an non-ONAP K8S cluster, a cluster configuration file that is specific to the cluster must be uploaded.

To add a cluster configuration file of a cluster, create a POST request .../aslcm/v1/clusterconfigs will be performed.

CNFM receives the clusterconfigs info and creates a cluster configuration file (cluster name + "." + "config" to the ./kube directory.

The kubectl configuration file is used to the body of the request to define external K8S clusters.

The cluster configuration file for a particular cluster must be retrieved from the cluster administrator.

Should the cluster configuration file change for any reason, e.g., CA certificate rotation on the target cluster or client key expires, then the cluster file registered in SO CNFM/AAI shall need to be updated.

The target cluster server and port must be reachable from the SO CNFM.

to verify the connectdion to the target cluster, run the following command from the ONAP K8S cluster
- kubectl --kubeconfig ${PATH_TO_TARGET_CLUSTER_CONFIGURATION_FILE} get namespaces

Deregister K8S Clusters

To remove a cluster configruation file, create a DELETE request. .../aslcm/v1/clusterconfig/{configName}

CNFM will remove the "configName" + "." + "config" file from the .kube directory.

The command returns the HTTP status code 204 No Content

List Registered K8S Clusters

To get details about registered clusters, create a GET request .../aslcm/v1/clusterconfigs

The API returns a paginated response, but if a customized response is needed, additional parameters for page, size, sor and filtering could be applied.

Swagger File

The following is ASD LCM Restful API Swagger file (work in progress)

ASLifecycleManagement-API-v10.yaml

Kubernetes Resource Query

Get Nodes

GET /api/v1/nodes

with parameters

Based on the node label (which is not unique), get a node list. To narrow it further, use the additional findSelector parameter.

Once we get target node(s), get node status.

Get Node Status

GET /api/v1/nodes/{name}/status

Query Cluster Capacity

Allocatable capacity of CPU and Memory from the target cluster could be discovered by SO CNFM (later this function could be moved to NFVO) using the K8S API (GET /api/v1/nodes/{name}/status).

Name	Description	Required	Schema
cluster_name	name of Kubernetes Cluster	true	String
namespace	namespace	true	String
node_name	name of Kubernetes node	true	String
capacity	Capacity represents the total resources of a node.	false	object
allocatable	Allocatable represents the resources of a node that are available for scheduling. Defaults to Capacity	false	object cpu memory storage
conditions	Conditions is an array of current observed node conditions.	false	v1.NodeCondition array
addresses	List of addresses reachable to the node.	false	v1.NodeAddresses array
daemonEndpoints	Endpoints of daemons running on the Node.	false	v1.NodeDaemonEndpoints
nodeInfo	Set of ids/uuids to uniquely identify the node.	false	v1.NodeSystemInfo
images	List of container images on this node	false	v1.ContainerImage array
volumesInUse	List of attachable volumes in use (mounted) by the node.	false	v1.UniqueVolumeName array
volumesAttached	List of volumes that are attached to the node.	false	v1.AttachedVolume array

Query Cluster POD by name

kubectl get pods -o=name
kubectl get pods -o=name | sed "s/^.\{4\}//" | grep <release name>
kubectl get pods --no-headers -o custom-columns=":metadata.name" | grep <release name>
kubectl get pod -A | grep <release name> | awk '{print $2}'. // find a POD by release name
kubectl get --no-headers=true pods -o name | awk -F "/" '{print $2}'

Get PoD / container status

kubectl get pods <pod name> --no-headers -o custom-columns=":status.phase". // check if pod is running or not
kubectl get pod <pod name> --output="jsonpath={.status.containerStatuses[*].ready}". // check if the container is ready, which is true

Others

kubectl get ns -A
kubectl get pods - n <namespace>
kubectl get pod -o=custom-columns=NAME:.metadata.name,NODE:.spec.nodeName
kubectl get pod -o=custom-columns=NAME:.metadata.name,NODE:.spec.nodeName | awk '{print $2}'. // get the node name
kubectl get pod -o=custom-columns=NAME:.metadata.name,NODE:.spec.nodeName | grep <release name>. // get the pod and node names for release name
kubectl get pod -o=custom-columns=NAME:.metadata.name,NODE:.spec.nodeName | grep <release name> | awk '{print $2}'. // get the node name for release name
kubectl describe pods <pod name> // get a pod description
kubectl describe pods <pod name> | grep 'Container ID'. // get container id(s) from a pod
kubectl get pod <pod name> -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainers[*].name,CONTAINERS:.spec.containers[*].name". // get pod, init-container and containers
kubectl get pods <pod name> -o jsonpath='{range .spec.containers[*]}{.name}{"\n"}{end}'. // get container name(s) from the pod name
kubectl exec -it <pod name> -c <container name> bash. // access pod

API Use in ONAP SO

Create and Instantiate AS

<Preconditions>:
- Registar K8S Cluster(s) to ONAP
- During the registration, SO CNFM creates .kube/{target K8S cluster name}.config file(s)

<SO BPMN Infra>
- BPMN Infra workflow gets VF Resource TOSCA.meta.
- In there, the entity_defintion_type is found and it is set to "asd", BPMN infra launches the AS LCM Workflow as the above.
- The workflows sends a request for Create AS Identifier to SO CNFM

<Create an AS Identifier>

SO CNFM receives the create request (CreateAsRequest) and processes the request and return a response with AsInstance

CreateAsRequest:

asdId	Identifier (UUID)	1	Identifier that identifies the ASD which defines the AS instance to be created.
asInstanceDescription	String	0..1	Human-readable description of the AS instance to be created.
asInstanceName	String	0..1	Human-readable name of the AS instance to be created.

The client passes "asdId" to SO CNFM, so CNFM can query an ASD with the asdId from the ASD Repository
SO CNFM copies the incoming CreateAsRequest attributes into an AsInstance structure, and persists the AsInstance, which has a connection between the asInstanceId and asdId
note: SO CNFM will uses the asdId when it queries an ASD from the ASD Repository during the instantiation operation

When the Create AS instance is successful, SO CNFM returns 200 OK with the AsInstance
- The returned AsInstance contains the asInstanceId
- Subsequent operations uses the asInstanceId; i.e., the Client use the asIstanceId in the REST Api path

<Instantiate an AS>

BPMN Infra sends an AS instantiate request to SO CNFM with the asInstanceId as follows:

POST .../as_instances/{asInstanceId}/instantiate (InstantiateAsRequest)

InstantiateAsRequest

Attribute Name	Data Type	Cardinality	Description
asdExtCpdInputParams	ExtCpdParams	0..N	contains ext cpd parameter instance-level value
deploymentItems	DeploymentItems	1..N	contains lifecycle parameters for deploymentItems
additionalParams	KeyValuesPairs	0..1	Additional input parameters for the instantiation process (this is a pace holder to hold any additional parameters which are needed by SO CNFM)

SO CNFM retrieves the corresponding AsInstance from its DB; it is retrieve the "asdId" for the ASD query
SO CNFM queries an ASD with the asdId from the ASD Repository; caches the retrieved ASD in the memory during the Instantiate operations
SO CNFM reads thru the ASD DeploymentItems, and per deploymentItems, SO CNFM queries for the associated Helm Chart (1:1) from the Helm Chart Repository
- caches the retrieved Helm Charts in the memory during the Instantiate operations
SO CNFM reads the deploymentItems.deploymentOrder. Based on the order sequence, SO CNFM processes the deploymentItems one by one
For each, deployment item,
- SO CNFm creates vf-modules in AAI
- SO CNFM assignes vf-modules in SDNC
- From the InstantiateAsRequest, SO CNFM retrieves the deploymentItems
  - DeploymentItems
    deploymentItemId Identifier 1 Identifies which deploymentItem
    lifecycleParameterKeyValues
    KeyValuesPairs 0..N provides lifecycle parameter keys and values
  - The lifecycleParameterKeyValues contains a list of customizable attributes (key) in the values.yaml with instance-level values
- From the associated Helm Chart, SO CNFM gets the values.yaml
- SO CNFM creates a new values.yaml, based on the retrieved values.yaml + lifecycleParameterKeyValues
- ==== SO CNFM processes the asdExtCpdInputParam ==== TBD
- SO CNFM performs "helm template " to render K8S resource template(s)
- With the rendered k8S resource template(s), SO CNFM gets a placement decision from the Placement component (e.g., OOF)
  - Currently, use of OOF is out of the scope from the initial PoC
  - In the initial PoC, a simplified placement function will be used
- Based on the placement decision, SO CNFM determines the target K8S cluster
- Set the Helm command environment to connect to the target K8S cluster
  - set .kube/{target K8S cluster name}.config
- SO CNFM invokes "helm install" command with the corresponding Helm Chart and a new values.yaml
  - SO CNFM will have a few South-Bound plugin (helm client, CNF Adapter, others)
  - in the initial PoC, the helm client will be used
  - SO CNFM Helm Client will select a target Kubernetes cluster
    - e.g., helm install <release> <chart> --kubeconfig ~/.kubeconfigs/<cluster_name>.kubeconfig
    - in the initial PoC, the target cluster is selected by the user and its name will be passed thru the user params (in SO) and additionalParams (in SO-CNFM)
- If successful, SO CNFM update the corresponding vf-module in the AAI

<Postconditions>:
- each POD is a Ready status. This ensures that each POD can serve requests.
- Some of the PODs will not reach the Ready status until a day 1 configuration is applied
- The PODs still need verification of containers with a specific ContainerState

AS LCM APIs

Create an AS Identifier

In order for an application (CNF) to be eligible for instantiation, a CNF identifier must be created to identify the application service as a unique instance.

An AS instance supports multiple deployment items (helm charts). So, the AS instance id could be used for helm release names as the prefix.

Query AS Instance Identifiers

To list all AS instances, make a GET request to SO CNFM.

GET .../aslcm/v1/as_instances

Query an AS instance by Instance ID

To query a specific AS instance by its instance ID, make a GET request to SO CNFM.

GET .../aslcm/v1/as_instances/{asInstanceId}

Instantiate an AS

To instantiate an AS, make a poST request to SO CNFM. The target cluster name is included in the requrest parameter.

POST .../as_instances/{asInstanceId}/instantiate

if some of the parameters required for instantiation are complex and cannot fit into the additionalParams section of the JSON request body (TBD).

Upgrading an AS

This operation is referred to as the changing of an AS package.

POST .../as_instances/{asInstanceId}/change_aspkg

Terminate an AS Instance

This operation terminates an AS instance.

~~POST .../as_instances/{asInstnaceId}/terminate (TerminateAsRequest)~~

Note: for PoC, GRACEFUL termination type is not supported.

Delete an AS Identifier

This operation deletes an AS Identifier.

DELETE .../as_instances/{as_instanceId}

Note: optionally, Delete an AS Identifier could clean up resources of an AS Instance, e.g., Persistent Volumes (PVs)

The Delete operation may need to update AAI.