Modeling Concepts

Migrated to ReadTheDocs

Further updates must be done in the corresponding RST file(s) by following the build process for documentation.

https://docs.onap.org/projects/onap-ccsdk-cds/en/latest/modelingconcepts/index.html

2020-08-11, Jakob Krieg


CDS is a framework to automate the resolution of resources for instantiation and any config provisioning operation, such as day0, day1 or day2 configuration.

CDS has a both design time and run time activities; during design time, Designer can define what actions are required for a given service, along with anything comprising the action. The design produce a CBA Package . Its content is driven from a catalog of reusable data dictionary and component, delivering a reusable and simplified self service experience.

CDS modelling is mainly based on TOSCA standard, using JSON as reprensentation.

Most of the TOSCA modeled entity presented in the bellow documentation can be found here.



 CBA

Controller Blueprint Archive (.cba)

The Controller Blueprint Archive is the overall service design, fully model-driven, intent based package needed for provisioning and configuration management automation.

The CBA is .zip file, comprised of the following folder structure, the files may vary:

├── Definitions
│   ├── blueprint.json							Overall TOSCA service template (workflow + node_template)
│   ├── artifact_types.json 					(generated by enrichment)
│   ├── data_types.json 						(generated by enrichment)
│   ├── policy_types.json 						(generated by enrichment)
│   ├── node_types.json 						(generated by enrichment)
│   ├── relationship_types.json 				(generated by enrichment)
│   ├── resources_definition_types.json 		(generated by enrichment)
|   └── *-mapping.json							One per Template
├── Environments								Contains *.properties files as required by the service
├── Plans										Contains Directed Graph
├── Tests										Contains uat.yaml file for testing cba actions within a cba package 
├── Scripts										Contains scripts
│   ├── python					 				Python scripts
│   └── kotlin									Kotlin scripts
├── TOSCA-Metadata
│   └── TOSCA.meta								Meta-data of overall package
└── Templates									Contains combination of mapping and template

To process a CBA for any service we need to enrich it first. This will gather all the node-type, data-type, artifact-type, data-dictionary definitions provided in the blueprint.json.

 Tosca.Meta

Tosca Meta

Tosca meta file is captures the model entities that compose the cba package name, version, type and searchable tags. 


AttributeR/C/OData TypeDescription
TOSCA-Meta-File-VersionRequiredStringThe attribute that holds TOSCA-Meta-File-Version.   Set to 1.0.0
CSAR-VersionRequiredString

The attribute that holds CSAR-version.   Set to 1.0

Created-ByRequiredStringThe attribute that holds the entry points
Entry-DefinitionsRequiredString

The attribute that holds the entry points file PATH to the main cba tosca definition file or non tosca script file. 

Template-NameRequiredStringThe attribute that holds the blueprint name
Template-VersionRequiredString

The attribute that holds the blueprint version 

X.Y.Z

X=Major version

Y=Minor Version

Z=Revision Version 

Ex. 1.0.0

Template-TypeRequiredString

The attribute that holds the blueprint package types. 

Valid Options: 

  • "DEFAULT"  –   .JSON file consistent of tosca based cba package that describes the package intent.
  • "KOTLIN_DSL" – .KT file consistent of tosca based cba package that describes the package intent composed using Domain Specific Language (DSL). 
  • "GENERIC_SCRIPT" – Script file consistent of NONE tosca based cba package that describes the package intent using DSL Language. 

If not specified in the tosca.meta file the default is "DEFAULT"

Template-TagsRequiredStringThe attribute that holds the blueprint package comma delimited list of Searchable attributes. 

Template Type Reference

Default Template Type

https://git.onap.org/ccsdk/cds/tree/components/model-catalog/blueprint-model/test-blueprint/capability_cli/TOSCA-Metadata/TOSCA.meta

KOTLIN_DSL Template Type

https://git.onap.org/ccsdk/cds/tree/components/model-catalog/blueprint-model/test-blueprint/resource-audit/TOSCA-Metadata/TOSCA.meta

GENERIC_SCRIPT Template Type

https://git.onap.org/ccsdk/cds/tree/components/model-catalog/blueprint-model/test-blueprint/capability_python/TOSCA-Metadata/TOSCA.meta

 Dynamic Payload

Dynamic payload

One of the most important API provided by the run time is to execute a CBA Package.

The nature of this API request and response is model driven and dynamic.

Here is how the a generic request and response look like.

request
{
  "commonHeader": {
    "originatorId": "",
    "requestId": "",
    "subRequestId": ""
  },
  "actionIdentifiers": {
    "blueprintName": "",
    "blueprintVersion": "",
    "actionName": "",
    "mode": ""
  },
  "payload": {
    "$actionName-request": {
      "$actionName-properties": {
      }
    }
  }
}
response
{
  "commonHeader": {
    "originatorId": "",
    "requestId": "",
    "subRequestId": ""
  },
  "actionIdentifiers": {
    "blueprintName": "",
    "blueprintVersion": "",
    "actionName": "",
    "mode": ""
  },
  "payload": {
    "$actionName-response": {
    }
  }
}

The actionName, under the actionIdentifiers refers to the name of a Workflow (see Modeling Concepts#workflow)

The content of the payload is what is fully dynamic / model driven.

The first top level element will always be either $actionName-request for a request or $actionName-response for a response.

Then the content within this element is fully based on the workflow inputs and outputs.

During the Enrichment, CDS will aggregate all the resources defined to be resolved as input, within mapping definition files, as data-type, that will then be use as type of an input called $actionName-properties.

 Enrichment

Enrichment

The idea is that the CBA is a self-sufficient package, hence requires all the various types definition its using.

Reason for this is the types its using might evolve. In order for the CBA to be bounded to the version it has been using when it has been designed, these types are embedded in the CBA, so if they change, the CBA is not affected.

The enrichment process will complete the package by providing all the definition of types used:

  • gather all the node-type used and put them into a node_types.json file
  • gather all the data-type used and put them into a data_types.json file
  • gather all the artifact-type used and put them into a artifact_types.json file
  • gather all the data dictionary definitions used from within the mapping files and put them into a resources_definition_types.json file


Before uploading a CBA, it must be enriched. If your package is already enrich, you do not need to perform enrichment again.


The enrichment can be run using REST API, and required the .zip file as input. It will return an enriched-cba.zip file.

curl -X POST \
  'http://{{ip}}:{{cds-designtime}}/api/v1/blueprint-model/enrich' \
  -H 'content-type: multipart/form-data' \
  -F file=@cba.zip

The enrichment process will also, for all resources to be resolved as input and default:

  • dynamically gather them under a data-type, named dt-${actionName}-properties
  • will add it as a input of the workflow, as follow using this name: ${actionName}-properties

Example for workflow named resource-assignment:

dynamic input
  "resource-assignment-properties": {
    "required": true,
    "type": "dt-resource-assignment-properties"
  }
 Flexible Plug-In

External Systems support

Interaction with external systems is made dynamic, removing development cycle to support new endpoint.

In order to define the external system information, TOSCA provides dsl_definitions. Link to TOSCA spec info 1, info 2.

Use cases:


Here are some examples on how to populate the system information within the package:

token-auth
{
  . . .
  "dsl_definitions": {
    "ipam-1": {
      "type": "token-auth",
      "url": "http://netbox-nginx.netprog:8080",
      "token": "Token 0123456789abcdef0123456789abcdef01234567"
    }
  }
  . . .
}
basic-auth
{
  . . .
  "dsl_definitions": {
    "ipam-1": {
      "type": "basic-auth",
      "url": "http://localhost:8080",
      "username": "bob",
      "password": "marley"
    }
  }
  . . .
}
ssl-basic-auth
{
  . . .
  "dsl_definitions": {
    "ipam-1": {
      "type" : "ssl-basic-auth",
      "url" : "http://localhost:32778",
      "keyStoreInstance": "JKS or PKCS12",
      "sslTrust": "trusture",
      "sslTrustPassword": "trustore password",
      "sslKey": "keystore",
      "sslKeyPassword: "keystore password"
    }
  }
  . . .
}
grpc-executor
{
  . . .
  "dsl_definitions": {
    "remote-executor": {
      "type": "token-auth",
      "host": "cds-command-executor.netprog",
      "port": "50051",
      "token": "Basic Y2NzZGthcHBzOmNjc2RrYXBwcw=="
    }
  }
  . . .
}
maria-db
{
  . . .
  "dsl_definitions": {
    "netprog-db": {
      "type": "maria-db",
      "url": "jdbc:mysql://10.195.196.123:32050/netprog",
      "username": "netprog",
      "password": "netprog"
    }
  }
  . . .
}
 Expression

Expression

TOSCA provides for a set of functions to reference elements within the template or to retrieve runtime values.


Below is a list of supported expressions


 get_input

get_input

The get_input function is used to retrieve the values of properties declared within the inputs section of a TOSCA Service Template.

Within CDS, this is mainly Workflow inputs.

TOSCA specification

Example:

https://github.com/onap/ccsdk-cds/blob/master/components/model-catalog/blueprint-model/test-blueprint/golden/Definitions/golden-blueprint.json#L210

"resolution-key": {
  "get_input": "resolution-key"
}
 get_property

get_property

The get_property function is used to retrieve property values between modelable entities defined in the same service template.

TOSCA specification

Example

TBD

 get_attribute

get_attribute

The get_attribute function is used to retrieve the values of named attributes declared by the referenced node or relationship template name.

TOSCA specification

Example:

https://github.com/onap/ccsdk-cds/blob/master/components/model-catalog/blueprint-model/test-blueprint/golden/Definitions/golden-blueprint.json#L64-L67

"get_attribute": [
  "resource-assignment",
  "assignment-params"
]
 get_operation_output

get_operation_output

The get_operation_output function is used to retrieve the values of variables exposed / exported from an interface operation.

TOSCA specification

Example

TBD

 get_artifact

get_artifact

The get_artifact function is used to retrieve artifact location between modelable entities defined in the same service template.

TOSCA specification

Example

TBD

 Data dictionary

Data Dictionary

A data dictionary models the how a specific resource can be resolved.

A resource is a variable/parameter in the context of the service. It can be anything, but it should not be confused with SDC or Openstack resources.

A data dictionary can have multiple sources to handle resolution in different ways.

The main goal of data dictionary is to define re-usable entity that could be shared.

Creation of data dictionaries is a standalone activity, separated from the blueprint design.

As part of modelling a data dictionary entry, the following generic information should be provided:

PropertyDescriptionScope
nameData dictionary nameMandatory
tagsInformation relatedMandatory
updated-byThe creatorMandatory
propertyDefines type and description, as nested JSONMandatory
sourcesList of resource source instance (see resource source)Mandatory

Bellow are properties that all the resource source can have

The modeling does allow for data translation between external capability and CDS for both input and output key mapping.

PropertyDescriptionScope
input-key-mappingmap of resources required to perform the request/query. The left hand-side is what is used within the query/request, the right hand side refer to a data dictionary instance.Optional
output-key-mappingname of the resource to be resolved mapped to the value resolved by the request/query.Optional
key-dependencies

list of data dictionary instances to be resolved prior the resolution of this specific resource.

during run time execution the key dependencies are recursively sorted and resolved in batch processing using the acyclic graph algorithm.

Optional

Example:
vf-module-model-customization-uuid and vf-module-label are two data dictionaries. A SQL table, VF_MODULE_MODEL, exist to correlate them.

Here is how input-key-mapping, output-key-mapping and key-dependencies can be used:

vf-module-label data dictionary
{
  "name" : "vf-module-label",
  "tags" : "vf-module-label",
  "updated-by" : "adetalhouet",
  "property" : {
    "description" : "vf-module-label",
    "type" : "string"
  },
  "sources" : {
    "primary-db" : {
      "type" : "source-primary-db",
      "properties" : {
        "type" : "SQL",
        "query" : "select sdnctl.VF_MODULE_MODEL.vf_module_label as vf_module_label from sdnctl.VF_MODULE_MODEL where sdnctl.VF_MODULE_MODEL.customization_uuid=:customizationid",
        "input-key-mapping" : {
          "customizationid" : "vf-module-model-customization-uuid"
        },
        "output-key-mapping" : {
          "vf-module-label" : "vf_module_label"
        },
        "key-dependencies" : [ "vf-module-model-customization-uuid" ]
      }
    }
  }
}
 Data Type

Data type

Represents the schema of a specific type of data.

Supports both primitive and complex data types:

PrimitiveComplex
  • string
  • integer
  • float
  • double
  • boolean
  • timestamp
  • null
  • json
  • list
  • array

For complex data type, an entry schema is required, defining the type of value contained within the complex type, if list or array.

Users can create as many data type as needed.

Creating Custom Data Types

To create a custom data-type you can use a POST call to CDS endpoint: "<cds-ip>:<cds-port>/api/v1/model-type"
Payload:
{

  "model-name": "<model-name>",
  "derivedFrom": "tosca.datatypes.Root",

  "definitionType": "data_type",

  "definition": {

    "description": "<description>",

    "version": "<version-number: eg 1.0.0>",

    "properties": {<add properties of your custom data type in JSON format>},

    "derived_from": "tosca.datatypes.Root"

  },

  "description": "<description",

  "version": "<version>",

  "tags": "<model-name>,datatypes.Root.data_type",

  "creationDate": "<creation timestamp>",

  "updatedBy": "<name>"

}


Data type are useful to manipulate data during resource resolution. They can be used to format the JSON output as needed.

List of existing data type: https://github.com/onap/ccsdk-cds/tree/master/components/model-catalog/definition-type/starter-type/data_type

TOSCA specification


Below is a list of existing data types

 resources-assignment

datatype-resource-assignment

Used to define entries within Modeling Concepts#artifact-mapping-resource

That datatype represent a resource to be resolved. We also refer this as an instance of a data dictionary as it's directly linked to its definition.

PropertyDescription
propertyDefines how the resource looks like (see Modeling Concepts#datatype-property)
input-paramWhether the resource can be provided as input.
dictionary-nameReference to the name of the data dictionary (see Data Dictionary).
dictionary-sourceReference the source to use to resolve the resource (see Resource source).
dependenciesList of dependencies required to resolve this resource.
updated-dateDate when mapping was upload.
updated-byName of the person that updated the mapping.

https://github.com/onap/ccsdk-cds/blob/master/components/model-catalog/definition-type/starter-type/data_type/datatype-resource-assignment.json

datatype-resource-assignment
{
	"version": "1.0.0",
	"description": "This is Resource Assignment Data Type",
	"properties": {
		"property": {
			"required": true,
			"type": "datatype-property"
		},
		"input-param": {
			"required": true,
			"type": "boolean"
		},
		"dictionary-name": {
			"required": false,
			"type": "string"
		},
		"dictionary-source": {
			"required": false,
			"type": "string"
		},
		"dependencies": {
			"required": true,
			"type": "list",
			"entry_schema": {
				"type": "string"
			}
		},
		"updated-date": {
			"required": false,
			"type": "string"
		},
		"updated-by": {
			"required": false,
			"type": "string"
		}
	},
	"derived_from": "tosca.datatypes.Root"
}
 property

datatype-property

Used to defined the property entry of a resource assignment.

PropertyDescription
typeWhether it's a primitive type, or a defined data-type
descriptionDescription of for the property
requiredWhether it's required or not
defaultIf there is a default value to provide
entry_schemaIf the type is a complex one, such as list, define what is the type of element within the list.

https://github.com/onap/ccsdk-cds/blob/master/components/model-catalog/definition-type/starter-type/data_type/datatype-property.json

datatype-property
{
  "version": "1.0.0",
  "description": "This is Entry point Input Data Type, which is dynamic datatype, The parameter names will be populated during the Design time for each inputs",
  "properties": {
    "type": {
      "required": true,
      "type": "string"
    },
    "description": {
      "required": false,
      "type": "string"
    },
    "required": {
      "required": false,
      "type": "boolean"
    },
    "default": {
      "required": false,
      "type": "string"
    },
    "entry_schema": {
      "required": false,
      "type": "string"
    }
  },
  "derived_from": "tosca.datatypes.Root"
}


 Artifact Type

Artifact Type

Represents the type of a artifact, used to identify the implementation of the functionality supporting this type of artifact.

TOSCA definition

This node was created, derived from tosca.artifacts.Root to be the root TOSCA node for all artifact.

tosca.artifacts.Implementation
{
  "description": "TOSCA base type for implementation artifacts",
  "version": "1.0.0",
  "derived_from": "tosca.artifacts.Root"
}


Bellow is a list of supported artifact types

 Velocity

artifact-template-velocity

Represents an Apache Velocity template.

Apache Velocity allow to insert logic (if / else / loops / etc) when processing the output of a template/text.

File must have .vtl extension.

The template can represent anything, such as device config, payload to interact with 3rd party systems, resource-accumulator template, etc...

Often a template will be parameterized, and each parameter must be defined within an mapping file.

Velocity reference document

Here is the TOSCA artifact type:

artifact-template-velocity
{
  "description": " Velocity Template used for Configuration",
  "version": "1.0.0",
  "file_ext": [
    "vtl"
  ],
  "derived_from": "tosca.artifacts.Implementation"
}
 Jinja

artifact-template-jinja

Represents an Jinja template.

Jinja template allow to insert logic (if / else / loops / etc) when processing the output of a template/text.

File must have .jinja extension.

The template can represent anything, such as device config, payload to interact with 3rd party systems, resource-accumulator template, etc...

Often a template will be parameterized, and each parameter must be defined within an mapping file.

Jinja reference document

Here is the TOSCA artifact type:

artifact-template-jinja
{
  "description": " Jinja Template used for Configuration",
  "version": "1.0.0",
  "file_ext": [
    "jinja"
  ],
  "derived_from": "tosca.artifacts.Implementation"
}
 Mapping

artifact-mapping-resource

This type is meant to represent mapping files defining the contract of each resource to be resolved.

Each parameter in a template must have a corresponding mapping definition, modeled using Modeling Concepts#datatype-resource-assignment.

Hence the mapping file is meant to be a list of entries defined using Modeling Concepts#datatype-resource-assignment.

File must have .json extension.

Here is the TOSCA artifact type:

artifact-mapping-resource
{
  "description": "Resource Mapping File used along with Configuration template",
  "version": "1.0.0",
  "file_ext": [
    "json"
  ],
  "derived_from": "tosca.artifacts.Implementation"
}

The mapping file basically contains a reference to the data dictionary to use to resolve a particular resource.

The data dictionary defines the HOW and the mapping defines the WHAT.

Relation between data dictionary, mapping and template.

Below are two examples using color coding to help understand the relationships.

In orange is the information regarding the template. As mentioned before, template is part of the blueprint itself, and for the blueprint to know what template to use, the name has to match.

In green is the relationship between the value resolved within the template, and how it's mapped coming from the blueprint.

In blue is the relationship between a resource mapping to a data dictionary.

In red is the relationship between the resource name to be resolved and the HEAT environment variables.

The key takeaway here is that whatever the value is for each color, it has to match all across. This means both right and left hand side are equivalent; it's all on the designer to express the modeling for the service. That said, best practice is example 1.

 Directed Graph

artifact-directed-graph

Represents a directed graph.

This is to represent a workflow.

File must have .xml extension.

Here is the list of executors currently supported (see here for explanation and full potential list: Service Logic Interpreter Nodes)

  • execute
  • block
  • return
  • break
  • exit

Here is the TOSCA artifact type:

artifact-directed-graph
{
  "description": "Directed Graph File",
  "version": "1.0.0",
  "file_ext": [
    "json",
    "xml"
  ],
  "derived_from": "tosca.artifacts.Implementation"
}
<