DCAE MOD User Guide

Table of Contents

Types of Users and Usage Instructions:

...

Using DCAE MOD without an Ingress Controller

...

Types of Users and Usage Instructions:

Short Video Series available at : https://www.youtube.com/playlist?list=PLj-oRfbkqkfnN_2vnfhivCesJ118SA_zG

Demo day demonstration recording and slides available at : https://lf-onap.atlassian.net/wiki/display/DW/2020-04-16+DCAE+Demo

Sr.No	User	Usage Instructions
1.	Developers who are looking to onboard their mS	· Access the Nifi Web UI url provided to you · Follow steps 2.b to 2.d · You should be able to see your microservices in the Nifi Web UI by clicking and dragging ‘Processor’ on the canvas, and searching for the name of the microservice/component/processor.
2.	Designers who are building the flows through UI and triggering distribution	· Access the Nifi Web UI url provided to you · Follow steps 3 to the end of the document
3.	Infrastructure/ Admins who want to stand up DCAE Mod and validate it	· Follow start to the end

1. Deployment of DCAE MOD components via Helm charts

The DCAE MOD components are deployed using the standard ONAP OOM deployment process. When deploying ONAP using the helm deploy command, DCAE MOD components are deployed when the dcaemod.enabled flag is set to true, either via a --set option on the command line or by an entry in an overrides file. In this respect, DCAE MOD is no different from any other ONAP subsystem.

The default DCAE MOD deployment relies on an nginx ingress controller being available in the Kubernetes cluster where DCAE MOD is being deployed. The Rancher RKE installation process sets up a suitable ingress controller. In order to enable the use of the ingress controller, it is necessary to override the OOM default global settings for ingress configuration. Specifically, the installation needs to set the following configuration in an override file:

...

Code Block

language	yml
theme	RDark
title	Global Ingress configuration

  #Global ingress configuration
  ingress:

...

   enabled: true

...

   virtualhost:

...

baseurl:

...

"simpledemo.onap.org"

When DCAE MOD is deployed with an ingress controller, several endpoints are exposed outside the cluster at the ingress controller's external IP address and port. (In the case of a Rancher RKE installation, there is an ingress controller on every worker node, listening at the the standard HTTP port (80).) These exposed endpoints are needed by users using machines outside the Kubernetes cluster.

...

Configuration Item	Endpoint URL
Registry client	http://dcaemod-nifi-registry:18080
Distribution target	http://dcaemod-runtime-api:9090

Using DCAE MOD without an Ingress Controller

...

As OOM/ingress template has been updated in Guilin release to enable virtual host, MOD API's and UI access via ingress should use dcaemod.api.simpledemo.onap.org

Add entry for dcaemod.simpledemo.onap.org in /etc/hosts with the correct IP (any of K8S node IP can be specified)

Using DCAE MOD without an Ingress Controller

Not currently supported.

2. Configuring DCAE mod

Our demo is hosted on 10.12.7.116. The IP Address for the purpose of this demo will hence be 10.12.7.116. In case of other deployments, we would have used the IP Address, or the DNS FQDN, if there is one, for one of the Kubernetes nodes.

Now let’s access the Nifi (DCAE designer) UI - http://10dcaemod.12simpledemo.7onap.116org/nifi/

a) Get the artifacts to test and onboard.

Let's fetch the artifacts/ spec files

A sample Component DCAE-VES-Collector : https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json

A sample Data Format : https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/data-formats/VES-5.28.4-dataformat.json

For the purpose of onboarding, a Sample Request body should be of the type -

{ "owner": "<some value>", "spec": <some json object> }

where the json object inside the spec field can be a component spec json.

You would have to use a request body Request bodies of this type will be used in the onboarding requests you make using curl or the onboarding swagger interface.

The prepared Sample Request body for a component dcae-ves-collector looks like so

...

Expand

Elite soft json viewer

width	1000
height	1000

{

  "spec": {

    "self": {

      "version": "1.5.4",

      "name": "dcae-ves-collector",

      "description": "Collector for receiving VES events through restful interface",

      "component_type": "docker"

    },

    "streams": {

      "subscribes": [

       

      ],

      "publishes": [

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-fault"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-measurement"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-syslog"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-heartbeat"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-other"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-mobileflow"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-statechange"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-thresholdCrossingAlert"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-voicequality"

        },

        {

          "format": "VES_specification",

          "version": "5.28.4",

          "type": "message router",

          "config_key": "ves-sipsignaling"

        },

        {

          "format": "VES_specification",

          "version": "7.30.1",

          "type": "message router",

          "config_key": "ves-pnfRegistration"

        },

        {

          "format": "VES_specification",

          "version": "7.30.1",

          "type": "message router",

          "config_key": "ves-notification"

        },

        {

          "format": "VES_specification",

          "version": "7.30.1",

          "type": "message router",

          "config_key": "ves-perf3gpp"

        }

      ]

    },

    "services": {

      "calls": [

       

      ],

      "provides": [

        {

          "route": "/eventListener/v1",

          "verb": "POST",

          "request": {

            "format": "VES_specification",

            "version": "4.27.2"

          },

          "response": {

            "format": "ves.coll.response",

            "version": "1.0.0"

          }

        },

        {

          "route": "/eventListener/v2",

          "verb": "POST",

          "request": {

            "format": "VES_specification",

            "version": "4.27.2"

          },

          "response": {

            "format": "ves.coll.response",

            "version": "1.0.0"

          }

        },

        {

          "route": "/eventListener/v3",

          "verb": "POST",

          "request": {

            "format": "VES_specification",

            "version": "4.27.2"

          },

          "response": {

            "format": "ves.coll.response",

            "version": "1.0.0"

          }

        },

        {

          "route": "/eventListener/v4",

          "verb": "POST",

          "request": {

            "format": "VES_specification",

            "version": "4.27.2"

          },

          "response": {

            "format": "ves.coll.response",

            "version": "1.0.0"

          }

        },

        {

          "route": "/eventListener/v5",

          "verb": "POST",

          "request": {

            "format": "VES_specification",

            "version": "5.28.4"

          },

          "response": {

            "format": "ves.coll.response",

            "version": "1.0.0"

          }

        },

        {

          "route": "/eventListener/v7",

          "verb": "POST",

          "request": {

            "format": "VES_specification",

            "version": "7.30.1"

          },

          "response": {

            "format": "ves.coll.response",

            "version": "1.0.0"

          }

        }

      ]

    },

    "parameters": [

      {

        "name": "collector.service.port",

        "value": 8080,

        "description": "standard http port collector will open for listening;",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "collector.service.secure.port",

        "value": 8443,

        "description": "secure http port collector will open for listening ",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": true

      },

      {

        "name": "collector.keystore.file.location",

        "value": "/opt/app/dcae-certificate/cert.jks",

        "description": "fs location of keystore file in vm",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "collector.keystore.passwordfile",

        "value": "/opt/app/dcae-certificate/jks.pass",

        "description": "location of keystore password file in vm",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "collector.truststore.file.location",

        "value": "/opt/app/dcae-certificate/trust.jks",

        "description": "fs location of truststore file in vm",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "collector.truststore.passwordfile",

        "value": "/opt/app/dcae-certificate/trust.pass",

        "description": "location of truststore password file in vm",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "collector.dmaap.streamid",

        "value": "fault=ves-fault|syslog=ves-syslog|heartbeat=ves-heartbeat|measurementsForVfScaling=ves-measurement|measurement=ves-measurement|mobileFlow=ves-mobileflow|other=ves-other|stateChange=ves-statechange|thresholdCrossingAlert=ves-thresholdCrossingAlert|voiceQuality=ves-voicequality|sipSignaling=ves-sipsignaling|notification=ves-notification|pnfRegistration=ves-pnfRegistration|perf3gpp=ves-perf3gpp",

        "description": "domain-to-streamid mapping used by VESCollector to distributes events based on domain. Both primary and secondary config_key are included for resilency (multiple streamid can be included commma separated). The streamids MUST match to topic config_keys. For single site without resiliency deployment - configkeys with -secondary suffix can be removed",

        "sourced_at_deployment": true,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "auth.method",

        "value": "noAuth",

        "description": "Property to manage application mode, possible configurations: noAuth - default option - no security (http) , certOnly - auth by certificate (https), basicAuth - auth by basic auth username and password (https),certBasicAuth - auth by certificate and basic auth username / password (https),",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "header.authlist",

        "value": "sample1,$2a$10$pgjaxDzSuc6XVFEeqvxQ5u90DKJnM/u7TJTcinAlFJVaavXMWf/Zi|userid1,$2a$10$61gNubgJJl9lh3nvQvY9X.x4e5ETWJJ7ao7ZhJEvmfJigov26Z6uq|userid2,$2a$10$G52y/3uhuhWAMy.bx9Se8uzWinmbJa.dlm1LW6bYPdPkkywLDPLiy",

        "description": "List of id and base 64 encoded password.For each onboarding VNF - unique userid and password should be assigned and communicated to VNF owner. Password value should be base64 encoded in config here",

        "policy_editable": false,

        "sourced_at_deployment": true,

        "designer_editable": true

      },

      {

        "name": "collector.schema.checkflag",

        "value": 1,

        "description": "Schema check validation flag. When enabled, collector will validate input VES events against VES Schema defined on collector.schema.file ",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "collector.schema.file",

        "value": "{\"v1\":\"./etc/CommonEventFormat_27.2.json\",\"v2\":\"./etc/CommonEventFormat_27.2.json\",\"v3\":\"./etc/CommonEventFormat_27.2.json\",\"v4\":\"./etc/CommonEventFormat_27.2.json\",\"v5\":\"./etc/CommonEventFormat_28.4.1.json\",\"v7\":\"./etc/CommonEventFormat_30.1.1.json\"}",

        "description": "VES schema file name per version used for validation",

        "designer_editable": true,

        "sourced_at_deployment": false,

        "policy_editable": false

      },

      {

        "name": "event.transform.flag",

        "value": 1,

        "description": "flag to enable tranformation rules defined under eventTransform.json; this is applicable when event tranformation rules preset should be activated for transforming <VES5.4 events to 5.4",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      },

      {

        "name": "tomcat.maxthreads",

        "value": "200",

        "description": "Tomcat control for concurrent request",

        "sourced_at_deployment": false,

        "policy_editable": false,

        "designer_editable": false

      }

    ],

    "auxilary": {

      "healthcheck": {

        "type": "http",

        "interval": "15s",

        "timeout": "1s",

        "endpoint": "/healthcheck"

      },

      "volumes": [

        {

          "container": {

            "bind": "/opt/app/dcae-certificate"

          },

          "host": {

            "path": "/opt/app/dcae-certificate"

          }

        },

        {

          "container": {

            "bind": "/opt/app/VESCollector/logs"

          },

          "host": {

            "path": "/opt/logs/DCAE/VESCollector/logs"

          }

        },

        {

          "container": {

            "bind": "/opt/app/VESCollector/etc"

          },

          "host": {

            "path": "/opt/logs/DCAE/VESCollector/etc"

          }

        }

      ],

      "ports": [

        "8080:0",

        "8443:0"

      ],

      "log_info": {

        "log_directory": "/opt/app/VESCollector/logs/"

      },

      "tls_info": {

        "cert_directory": "/opt/app/dcae-certificate/",

        "use_tls": true

      }

    },

    "artifacts": [

      {

        "type": "docker image",

        "uri": "nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:latest"

      }

    ]

  },

  "owner": "onboarding_dev"

}

A The prepared Sample request body for a sample data format looks like so -

...

These requests would be of the type-

curl -X POST -u <user>:<password> http://<onboardingapi host>/onboarding/dataformats     -H "Content-Type: application/json" -d @<filepath to request>
curl -X POST -u <user>:<password> http://<onboardingapi host>/onboarding/components     -H "Content-Type: application/json" -d @<filepath to request>   Inrequest>   

In our case, curl 
curl -X POST -u acumos:integration2019 http://10dcaemod.12simpledemo.7onap.116org/onboarding/dataformats     -H "Content-Type: application/json" -d @<filepath to request>
curl -X POST -u acumos:integration2019  http:http://10dcaemod.12simpledemo.7onap.116org/onboarding/components     -H "Content-Type: application/json" -d @<filepath to request>

You can download the Components and Data Formats for the demo from –

Components:

https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json

https://git.onap.org/dcaegen2/analytics/tca-gen2/tree/dcae-analytics/dpo/tcagen2_spec.json

Corresponding Data Formats:

https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/data-formats
https://git.onap.org/dcaegen2/analytics/tca-gen2/tree/dcae-analytics/dpo/

...

c) Verify the resources were created using

curl -X GET -u acumos:integration2019 http://10dcaemod.12simpledemo.7onap.116org/onboarding/dataformats

curl -X GET -u acumos:integration2019 http://10dcaemod.12simpledemo.7onap.116org/onboarding/components

d) Verify the genprocessor (which polls onboarding periodically to convert component specs to nifi processor), converted the component

Open http://10dcaemod.12simpledemo.7onap.116org/nifi-jars in / in a browser.

These jars should now be available for you to use in the nifi UI as processors.

3. Design & Distribution Flow

To start creating flows, we need to create a process group first. The name of the process group will be the name of the flow. Drag and Drop on the canvas, the ‘Processor Group’ icon from the DCAE Designer bar on the top.

Image Removed

a) Configure Nifi Registry url

...

Add a registry client. The Registry client url will be http://dcaemod-nifi-registry:18080

b) Add distribution target which will be the runtime api url

Set the distribution target in the controller settings

Image Added

Distribution target URL will be http://dcaemod-runtime-api:9090

c) To start creating flows, we need to create a process group first. The name of the process group will be the name of the flow. Drag and Drop on the canvas, the ‘Processor Group’ icon from the DCAE Designer bar on the top.

Image Added

Now enter the process group by double clicking it,

...

Note: Even if you move a component around on the canvas, and its position on the canvas changes, it is recognized as a change, and it will have to recommitted.

bd) Adding components and building the flow

...

The complete flow after joining our components looks like so

c) Add distribution target which will be the runtime api url

Once we have the desired flow checked in, we can go ahead and set the distribution target in the controller settings

Image Removed

Distribution target URL will be http://dcaemod-runtime-api:9090

d

e) Submit/ Distribute the flow:

...

The runtime is supposed to generate the blueprint out of the packaged design/flow and push it to the DCAE inventory and the DCAE Dasboard.

ef) Checking the components in the DCAE Dashboard

...

Finally, the generated Blueprint can be deployed.Image Removed.

Image Added

You can use/import the attached input configurations files to deploy. Drag and Drop these sample JSON files to fill in the configuration values.

NOTE 1: Increase memory limit to 512Mi

NOTE 2: Verify image URL

View file

name	tca-deploy.input.json
height	250

View file

name	ves-deploy.input.json
height	250

Image Added

Versions Compared

Old Version 28

New Version Current

Key

DCAE MOD User Guide

Types of Users and Usage Instructions:

Types of Users and Usage Instructions:

1. Deployment of DCAE MOD components via Helm charts

Using DCAE MOD without an Ingress Controller

Using DCAE MOD without an Ingress Controller

2. Configuring DCAE mod

3. Design & Distribution Flow

Page Comparison

Versions Compared

Old Version 28

New Version Current

Key

DCAE MOD User Guide

Types of Users and Usage Instructions:

Types of Users and Usage Instructions:

1. Deployment of DCAE MOD components via Helm charts

Using DCAE MOD without an Ingress Controller

Using DCAE MOD without an Ingress Controller

2. Configuring DCAE mod

3. Design & Distribution Flow