...
The below steps covers manual setup of DCAE VM’s and DCAE service components.
VESCollector
DCAE VES Collector can be configured on VM with ubuntu-16.04 image (m1.small should suffice if this is only service) and 20Gb cinder storage
...
sudo apt install docker.io
...
2) Pull the latest container from onap nexus
sudo docker login -u docker -p docker nexusnexus3.onap.org:10001sudo
sudo docker pull
...
nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:v1.1.
...
03) Start the VESCollector with below command
sudo docker run -d --name vescollector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST='<dmaap IP>' nexusnexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:v1.1.10
Note: Change the dmaaphost to required DMAAP ip. To change the dmaap information for a running container, stop the active container and rerun above command changing the dmaap IP.
4) Verification
- Check logs under container /opt/app/VESCollector/logs/collector.log for errors
- If no active feed, you can simulate an event into collector via curl
...
Measurement -http://<dmaaphost>:3904/events/unauthenticated.SEC_MEASUREMENT_OUTPUT
...
VM Init
To address windriver server in-stability, the below init.sh script was used to start the container on VM restart.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
#!/bin/sh sudo docker ps | grep "vescollector" if [ $? -ne 0 ]; then sudo docker login -u docker -p docker nexusnexus3.onap.org:10001 sudo docker pull nexusnexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1 sudo docker rm -f vescollector echo "Collector process not running - $(date)" >> /home/ubuntu/startuplog sudo docker run -d --name vescollector -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST='10.12.25.96' nexusnexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:v1.1.10 else echo "Collector process running - $(date)" >> /home/ubuntu/startuplog fi |
...
sudo update-rc.d init.sh start 2
...
ThresholdCrossingAnalysis (TCA/CDAP)
...
The platform deploys CDAP as cluster and instantiates TCA. For the manual setup, we will leverage the CDAP SDK docker container to deploy TCA instances. To setup TCA, choose VM with ubuntu-16.04 image, m1.medium size and 50gb cinder volumes.
1) Install docker
sudo apt-get update
...
sudo docker run -d --name cdap-sdk-2 -p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.22
4) Create Namespace on CDAP application
curl -X PUT http://localhost:11015/v3/namespaces/cdap_tca_hi_lo
5) Create TCA app preference file config file - "tca_app_config.json" under ~ubuntu as below
| Code Block | |||||
|---|---|---|---|---|---|
| |||||
{
"artifact": {
"publisherContentTypename" : "application/jsondcae-analytics-cdap-tca",
"publisherHostNameversion" : "102.120.25.961",
"publisherHostPortscope" : "3904user"
},
"publisherMaxBatchSizeconfig" : "1",
"publisherMaxRecoveryQueueSize" {
"appName": "100000dcae-tca",
"publisherPollingIntervalappDescription" : "20000DCAE Analytics Threshold Crossing Alert Application",
"publisherProtocoltcaVESMessageStatusTableName" : "httpTCAVESMessageStatusTable",
"publisherTopicNametcaVESMessageStatusTableTTLSeconds" : "unauthenticated.DCAE_CL_OUTPUT"86400.0,
"subscriberConsumerGrouptcaAlertsAbatementTableName" : "OpenDCAE-c1TCAAlertsAbatementTable",
"subscriberConsumerIdtcaAlertsAbatementTableTTLSeconds" : "c1"1728000.0,
"subscriberContentTypetcaVESAlertsTableName" : "application/jsonTCAVESAlertsTable",
"subscriberHostNametcaVESAlertsTableTTLSeconds" : "10.12.25.96"1728000.0,
"subscriberHostPortthresholdCalculatorFlowletInstances" : "3904"2.0,
"subscriberMessageLimittcaSubscriberOutputStreamName" : "-1TCASubscriberOutputStream",
}
"subscriberPollingInterval" : "20000",}
|
6) Create TCA app preference file - "tca_app_preferences.json" under ~ubuntu as below
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
{ "subscriberProtocolpublisherContentType" : "httpapplication/json", "subscriberTimeoutMSpublisherHostName" : "-110.12.25.96", "subscriberTopicNamepublisherHostPort" : "unauthenticated.SEC_MEASUREMENT_OUTPUT3904", "enableAAIEnrichmentpublisherMaxBatchSize" : false"1", "aaiEnrichmentHostpublisherMaxRecoveryQueueSize" : "10.12.25.72100000", "aaiEnrichmentPortNumberpublisherPollingInterval" : 8443"20000", "aaiEnrichmentProtocolpublisherProtocol" : "httpshttp", "aaiEnrichmentUserNamepublisherTopicName" : "unauthenticated.DCAE_CL_OUTPUT", "aaiEnrichmentUserPasswordsubscriberConsumerGroup" : "DCAEOpenDCAE-c1", "aaiEnrichmentIgnoreSSLCertificateErrorssubscriberConsumerId" : false"c1", "aaiVNFEnrichmentAPIPathsubscriberContentType" : "/aai/v11/network/generic-vnfs/generic-vnf", application/json", "aaiVMEnrichmentAPIPathsubscriberHostName" : "/aai/v11/search/nodes-query "10.12.25.96", "tca_policysubscriberHostPort" : "3904"{, \"domain\subscriberMessageLimit" : \"measurementsForVfScaling\-1", \"metricsPerEventName\subscriberPollingInterval" : [{ \"eventName\": \"vFirewallBroadcastPackets\"20000", "subscriberProtocol" : "http", \"controlLoopSchemaType\subscriberTimeoutMS" : \"VNF\-1", \"policyScope\subscriberTopicName" : \"DCAE\", \"policyName\": \"DCAE.Config_tca-hi-lo\", \"policyVersion\": \"v0.0.1\", \"thresholds\": [{ \"closedLoopControlName\": \"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\", \"version\": \"1.0.2\", \"fieldPath\": \"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta\", \"thresholdValue\": 300, \"direction\": \"LESS_OR_EQUAL\", \"severity\": \"MAJOR\", \"closedLoopEventStatus\": \"ONSET\" }, { \"closedLoopControlName\": \"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\", \"version\": \"1.0.2\", \"fieldPath\": \"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta\", \"thresholdValue\": 700, \"direction\": \"GREATER_OR_EQUAL\", \"severity\": \"CRITICAL"unauthenticated.SEC_MEASUREMENT_OUTPUT", "enableAAIEnrichment" : false, "aaiEnrichmentHost" : "10.12.25.72", "aaiEnrichmentPortNumber" : 8443, "aaiEnrichmentProtocol" : "https", "aaiEnrichmentUserName" : "DCAE", "aaiEnrichmentUserPassword" : "DCAE", "aaiEnrichmentIgnoreSSLCertificateErrors" : true, "aaiVNFEnrichmentAPIPath" : "/aai/v11/network/generic-vnfs/generic-vnf", "aaiVMEnrichmentAPIPath" : "/aai/v11/search/nodes-query", "tca_policy" : "{ \"domain\": \"measurementsForVfScaling\", \"metricsPerEventName\": [{ \"eventName\": \"vFirewallBroadcastPackets\", \"controlLoopSchemaType\": \"VNF\", \"policyScope\": \"DCAE\", \"policyName\": \"DCAE.Config_tca-hi-lo\", \"policyVersion\": \"v0.0.1\", \"thresholds\": [{ \"closedLoopControlName\": \"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\", \"closedLoopEventStatusversion\": \"ONSET1.0.2\" }] }, { \"eventNamefieldPath\": \"vLoadBalancer$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta\", \"controlLoopSchemaTypethresholdValue\": \"VM\"300, \"policyScopedirection\": \"DCAELESS_OR_EQUAL\", \"policyNameseverity\": \"DCAE.Config_tca-hi-loMAJOR\", \"policyVersionclosedLoopEventStatus\": \"v0.0.1ONSET\", \"thresholds\": [}, { \"closedLoopControlName\": \"ControlLoop-vDNSvFirewall-6f37f56dd0a1dfc6-a87d94f5-4b854fd4-b6a9a5b5-cc953cf779b34630b438850a\", \"version\": \"1.0.2\", \"fieldPath\": \"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta\", \"thresholdValue\": 300700, \"direction\": \"GREATER_OR_EQUAL\", \"severity\": \"CRITICAL\", \"closedLoopEventStatus\": \"ONSET\" }] }, { \"eventName\": \"Measurement_vGMUXvLoadBalancer\", \"controlLoopSchemaType\": \"VNFVM\", \"policyScope\": \"DCAE\", \"policyName\": \"DCAE.Config_tca-hi-lo\", \"policyVersion\": \"v0.0.1\", \"thresholds\": [{ \"closedLoopControlName\": \"ControlLoop-vCPEvDNS-48f0c2c36f37f56d-a172a87d-41924b85-9ae3b6a9-052274181b6ecc953cf779b3\", \"version\": \"1.0.2\", \"fieldPath\": \"$.event.measurementsForVfScalingFields.additionalMeasurementsvNicUsageArray[*].arrayOfFields[0].valuereceivedTotalPacketsDelta\", \"thresholdValue\": 0300, \"direction\": \"GREATER_OR_EQUAL\", \"severity\": \"MAJORCRITICAL\", \"closedLoopEventStatus\": \"ABATEDONSET\" }] }, { \"closedLoopControlNameeventName\": \"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6eMeasurement_vGMUX\", \"versioncontrolLoopSchemaType\": \"1.0.2VNF\", \"fieldPathpolicyScope\": \"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].valueDCAE\", \"thresholdValuepolicyName\": 0, \"direction\"DCAE.Config_tca-hi-lo\", \"policyVersion\": \"GREATERv0.0.1\", \"thresholds\": [{ \"severityclosedLoopControlName\": \"CRITICALControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\", \"version\"closedLoopEventStatus\": \"ONSET\" }] }] }" } |
Note: Dmaap configuration are specified on this file on publisherHostName and subscriberHostName. To be changed as required**
...
: \"1.0.2\",
\"fieldPath\": \"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",
\"thresholdValue\": 0,
\"direction\": \"EQUAL\",
\"severity\": \"MAJOR\",
\"closedLoopEventStatus\": \"ABATED\"
}, {
\"closedLoopControlName\": \"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",
\"version\": \"1.0.2\",
\"fieldPath\": \"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",
\"thresholdValue\": 0,
\"direction\": \"GREATER\",
\"severity\": \"CRITICAL\",
\"closedLoopEventStatus\": \"ONSET\"
}]
}]
}"
} |
Note: Dmaap configuration are specified on this file on publisherHostName and subscriberHostName. To be changed as required**
6 a) To enable AAI Enrichment in TCA for CL event, following updates should be done to TCA app preference file before proceeding
"enableAAIEnrichment" : true "aaiEnrichmentHost" : <should be set to aai1_ip_addr> "aaiEnrichmentIgnoreSSLCertificateErrors" : true7) Copy below script to CDAP server (this gets latest image from nexus and deploys TCA application) and execute it
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
#!/bin/sh TCA_JAR=dcae-analytics-cdap-tca-2.0.0.jar rm -f /home/ubuntu/$TCA_JAR cd /home/ubuntu/ wget https://nexusnexus3.onap.org/service/local/repositories/stagingreleases/content/org/onap/dcaegen2/analytics/tca/dcae-analytics-cdap-tca/2.0.0/$TCA_JAR if [ $? -eq 0 ]; then if [ -f /home/ubuntu/$TCA_JAR ]; then echo "Restarting TCA CDAP application using $TCA_JAR artifact" else echo "ERROR: $TCA_JAR missing" exit 1 fi else echo "ERROR: $TCA_JAR not found in nexus" exit 1 fi # stop programs curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/stop curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/stop curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/stop # delete application curl -X DELETE http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca#tca # delete artifact curl -X DELETE http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/artifacts/dcae-analytics-cdap-tca/versions/2.0.0#1 # load artifact curl -X POST --data-binary @/home/ubuntu/$TCA_JAR http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/artifacts/dcae-analytics-cdap-tcaecho#tca # create app curl -X PUT -d @/home/ubuntu/tca_app_config.json http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca#tca # load preferences curl -X PUT -d @/home/ubuntu/tca_app_preferences.json http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/preferences#preferences # start programs curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/startecho#start echo # get status of programs curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/status curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/status curl http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/statusecho |
...
status
echo |
Note: Attached the script also for reference : reload-tca.sh
8) Verify TCA application and logs via CDAP GUI processes
...
http://<hostname/ip>:11011/oldcdap/ns/cdap_tca_hi_lo/apps/dcae-tca/programs/flows/TCAVESCollectorFlow/runs
...
TCA Configuration Change
Typical configuration changes include changing DMAAP host and/or Policy configuration. If necessary, modify the file on step #5 #6 and run the script noted as step #6 #7 to redeploy TCA with updated configuration.
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
#!/bin/sh #docker run -d --name cdap-sdk -p 11011:11011 -p 11015:11015 caskdata/cdap-standalone:4.1.2 sudo docker restart cdap-sdk-2 sleep 30 # start program curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRPublisherWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/workers/TCADMaaPMRSubscriberWorker/start curl -X POST http://localhost:11015/v3/namespaces/cdap_tca_hi_lo/apps/dcae-tca/flows/TCAVESCollectorFlow/start |
...
This script was invoked via VM init script (rc.d).
...