DCAE R6 TCA-GEN2 MicroService

Owned by Vijay Kumar
Last updated: Apr 15, 2020
9 min read

Overview


The TCA-gen2 is docker based mS intended to replace TCA/cdap version, which was first delivered as part of ONAP R0. Functionality of TCA-gen2 is identical to that of TCA - where meaurement events are subscribed from DMAAP in VES structure, once events are recieved TCA-gen2 performs a comparison of an incoming performance metric(s) against both a high and low threshold defined and generates CL events when threshold are exceeded. When the original threshold defined are cleared, TCA-Gen2 will generate an ABATEMENT event to notify the downstream system on original problem being cleared.


Artifacts

Βlueprint (deployment artifact) :  https://git.onap.org/dcaegen2/platform/blueprints/tree/blueprints/k8s-tcagen2.yaml

Input file (deployment input)    :   https://git.onap.org/oom/tree/kubernetes/dcaegen2/components/dcae-bootstrap/resources/inputs/k8s-tcagen2-inputs.yaml

Docker image                            :   nexus3.onap.org:10001/onap/org.onap.dcaegen2.analytics.tca-gen2.dcae-analytics-tca-web:1.0.1



Deployment Prerequisite/dependencies

  • DCAE and DMaaP pods should be up and running.
  • MongoDB should be up and running


mongodb
 root@onap2-nfs:~# kubectl get pods -n onap | grep mongo
 dev-dcae-mongo-0 1/1 Running 0 5d22h


Deployment Steps


Deployment of Tca-gen2 can be done using Dashboard UI or CloudifyUI or via CLI. Below steps are based on CLI.


  • Log-in to the DCAE bootstrap POD's main container
  • Update blueprint file in DCAE bootstrap POD under /blueprints directory if required
  • Update blueprint component inputs file in DCAE bootstrap POD under /inputs directory

  • Validate blueprint

    Validate Blueprint
    cfy blueprints validate /blueprints/k8s-tca-gen2.yaml

  • Verify Plugin versions in target Cloudify instance match to blueprint imports

    Verify Plugin version
    cfy plugins list

    If the version of plugin used are different, update the blueprint import to match.

  • Deploy Service

    Upload and deploy blueprint
    cfy install --blueprint-id tcagen2 --deployment-id tcagen2 -i /inputs/k8s-tcagen2-inputs.yaml /blueprints/k8s-tcagen2.yaml

To un-deploy

  • Uninstall running component and delete deployment

    Uninstall component
    cfy uninstall tcagen2

  • Delete blueprint

    Delete blueprint
    cfy blueprints delete tcagen2

Initial Validation

After deployment, verify if tcagen2 POD and mongoDB pod are running correctly

Verify Heartbeat is running
root@k8s-rancher:~# kubectl get pods -n onap | egrep "tcagen2|mongoDB"


And then check the logs to see if it can connect to DMaaP, polling for events.

Verify Logs for Dmaap poll
kubectl logs -f -n onap dep-dcae-tca-gen2-service-5ff7558fd-4nt6z  dcae-tcagen2
.

Functional tests

Following default configuration is loaded into tcagen2 (set in blueprint configuration)


Configuration
{
	"spring.data.mongodb.uri": "mongodb://dcae-mongohost/dcae-tcagen2",
	"streams_subscribes": {
		"tca_handle_in": {
			"type": "message_router",
			"dmaap_info": {
				"topic_url": "http://message-router:3904/events/unauthenticated.VES_MEASUREMENT_OUTPUT/"
			}
		}
	},
	"tca.enable_ecomp_logging": true,
	"tca.enable_abatement": true,
	"tca.aai.password": "DCAE",
	"streams_subscribes.tca_handle_in.consumer_group": "cg1",
	"streams_subscribes.tca_handle_in.polling.auto_adjusting.step_up": 10000,
	"tca.aai.node_query_path": "aai/v11/search/nodes-query",
	"streams_publishes": {
		"tca_handle_out": {
			"type": "message_router",
			"dmaap_info": {
				"topic_url": "http://message-router:3904/events/unauthenticated.TCAGEN2_OUTPUT/"
			}
		}
	},
	"streams_subscribes.tca_handle_in.consumer_ids[1]": "c1",
	"tca.aai.generic_vnf_path": "aai/v11/network/generic-vnfs/generic-vnf",
	"streams_subscribes.tca_handle_in.polling.auto_adjusting.step_down": 30000,
	"streams_subscribes.tca_handle_in.polling.auto_adjusting.max": 60000,
	"tca.aai.username": "DCAE",
	"streams_subscribes.tca_handle_in.polling.auto_adjusting.min": 30000,
	"tca.aai.url": "http://aai.onap.svc.cluster.local",
	"streams_subscribes.tca_handle_in.timeout": -1,
	"tca.aai.enable_enrichment": true,
	"tca.policy": "{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"vFirewallBroadcastPackets\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"LESS_OR_EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ONSET\"},{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":700,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"vLoadBalancer\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"Measurement_vGMUX\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"GREATER\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}",
	"tca.processing_batch_size": 10000,
	"streams_subscribes.tca_handle_in.polling.fixed_rate": 0,
	"streams_subscribes.tca_handle_in.message_limit": 50000,
	"service_calls": [],
	"streams_subscribes.tca_handle_in.consumer_ids[0]": "c0"
}



TCA Control Loop configuration

CL configuration 
{
	"domain": "measurementsForVfScaling",
	"metricsPerEventName": [{
		"eventName": "vFirewallBroadcastPackets",
		"controlLoopSchemaType": "VM",
		"policyScope": "DCAE",
		"policyName": "DCAE.Config_tca-hi-lo",
		"policyVersion": "v0.0.1",
		"thresholds": [{
			"closedLoopControlName": "ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a",
			"version": "1.0.2",
			"fieldPath": "$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta",
			"thresholdValue": 300,
			"direction": "LESS_OR_EQUAL",
			"severity": "MAJOR",
			"closedLoopEventStatus": "ONSET"
		}, {
			"closedLoopControlName": "ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a",
			"version": "1.0.2",
			"fieldPath": "$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta",
			"thresholdValue": 700,
			"direction": "GREATER_OR_EQUAL",
			"severity": "CRITICAL",
			"closedLoopEventStatus": "ONSET"
		}]
	}, {
		"eventName": "vLoadBalancer",
		"controlLoopSchemaType": "VM",
		"policyScope": "DCAE",
		"policyName": "DCAE.Config_tca-hi-lo",
		"policyVersion": "v0.0.1",
		"thresholds": [{
			"closedLoopControlName": "ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3",
			"version": "1.0.2",
			"fieldPath": "$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta",
			"thresholdValue": 300,
			"direction": "GREATER_OR_EQUAL",
			"severity": "CRITICAL",
			"closedLoopEventStatus": "ONSET"
		}]
	}, {
		"eventName": "Measurement_vGMUX",
		"controlLoopSchemaType": "VNF",
		"policyScope": "DCAE",
		"policyName": "DCAE.Config_tca-hi-lo",
		"policyVersion": "v0.0.1",
		"thresholds": [{
			"closedLoopControlName": "ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e",
			"version": "1.0.2",
			"fieldPath": "$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value",
			"thresholdValue": 0,
			"direction": "EQUAL",
			"severity": "MAJOR",
			"closedLoopEventStatus": "ABATED"
		}, {
			"closedLoopControlName": "ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e",
			"version": "1.0.2",
			"fieldPath": "$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value",
			"thresholdValue": 0,
			"direction": "GREATER",
			"severity": "CRITICAL",
			"closedLoopEventStatus": "ONSET"
		}]
	}]
}


To simulate the event flow and trigger missing tca-gen2 event, we can simulate a VES event into tca-gen2 subscription topic (using curl).

Generate tca-gen2 CL Onset

Send a triggering event to VES Collector; the output gets published to DMaaP topic unauthenticated.VES_MEASUREMENT_OUTPUT

Before sending, validate following

  • "eventName" field in VES input matches eventName configuration set in tca-gen2 Service
  • "lastEpochMicrosec" in reflecting current time stamp approximately


Trigger a Heartbeat 
curl -i -u 'sample1:sample1' -X POST -d @f5-cl.txt --header "Content-Type: application/json" https://10.12.7.116:30417/eventListener/v5 -k
HTTP/1.1 202
Content-Type: application/json
Content-Length: 8
Date: Tue, 14 Apr 2020 16:36:43 GMT


Where f5-cl.txt contains below VES event

{
	"event": {
		"commonEventHeader": {
			"startEpochMicrosec": 1507750704126521,
			"sourceId": "4c32fa09-83e7-43cf-8bba-c72557afe408",
			"eventId": "TrafficStats_1.2.3.4",
			"nfcNamingCode": "vVNF",
			"reportingEntityId": "No UUID available",
			"eventType": "HTTP request rate",
			"priority": "Normal",
			"version": 1.2,
			"reportingEntityName": "fwll",
			"sequence": 85,
			"domain": "measurementsForVfScaling",
			"lastEpochMicrosec": 1507750714317473,
			"eventName": "vFirewallBroadcastPackets",
			"sourceName": "zdfw1fwl01fwl01",
			"nfNamingCode": "vVNF"
		},
		"measurementsForVfScalingFields": {
			"cpuUsageArray": [{
				"percentUsage": 0,
				"cpuIdentifier": "cpu1",
				"cpuIdle": 100,
				"cpuUsageSystem": 0,
				"cpuUsageUser": 0
			}],
			"measurementInterval": 10,
			"requestRate": 2256,
			"vNicPerformanceArray": [{
				"transmittedOctetsDelta": 0,
				"receivedTotalPacketsDelta": 1002,
				"vNicIdentifier": "eth0",
				"valuesAreSuspect": "true",
				"transmittedTotalPacketsDelta": 0,
				"receivedOctetsDelta": 61200
			}],
			"measurementsForVfScalingVersion": 2.1
		}
	}
}


VES output
curl  https://10.12.7.116:30226/events/unauthenticated.VES_MEASUREMENT_OUTPUT/vv/1 -k

["{\"event\":{\"commonEventHeader\":{\"startEpochMicrosec\":1507750704126521,\"sourceId\":\"4c32fa09-83e7-43cf-8bba-c72557afe408\",\"eventId\":\"TrafficStats_1.2.3.4\",\"nfcNamingCode\":\"vVNF\",\"reportingEntityId\":\"No UUID available\",\"internalHeaderFields\":{\"collectorTimeStamp\":\"Tue, 04 14 2020 04:39:02 UTC\"},\"eventType\":\"HTTP request rate\",\"priority\":\"Normal\",\"version\":1.2,\"reportingEntityName\":\"fwll\",\"sequence\":85,\"domain\":\"measurementsForVfScaling\",\"lastEpochMicrosec\":1507750714317473,\"eventName\":\"vFirewallBroadcastPackets\",\"sourceName\":\"zdfw1fwl01fwl01\",\"nfNamingCode\":\"vVNF\"},\"measurementsForVfScalingFields\":{\"cpuUsageArray\":[{\"percentUsage\":0,\"cpuIdentifier\":\"cpu1\",\"cpuIdle\":100,\"cpuUsageSystem\":0,\"cpuUsageUser\":0}],\"measurementInterval\":10,\"requestRate\":2256,\"measurementsForVfScalingVersion\":2.1,\"vNicPerformanceArray\":[{\"transmittedOctetsDelta\":0,\"receivedTotalPacketsDelta\":1002,\"vNicIdentifier\":\"eth0\",\"valuesAreSuspect\":\"true\",\"transmittedTotalPacketsDelta\":0,\"receivedOctetsDelta\":61200}]}}}"]


We can check that a new  TCAGEN2 output topic if CL event has been published (make sure you target the correct DMaaP IP address)

CL output
curl  https://10.12.7.116:30226/events/unauthenticated.TCAGEN2_OUTPUT/vv/1 -k
["{\"closedLoopEventClient\":\"dcae-tcagen2\",\"policyVersion\":\"v0.0.1\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyScope\":\"DCAE\",\"target_type\":\"VM\",\"AAI\":{\"vserver.vserver-name\":\"zdfw1fwl01fwl01\"},\"closedLoopAlarmStart\":1507750704126521,\"closedLoopEventStatus\":\"ONSET\",\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"target\":\"vserver.vserver-name\",\"requestID\":\"e727df74-07e8-4711-b31c-7c28adf721b2:0:0\",\"from\":\"DCAE\"}"]


TCA processing logs
TCA


83-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ, Elapsed Time: 621 ms, REST Endpoint Call: OK-GET-/events/unauthenticated.VES_MEASUREMENT_OUTPUT/cg1/c1
2020-04-14 16:37:11.468  INFO 1 --- [ask-scheduler-9] o.o.d.a.w.http.EelfAuditLogInterceptor   : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ, Elapsed Time: 621 ms, REST Endpoint Call: OK-GET-/events/unauthenticated.VES_MEASUREMENT_OUTPUT/cg1/c1
2020-04-14T16:37:11.470+0000|f4a06883-9f1e-4942-8ebe-73900e8af929|Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ, Messages Present: true, Next Polling Interval will be: 30000
2020-04-14 16:37:11.470 DEBUG 1 --- [ask-scheduler-9] o.o.d.a.w.d.MrSubscriberPollingAdvice    : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ, Messages Present: true, Next Polling Interval will be: 30000
2020-04-14 16:37:11.471 DEBUG 1 --- [ask-scheduler-9] o.o.d.a.web.dmaap.MrMessageSplitter      : mrMessageSplitter received message: GenericMessage [payload=["{\"event\":{\"commonEventHeader\":{\"startEpochMicrosec\":1507750704126521,\"sourceId\":\"4c32fa09-83e7-43cf-8bba-c72557afe408\",\"eventId\":\"TrafficStats_1.2.3.4\",\"nfcNamingCode\":\"vVNF\",\"reportingEntityId\":\"No UUID available\",\"internalHeaderFields\":{\"collectorTimeStamp\":\"Tue, 04 14 2020 04:36:43 UTC\"},\"eventType\":\"HTTP request rate\",\"priority\":\"Normal\",\"version\":1.2,\"reportingEntityName\":\"fwll\",\"sequence\":85,\"domain\":\"measurementsForVfScaling\",\"lastEpochMicrosec\":1507750714317473,\"eventName\":\"vFirewallBroadcastPackets\",\"sourceName\":\"zdfw1fwl01fwl01\",\"nfNamingCode\":\"vVNF\"},\"measurementsForVfScalingFields\":{\"cpuUsageArray\":[{\"percentUsage\":0,\"cpuIdentifier\":\"cpu1\",\"cpuIdle\":100,\"cpuUsageSystem\":0,\"cpuUsageUser\":0}],\"measurementInterval\":10,\"requestRate\":2256,\"measurementsForVfScalingVersion\":2.1,\"vNicPerformanceArray\":[{\"transmittedOctetsDelta\":0,\"receivedTotalPacketsDelta\":1002,\"vNicIdentifier\":\"eth0\",\"valuesAreSuspect\":\"true\",\"transmittedTotalPacketsDelta\":0,\"receivedOctetsDelta\":61200}]}}}"], headers={Cache-Control=no-store, no-cache, must-revalidate, Server=Jetty(9.4.z-SNAPSHOT), X-ECOMP-RequestID=f4a06883-9f1e-4942-8ebe-73900e8af929, X-ECOMP-TransactionID=US2MQ, X-Begin-Timestamp=2020-04-14T16:37:10.845+0000, Pragma=no-cache, id=7be9866a-c058-12e5-ed8b-4c115af62663, Content-Length=1100, contentType=application/json, http_statusCode=200, Date=1586882230000, timestamp=1586882231470}]
2020-04-14T16:37:10.845+0000|2020-04-14T16:37:11.498+0000|f4a06883-9f1e-4942-8ebe-73900e8af929|UNKNOWN_INSTANCE_ID|task-scheduler-9||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|653|dcae-tcagen2||org.onap.dcae.analytics.web.dmaap.MrMessageSplitter|||||||Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ, dmaapMessages: [{"event":{"commonEventHeader":{"startEpochMicrosec":1507750704126521,"sourceId":"4c32fa09-83e7-43cf-8bba-c72557afe408","eventId":"TrafficStats_1.2.3.4","nfcNamingCode":"vVNF","reportingEntityId":"No UUID available","internalHeaderFields":{"collectorTimeStamp":"Tue, 04 14 2020 04:36:43 UTC"},"eventType":"HTTP request rate","priority":"Normal","version":1.2,"reportingEntityName":"fwll","sequence":85,"domain":"measurementsForVfScaling","lastEpochMicrosec":1507750714317473,"eventName":"vFirewallBroadcastPackets","sourceName":"zdfw1fwl01fwl01","nfNamingCode":"vVNF"},"measurementsForVfScalingFields":{"cpuUsageArray":[{"percentUsage":0,"cpuIdentifier":"cpu1","cpuIdle":100,"cpuUsageSystem":0,"cpuUsageUser":0}],"measurementInterval":10,"requestRate":2256,"measurementsForVfScalingVersion":2.1,"vNicPerformanceArray":[{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":1002,"vNicIdentifier":"eth0","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":61200}]}}}], Received new messages from DMaaP MR. Count: 1
2020-04-14 16:37:11.500  INFO 1 --- [ask-scheduler-9] o.o.d.a.web.dmaap.MrMessageSplitter      : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ, dmaapMessages: [{"event":{"commonEventHeader":{"startEpochMicrosec":1507750704126521,"sourceId":"4c32fa09-83e7-43cf-8bba-c72557afe408","eventId":"TrafficStats_1.2.3.4","nfcNamingCode":"vVNF","reportingEntityId":"No UUID available","internalHeaderFields":{"collectorTimeStamp":"Tue, 04 14 2020 04:36:43 UTC"},"eventType":"HTTP request rate","priority":"Normal","version":1.2,"reportingEntityName":"fwll","sequence":85,"domain":"measurementsForVfScaling","lastEpochMicrosec":1507750714317473,"eventName":"vFirewallBroadcastPackets","sourceName":"zdfw1fwl01fwl01","nfNamingCode":"vVNF"},"measurementsForVfScalingFields":{"cpuUsageArray":[{"percentUsage":0,"cpuIdentifier":"cpu1","cpuIdle":100,"cpuUsageSystem":0,"cpuUsageUser":0}],"measurementInterval":10,"requestRate":2256,"measurementsForVfScalingVersion":2.1,"vNicPerformanceArray":[{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":1002,"vNicIdentifier":"eth0","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":61200}]}}}], Received new messages from DMaaP MR. Count: 1
2020-04-14T16:37:10.845+0000|2020-04-14T16:37:11.498+0000|f4a06883-9f1e-4942-8ebe-73900e8af929|UNKNOWN_INSTANCE_ID|task-scheduler-9||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|653|dcae-tcagen2||org.onap.dcae.analytics.web.dmaap.MrMessageSplitter|||||||Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ,  Max allowed messages per batch: 10000. No of batches created: 1
2020-04-14 16:37:11.501  INFO 1 --- [ask-scheduler-9] o.o.d.a.web.dmaap.MrMessageSplitter      : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929, Transaction Id: US2MQ,  Max allowed messages per batch: 10000. No of batches created: 1
2020-04-14T16:37:11.719+0000|f4a06883-9f1e-4942-8ebe-73900e8af929:0:0|Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0:0. Alert ClosedLoop Status is ONSET. Saving abatement Entity to repository with lookupKey: vFirewallBroadcastPackets:zdfw1fwl01fwl01:fwll:ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta
2020-04-14 16:37:11.719 DEBUG 1 --- [ask-scheduler-5] o.o.d.a.t.c.u.f.c.TcaAbatementCalculator : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0:0. Alert ClosedLoop Status is ONSET. Saving abatement Entity to repository with lookupKey: vFirewallBroadcastPackets:zdfw1fwl01fwl01:fwll:ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta
2020-04-14 16:37:11.771 DEBUG 1 --- [ask-scheduler-5] .d.a.w.h.HttpClientPreferencesCustomizer : Customizing Rest Template for Http Client Id: aai-rest-client
2020-04-14 16:37:11.771  INFO 1 --- [ask-scheduler-5] .d.a.w.h.HttpClientPreferencesCustomizer : Setting basic Authentication credentials for Http Client Id: aai-rest-client with username: DCAE
2020-04-14 16:37:11.773 DEBUG 1 --- [ask-scheduler-5] .d.a.w.h.HttpClientPreferencesCustomizer : Proxy not Enabled - bypassing setting Proxy settings for Http Client Id: aai-rest-client
2020-04-14 16:37:11.773  INFO 1 --- [ask-scheduler-5] .d.a.w.h.HttpClientPreferencesCustomizer : Ignore SSL Certificate Errors attributed is set to: false for Http Client Id: aai-rest-client
2020-04-14 16:37:11.773  INFO 1 --- [ask-scheduler-5] .d.a.w.h.HttpClientPreferencesCustomizer : SSL Validation will be enforced for Http Client Id: aai-rest-client
2020-04-14T16:37:11.781+0000|2020-04-14T16:37:11.785+0000|f4a06883-9f1e-4942-8ebe-73900e8af929:0:0|UNKNOWN_INSTANCE_ID|task-scheduler-5||dcae-tcagen2|DCAE|AAI_ENRICHMENT|GET-/aai/v11/search/nodes-query|ERROR|200|Timeout Error||ERROR|2|10.42.1.198|4|dcae-tcagen2||org.onap.dcae.analytics.web.http.EelfAuditLogInterceptor|||aai.onap.svc.cluster.local|||||Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0:0, Transaction Id: US2MQ, Elapsed Time: 4 ms, Error Message: java.net.UnknownHostException: aai.onap.svc.cluster.local: Name does not resolve
2020-04-14 16:37:11.785 ERROR 1 --- [ask-scheduler-5] o.o.d.a.w.http.EelfAuditLogInterceptor   : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0:0, Transaction Id: US2MQ, Elapsed Time: 4 ms, Error Message: java.net.UnknownHostException: aai.onap.svc.cluster.local: Name does not resolve
2020-04-14T16:37:11.789+0000|f4a06883-9f1e-4942-8ebe-73900e8af929:0:0|task-scheduler-5|dcae-tcagen2|tca-gen2|DCAE-TCA|dcae-tcagen2|ERROR|400|Schema Error|Request id: f4a06883-9f1e-4942-8ebe-73900e8af929:0:0. Unable to get A&AI enrichment details
2020-04-14 16:37:11.789 ERROR 1 --- [ask-scheduler-5] .o.d.a.t.w.a.TcaAaiEnrichmentServiceImpl : Request id: f4a06883-9f1e-4942-8ebe-73900e8af929:0:0. Unable to get A&AI enrichment details
2020-04-14T16:37:10.845+0000|2020-04-14T16:37:11.790+0000|f4a06883-9f1e-4942-8ebe-73900e8af929:0|UNKNOWN_INSTANCE_ID|task-scheduler-5||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|945|dcae-tcagen2||org.onap.dcae.analytics.tca.web.integration.TcaAlertTransformer|||||||Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0, Transaction Id: US2MQ, Message counts - Received: 1, Errors: 0, Terminated Early: 0, Abated: [TcaAlert(aai=Aai(genericVNFName=null, genericServerName=zdfw1fwl01fwl01), closedLoopAlarmEnd=null, closedLoopAlarmStart=1507750704126521, closedLoopControlName=ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a, closedLoopEventClient=dcae-tcagen2, closedLoopEventStatus=ONSET, from=DCAE, policyName=DCAE.Config_tca-hi-lo, policyScope=DCAE, policyVersion=v0.0.1, requestId=f4a06883-9f1e-4942-8ebe-73900e8af929:0:0, target=vserver.vserver-name, targetType=VM, version=1.0.2)], Alerts: 0, Alerts size: 1
2020-04-14 16:37:11.790  INFO 1 --- [ask-scheduler-5] o.o.d.a.t.w.i.TcaAlertTransformer        : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0, Transaction Id: US2MQ, Message counts - Received: 1, Errors: 0, Terminated Early: 0, Abated: [TcaAlert(aai=Aai(genericVNFName=null, genericServerName=zdfw1fwl01fwl01), closedLoopAlarmEnd=null, closedLoopAlarmStart=1507750704126521, closedLoopControlName=ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a, closedLoopEventClient=dcae-tcagen2, closedLoopEventStatus=ONSET, from=DCAE, policyName=DCAE.Config_tca-hi-lo, policyScope=DCAE, policyVersion=v0.0.1, requestId=f4a06883-9f1e-4942-8ebe-73900e8af929:0:0, target=vserver.vserver-name, targetType=VM, version=1.0.2)], Alerts: 0, Alerts size: 1
2020-04-14T16:37:11.796+0000|2020-04-14T16:37:12.165+0000|f4a06883-9f1e-4942-8ebe-73900e8af929:0|UNKNOWN_INSTANCE_ID|task-scheduler-5||dcae-tcagen2||DMAAP_MR_PUBLISHER|POST-/events/unauthenticated.TCAGEN2_OUTPUT/|COMPLETE|0|OK||INFO|0|10.42.1.198|369|dcae-tcagen2||org.onap.dcae.analytics.web.http.EelfAuditLogInterceptor|||message-router:3904|||||Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0, Transaction Id: US2MQ, Elapsed Time: 369 ms, REST Endpoint Call: OK-POST-/events/unauthenticated.TCAGEN2_OUTPUT/
2020-04-14 16:37:12.165  INFO 1 --- [ask-scheduler-5] o.o.d.a.w.http.EelfAuditLogInterceptor   : Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0, Transaction Id: US2MQ, Elapsed Time: 369 ms, REST Endpoint Call: OK-POST-/events/unauthenticated.TCAGEN2_OUTPUT/
2020-04-14T16:37:10.845+0000|2020-04-14T16:37:12.168+0000|f4a06883-9f1e-4942-8ebe-73900e8af929:0|UNKNOWN_INSTANCE_ID|task-scheduler-5||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|1323|dcae-tcagen2||org.onap.dcae.analytics.tca.web.integration.TcaPublisherResponseHandler|||||||Request Id: f4a06883-9f1e-4942-8ebe-73900e8af929:0, Transaction Id: US2MQ, Transaction completion Time: 1323 ms, DMaaP MR Publisher Response: {
    "serverTimeMs": 362,
    "count": 1
}


Generate TCAGEN2 CL Abatement Event


Send a triggering event to VESCollector. VESCollector publishes the event into unauthenticated.VES_MEASUREMENT_OUTPUT

Before sending, validate following

  • "eventName" field in VES input matches eventName configuration set in tca-gen2 Service
  • "lastEpochMicrosec" in reflecting current time stamp approximately
  •  Ensure the event is on the same "sourceName" as original event sent


Resend heartbeat with new timestamp
curl -i -u 'sample1:sample1' -X POST -d @f5-cl-abate.txt  --header "Content-Type: application/json" https://10.12.7.116:30417/eventListener/v5 -k
HTTP/1.1 202
Content-Type: application/json
Content-Length: 8
Date: Tue, 14 Apr 2020 16:57:30 GMT

Where f5-cl-abate.txt containes below event

{"event":{"commonEventHeader":{"startEpochMicrosec":1507750704126521,"sourceId":"4c32fa09-83e7-43cf-8bba-c72557afe408","eventId":"TrafficStats_1.2.3.4","nfcNamingCode":"vVNF","reportingEntityId":"No UUID available","eventType":"HTTP request rate","priority":"Normal","version":1.2,"reportingEntityName":"fwll","sequence":85,"domain":"measurementsForVfScaling","lastEpochMicrosec":1507750714317473,"eventName":"vFirewallBroadcastPackets","sourceName":"zdfw1fwl01fwl01","nfNamingCode":"vVNF"},"measurementsForVfScalingFields":{"cpuUsageArray":[{"percentUsage":0,"cpuIdentifier":"cpu1","cpuIdle":100,"cpuUsageSystem":0,"cpuUsageUser":0}],"measurementInterval":10,"requestRate":2256,"vNicPerformanceArray":[{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":100,"vNicIdentifier":"eth0","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":61200}],"measurementsForVfScalingVersion":2.1}}}


Logs showing CL abatement event generation
2020-04-14 16:58:11.459  INFO 1 --- [ask-scheduler-8] o.o.d.a.web.dmaap.MrMessageSplitter      : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6, Transaction Id: INWFH, dmaapMessages: [{"event":{"commonEventHeader":{"startEpochMicrosec":1507750704126521,"sourceId":"4c32fa09-83e7-43cf-8bba-c72557afe408","eventId":"TrafficStats_1.2.3.4","nfcNamingCode":"vVNF","reportingEntityId":"No UUID available","internalHeaderFields":{"collectorTimeStamp":"Tue, 04 14 2020 04:57:31 UTC"},"eventType":"HTTP request rate","priority":"Normal","version":1.2,"reportingEntityName":"fwll","sequence":85,"domain":"measurementsForVfScaling","lastEpochMicrosec":1507750714317473,"eventName":"vFirewallBroadcastPackets","sourceName":"zdfw1fwl01fwl01","nfNamingCode":"vVNF"},"measurementsForVfScalingFields":{"cpuUsageArray":[{"percentUsage":0,"cpuIdentifier":"cpu1","cpuIdle":100,"cpuUsageSystem":0,"cpuUsageUser":0}],"measurementInterval":10,"requestRate":2256,"measurementsForVfScalingVersion":2.1,"vNicPerformanceArray":[{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":100,"vNicIdentifier":"eth0","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":61200}]}}}], Received new messages from DMaaP MR. Count: 1
2020-04-14T16:58:10.845+0000|2020-04-14T16:58:11.459+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6|UNKNOWN_INSTANCE_ID|task-scheduler-8||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|614|dcae-tcagen2||org.onap.dcae.analytics.web.dmaap.MrMessageSplitter|||||||Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6, Transaction Id: INWFH,  Max allowed messages per batch: 10000. No of batches created: 1
2020-04-14 16:58:11.459  INFO 1 --- [ask-scheduler-8] o.o.d.a.web.dmaap.MrMessageSplitter      : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6, Transaction Id: INWFH,  Max allowed messages per batch: 10000. No of batches created: 1
2020-04-14T16:58:11.498+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0|Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0. Found previous Abatement Entity with lookupKey: vFirewallBroadcastPackets:zdfw1fwl01fwl01:fwll:ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta
2020-04-14 16:58:11.498 DEBUG 1 --- [ask-scheduler-3] o.o.d.a.t.c.u.f.c.TcaAbatementCalculator : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0. Found previous Abatement Entity with lookupKey: vFirewallBroadcastPackets:zdfw1fwl01fwl01:fwll:ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta
2020-04-14T16:58:11.498+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0|Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0. No previous abated alert was sent. Setting previous request id: e727df74-07e8-4711-b31c-7c28adf721b2:0:0
2020-04-14 16:58:11.498 DEBUG 1 --- [ask-scheduler-3] o.o.d.a.t.c.u.f.c.TcaAbatementCalculator : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0. No previous abated alert was sent. Setting previous request id: e727df74-07e8-4711-b31c-7c28adf721b2:0:0
2020-04-14T16:58:11.498+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0|Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0. Saving new entity with alert as sent with lookupKey: vFirewallBroadcastPackets:zdfw1fwl01fwl01:fwll:ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta
2020-04-14 16:58:11.498 DEBUG 1 --- [ask-scheduler-3] o.o.d.a.t.c.u.f.c.TcaAbatementCalculator : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0:0. Saving new entity with alert as sent with lookupKey: vFirewallBroadcastPackets:zdfw1fwl01fwl01:fwll:ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a:$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta
2020-04-14T16:58:10.845+0000|2020-04-14T16:58:11.502+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6:0|UNKNOWN_INSTANCE_ID|task-scheduler-3||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|657|dcae-tcagen2||org.onap.dcae.analytics.tca.web.integration.TcaAlertTransformer|||||||Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0, Transaction Id: INWFH, Message counts - Received: 1, Errors: 0, Terminated Early: 0, Abated: [TcaAlert(aai=Aai(genericVNFName=null, genericServerName=zdfw1fwl01fwl01), closedLoopAlarmEnd=1507750714317473, closedLoopAlarmStart=1507750704126521, closedLoopControlName=ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a, closedLoopEventClient=dcae-tcagen2, closedLoopEventStatus=ABATED, from=DCAE, policyName=DCAE.Config_tca-hi-lo, policyScope=DCAE, policyVersion=v0.0.1, requestId=e727df74-07e8-4711-b31c-7c28adf721b2:0:0, target=vserver.vserver-name, targetType=VM, version=1.0.2)], Alerts: 1, Alerts size: 1
2020-04-14 16:58:11.502  INFO 1 --- [ask-scheduler-3] o.o.d.a.t.w.i.TcaAlertTransformer        : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0, Transaction Id: INWFH, Message counts - Received: 1, Errors: 0, Terminated Early: 0, Abated: [TcaAlert(aai=Aai(genericVNFName=null, genericServerName=zdfw1fwl01fwl01), closedLoopAlarmEnd=1507750714317473, closedLoopAlarmStart=1507750704126521, closedLoopControlName=ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a, closedLoopEventClient=dcae-tcagen2, closedLoopEventStatus=ABATED, from=DCAE, policyName=DCAE.Config_tca-hi-lo, policyScope=DCAE, policyVersion=v0.0.1, requestId=e727df74-07e8-4711-b31c-7c28adf721b2:0:0, target=vserver.vserver-name, targetType=VM, version=1.0.2)], Alerts: 1, Alerts size: 1
2020-04-14T16:58:11.502+0000|2020-04-14T16:58:11.507+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6:0|UNKNOWN_INSTANCE_ID|task-scheduler-3||dcae-tcagen2||DMAAP_MR_PUBLISHER|POST-/events/unauthenticated.TCAGEN2_OUTPUT/|COMPLETE|0|OK||INFO|0|10.42.1.198|5|dcae-tcagen2||org.onap.dcae.analytics.web.http.EelfAuditLogInterceptor|||message-router:3904|||||Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0, Transaction Id: INWFH, Elapsed Time: 5 ms, REST Endpoint Call: OK-POST-/events/unauthenticated.TCAGEN2_OUTPUT/
2020-04-14 16:58:11.507  INFO 1 --- [ask-scheduler-3] o.o.d.a.w.http.EelfAuditLogInterceptor   : Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0, Transaction Id: INWFH, Elapsed Time: 5 ms, REST Endpoint Call: OK-POST-/events/unauthenticated.TCAGEN2_OUTPUT/
2020-04-14T16:58:10.845+0000|2020-04-14T16:58:11.508+0000|a75f69ca-8d30-4f38-b72b-0a1def672ce6:0|UNKNOWN_INSTANCE_ID|task-scheduler-3||dcae-tcagen2|tca-gen2|COMPLETE|0|||INFO|0|10.42.1.198|663|dcae-tcagen2||org.onap.dcae.analytics.tca.web.integration.TcaPublisherResponseHandler|||||||Request Id: a75f69ca-8d30-4f38-b72b-0a1def672ce6:0, Transaction Id: INWFH, Transaction completion Time: 663 ms, DMaaP MR Publisher Response: {
    "serverTimeMs": 0,
    "count": 1
}


We can check that a new DCAE_CL_OUTPUT event has been published (make sure you target the correct DMaaP IP address)

CL Abatement Published event
curl  https://10.12.7.116:30226/events/unauthenticated.TCAGEN2_OUTPUT/vv/1 -k
["{\"closedLoopEventClient\":\"dcae-tcagen2\",\"policyVersion\":\"v0.0.1\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyScope\":\"DCAE\",\"target_type\":\"VM\",\"AAI\":{\"vserver.vserver-name\":\"zdfw1fwl01fwl01\"},\"closedLoopAlarmStart\":1507750704126521,\"closedLoopEventStatus\":\"ABATED\",\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"target\":\"vserver.vserver-name\",\"closedLoopAlarmEnd\":1507750714317473,\"requestID\":\"e727df74-07e8-4711-b31c-7c28adf721b2:0:0\",\"from\":\"DCAE\"}"]


Dynamic Configuration Update

As the tcagen2 service periodically polls Consul KV using configbindingService api's - the run time configuration of tca-gen2 service can be updated dynamically without having to redeploy/restart the service. The updates to configuration can be made directly in Consul.

Locate the servicename by executing into tcagen2 Service pod and getting env HOSTNAME value

ServiceName 
root@onap2-nfs:~# kubectl get pods -n onap | grep tcagen2
dep-dcae-tcagen2-6647bc55b9-z25tr                  2/2     Running     0          19m

root@onap2-nfs:~# kubectl exec -it -n onap dep-dcae-tcagen2-6647bc55b9-z25tr /bin/sh
Defaulting container name to dcae-tcagen2.
Use 'kubectl describe pod/dep-dcae-tcagen2-6647bc55b9-z25tr -n onap' to see all of the containers in this pod.
/opt $ env | grep -i hostname
HOSTNAME=dcae-tcagen2

Change the configuration for Service in KV-store through UI

Consul URL
http://<k8snodeip>:30270/ui/#/dc1/kv/

Example: http://10.12.7.116:30270/ui/#/dc1/kv/dcae-tcagen2/ is modified for vFw rule to trigger an ABATED event when thresholdValue is less or equal to 300


{"spring.data.mongodb.uri": "mongodb://dcae-mongohost/dcae-tcagen2", "streams_subscribes": {"tca_handle_in": {"type": "message_router", "dmaap_info": {"topic_url": "http://message-router:3904/events/unauthenticated.VES_MEASUREMENT_OUTPUT/"}}}, "tca.enable_ecomp_logging": true, "tca.enable_abatement": true, "tca.aai.password": "DCAE", "streams_subscribes.tca_handle_in.consumer_group": "cg1", "streams_subscribes.tca_handle_in.polling.auto_adjusting.step_up": 10000, "tca.aai.node_query_path": "aai/v11/search/nodes-query", "streams_publishes": {"tca_handle_out": {"type": "message_router", "dmaap_info": {"topic_url": "http://message-router:3904/events/unauthenticated.TCAGEN2_OUTPUT/"}}}, "streams_subscribes.tca_handle_in.consumer_ids[1]": "c1", "tca.aai.generic_vnf_path": "aai/v11/network/generic-vnfs/generic-vnf", "streams_subscribes.tca_handle_in.polling.auto_adjusting.step_down": 30000, "streams_subscribes.tca_handle_in.polling.auto_adjusting.max": 60000, "tca.aai.username": "DCAE", "streams_subscribes.tca_handle_in.polling.auto_adjusting.min": 30000, "tca.aai.url": "http://aai.onap.svc.cluster.local", "streams_subscribes.tca_handle_in.timeout": -1, "tca.aai.enable_enrichment": true, "tca.policy": "{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"vFirewallBroadcastPackets\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"LESS_OR_EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":700,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"vLoadBalancer\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"Measurement_vGMUX\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"GREATER\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}", "tca.processing_batch_size": 10000, "streams_subscribes.tca_handle_in.polling.fixed_rate": 0, "streams_subscribes.tca_handle_in.message_limit": 50000, "service_calls": [], "streams_subscribes.tca_handle_in.consumer_ids[0]": "c0"}


Configuration Refresh from TCA-Gen2

tca config refresh log
2020-04-14 16:50:32.653 DEBUG 1 --- [ctor-http-nio-4] r.n.http.client.HttpClientOperations     : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Received last HTTP packet
2020-04-14 16:50:32.653  INFO 1 --- [ctor-http-nio-4] o.o.d.s.s.r.s.c.c.impl.CbsClientImpl     : Got successful response from Config Binding Service
2020-04-14 16:50:32.653 DEBUG 1 --- [ctor-http-nio-4] o.o.d.s.s.r.s.c.c.impl.CbsClientImpl     : CBS response: {"config":{"spring.data.mongodb.uri":"mongodb://dcae-mongohost/dcae-tcagen2","streams_subscribes":{"tca_handle_in":{"type":"message_router","dmaap_info":{"topic_url":"http://message-router:3904/events/unauthenticated.VES_MEASUREMENT_OUTPUT/"}}},"tca.enable_ecomp_logging":true,"tca.enable_abatement":true,"tca.aai.password":"DCAE","streams_subscribes.tca_handle_in.consumer_group":"cg1","streams_subscribes.tca_handle_in.polling.auto_adjusting.step_up":10000,"tca.aai.node_query_path":"aai/v11/search/nodes-query","streams_publishes":{"tca_handle_out":{"type":"message_router","dmaap_info":{"topic_url":"http://message-router:3904/events/unauthenticated.TCAGEN2_OUTPUT/"}}},"streams_subscribes.tca_handle_in.consumer_ids[1]":"c1","tca.aai.generic_vnf_path":"aai/v11/network/generic-vnfs/generic-vnf","streams_subscribes.tca_handle_in.polling.auto_adjusting.step_down":30000,"streams_subscribes.tca_handle_in.polling.auto_adjusting.max":60000,"tca.aai.username":"DCAE","streams_subscribes.tca_handle_in.polling.auto_adjusting.min":30000,"tca.aai.url":"http://aai.onap.svc.cluster.local","streams_subscribes.tca_handle_in.timeout":-1,"tca.aai.enable_enrichment":true,"tca.policy":"{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"vFirewallBroadcastPackets\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"LESS_OR_EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":700,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"vLoadBalancer\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"Measurement_vGMUX\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"GREATER\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}","tca.processing_batch_size":10000,"streams_subscribes.tca_handle_in.polling.fixed_rate":0,"streams_subscribes.tca_handle_in.message_limit":50000,"service_calls":[],"streams_subscribes.tca_handle_in.consumer_ids[0]":"c0"}}
2020-04-14 16:50:32.654 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] onStateChange(GET{uri=/service_component_all/dcae-tcagen2, connection=PooledConnection{channel=[id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443]}}, [disconnecting])
2020-04-14 16:50:32.654 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Releasing channel
2020-04-14 16:50:32.654 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Channel cleaned, now 0 active connections and 1 inactive connections
2020-04-14T16:51:10.845+0000|a73be204-5ba7-4bbb-964c-b6f7260841c9|Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9. Transaction Id: ZMTUT. Begin TS: 2020-04-14T16:51:10.845+0000. Starting new DMaaP MR Subscriber poll.
2020-04-14 16:51:10.845 DEBUG 1 --- [ask-scheduler-1] o.o.d.a.w.d.MrTriggerMessageProvider     : Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9. Transaction Id: ZMTUT. Begin TS: 2020-04-14T16:51:10.845+0000. Starting new DMaaP MR Subscriber poll.
2020-04-14T16:51:10.846+0000|2020-04-14T16:51:20.380+0000|a73be204-5ba7-4bbb-964c-b6f7260841c9|UNKNOWN_INSTANCE_ID|task-scheduler-1||dcae-tcagen2||DMAAP_MR_SUBSCRIBER|GET-/events/unauthenticated.VES_MEASUREMENT_OUTPUT/cg1/c1|COMPLETE|0|OK||INFO|0|10.42.1.198|9534|dcae-tcagen2||org.onap.dcae.analytics.web.http.EelfAuditLogInterceptor|||message-router:3904|||||Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9, Transaction Id: ZMTUT, Elapsed Time: 9534 ms, REST Endpoint Call: OK-GET-/events/unauthenticated.VES_MEASUREMENT_OUTPUT/cg1/c1
2020-04-14 16:51:20.380  INFO 1 --- [ask-scheduler-1] o.o.d.a.w.http.EelfAuditLogInterceptor   : Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9, Transaction Id: ZMTUT, Elapsed Time: 9534 ms, REST Endpoint Call: OK-GET-/events/unauthenticated.VES_MEASUREMENT_OUTPUT/cg1/c1
2020-04-14T16:51:20.381+0000|a73be204-5ba7-4bbb-964c-b6f7260841c9|Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9, Transaction Id: ZMTUT, Messages Present: false, Next Polling Interval will be: 60000
2020-04-14 16:51:20.381 DEBUG 1 --- [ask-scheduler-1] o.o.d.a.w.d.MrSubscriberPollingAdvice    : Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9, Transaction Id: ZMTUT, Messages Present: false, Next Polling Interval will be: 60000
2020-04-14T16:51:20.382+0000|a73be204-5ba7-4bbb-964c-b6f7260841c9|Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9, Transaction Id: ZMTUT, No new messages found in DMaaP MR Response. No further processing required
2020-04-14 16:51:20.382 DEBUG 1 --- [ask-scheduler-1] o.o.d.a.w.d.MrSubscriberPollingAdvice    : Request Id: a73be204-5ba7-4bbb-964c-b6f7260841c9, Transaction Id: ZMTUT, No new messages found in DMaaP MR Response. No further processing required
2020-04-14 16:51:32.632 DEBUG 1 --- [     parallel-1] o.o.d.s.s.r.s.c.c.impl.CbsClientImpl     : Calling https://config-binding-service:10443/service_component_all/dcae-tcagen2 for configuration
2020-04-14 16:51:32.633 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Channel acquired, now 1 active connections and 0 inactive connections
2020-04-14 16:51:32.633 DEBUG 1 --- [ctor-http-nio-4] o.o.d.s.s.r.s.a.http.RxHttpClient        : Request: GET / DefaultHttpHeaders[]
2020-04-14 16:51:32.633 DEBUG 1 --- [ctor-http-nio-4] r.netty.http.client.HttpClientConnect    : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Handler is being applied: {uri=https://config-binding-service:10443/service_component_all/dcae-tcagen2, method=GET}
2020-04-14 16:51:32.634 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] onStateChange(GET{uri=/service_component_all/dcae-tcagen2, connection=PooledConnection{channel=[id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443]}}, [request_sent])
2020-04-14 16:51:32.650 DEBUG 1 --- [ctor-http-nio-4] r.n.http.client.HttpClientOperations     : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Received response (auto-read:false) : [x-onap-requestid=ddd22db8-76c6-4a4f-8d01-ee4729cef690, Content-Type=application/json, Content-Length=3733, Date=Tue, 14 Apr 2020 16:51:32 GMT]
2020-04-14 16:51:32.650 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] onStateChange(GET{uri=/service_component_all/dcae-tcagen2, connection=PooledConnection{channel=[id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443]}}, [response_received])
2020-04-14 16:51:32.650 DEBUG 1 --- [ctor-http-nio-4] reactor.netty.channel.FluxReceive        : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Subscribing inbound receiver [pending: 0, cancelled:false, inboundDone: false]
2020-04-14 16:51:32.651 DEBUG 1 --- [ctor-http-nio-4] o.o.d.s.s.r.s.a.http.RxHttpClient        : Response status: 200 OK
2020-04-14 16:51:32.651 DEBUG 1 --- [ctor-http-nio-4] r.n.http.client.HttpClientOperations     : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] Received last HTTP packet
2020-04-14 16:51:32.651  INFO 1 --- [ctor-http-nio-4] o.o.d.s.s.r.s.c.c.impl.CbsClientImpl     : Got successful response from Config Binding Service
2020-04-14 16:51:32.651 DEBUG 1 --- [ctor-http-nio-4] o.o.d.s.s.r.s.c.c.impl.CbsClientImpl     : CBS response: {"config":{"spring.data.mongodb.uri":"mongodb://dcae-mongohost/dcae-tcagen2","streams_subscribes":{"tca_handle_in":{"type":"message_router","dmaap_info":{"topic_url":"http://message-router:3904/events/unauthenticated.VES_MEASUREMENT_OUTPUT/"}}},"tca.enable_ecomp_logging":true,"tca.enable_abatement":true,"tca.aai.password":"DCAE","streams_subscribes.tca_handle_in.consumer_group":"cg1","streams_subscribes.tca_handle_in.polling.auto_adjusting.step_up":10000,"tca.aai.node_query_path":"aai/v11/search/nodes-query","streams_publishes":{"tca_handle_out":{"type":"message_router","dmaap_info":{"topic_url":"http://message-router:3904/events/unauthenticated.TCAGEN2_OUTPUT/"}}},"streams_subscribes.tca_handle_in.consumer_ids[1]":"c1","tca.aai.generic_vnf_path":"aai/v11/network/generic-vnfs/generic-vnf","streams_subscribes.tca_handle_in.polling.auto_adjusting.step_down":30000,"streams_subscribes.tca_handle_in.polling.auto_adjusting.max":60000,"tca.aai.username":"DCAE","streams_subscribes.tca_handle_in.polling.auto_adjusting.min":30000,"tca.aai.url":"http://aai.onap.svc.cluster.local","streams_subscribes.tca_handle_in.timeout":-1,"tca.aai.enable_enrichment":true,"tca.policy":"{\"domain\":\"measurementsForVfScaling\",\"metricsPerEventName\":[{\"eventName\":\"vFirewallBroadcastPackets\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"LESS_OR_EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":700,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"vLoadBalancer\",\"controlLoopSchemaType\":\"VM\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.vNicPerformanceArray[*].receivedTotalPacketsDelta\",\"thresholdValue\":300,\"direction\":\"GREATER_OR_EQUAL\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]},{\"eventName\":\"Measurement_vGMUX\",\"controlLoopSchemaType\":\"VNF\",\"policyScope\":\"DCAE\",\"policyName\":\"DCAE.Config_tca-hi-lo\",\"policyVersion\":\"v0.0.1\",\"thresholds\":[{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"EQUAL\",\"severity\":\"MAJOR\",\"closedLoopEventStatus\":\"ABATED\"},{\"closedLoopControlName\":\"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e\",\"version\":\"1.0.2\",\"fieldPath\":\"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value\",\"thresholdValue\":0,\"direction\":\"GREATER\",\"severity\":\"CRITICAL\",\"closedLoopEventStatus\":\"ONSET\"}]}]}","tca.processing_batch_size":10000,"streams_subscribes.tca_handle_in.polling.fixed_rate":0,"streams_subscribes.tca_handle_in.message_limit":50000,"service_calls":[],"streams_subscribes.tca_handle_in.consumer_ids[0]":"c0"}}
2020-04-14 16:51:32.652 DEBUG 1 --- [ctor-http-nio-4] r.n.resources.PooledConnectionProvider   : [id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443] onStateChange(GET{uri=/service_component_all/dcae-tcagen2, connection=PooledConnection{channel=[id: 0x1b86fb8d, L:/10.42.1.198:57454 - R:config-binding-service/10.43.165.135:10443]}}, [disconnecting])