DCAE R6 SON-Handler (PCI Usecase)

Overview

SON-Handler application acts upon reception of  two types of events (Measurement and Fault events) and Neighbor list change notification from SDNR.

Fault events are sent whenever collisions/confusions found by the Ran-simulator due to pci changes and at the same time SDNR notification is also sent whenever there is change in pci value.Measurement events are periodically sent from Ran-simulator which gives the KPI information.

SON-Handler processes the events and calls OOF for global optimization. Then it determines the actions based on OOF response and writes to unauthenticated.DCAE_CL_OUTPUT for policy to take necessary actions. 

Blueprint/model/image

Βlueprint component file can be found in

https://git.onap.org/dcaegen2/platform/blueprints/tree/blueprints/k8s-sonhms.yaml?id=bd763c50872842b4147039531b93cdc3a10326d2

Blueprint inputs file can be found in

https://gerrit.onap.org/r/gitweb?p=dcaegen2/services/son-handler.git;a=blob;f=dpo/blueprints/k8s-sonhms-inputs.yaml;h=e417c01a7c0c1fc738cd154e3e3b5f9abbedf584;hb=refs/heads/master

Docker image metadata for component can be found in

https://nexus3.onap.org/repository/docker.snapshot/v2/onap/org.onap.dcaegen2.services.son-handler/manifests/2.0.0

Deployment Prerequisite/dependencies

SON-Handler service requires DMAAP and Policy components to be functional.

SON-hadler service requires  the following dmaap topics to be present in the running DMAAP instance :

1.PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO

2.unauthenticated.SEC_FAULT_OUTPUT

3.unauthenticated.SEC_MEASUREMENT_OUTPUT

4.DCAE_CL_RSP

Policy model required for SON-handler service should be created and pushed to policy component.

Steps for creating and pushing the policy model:

1.Login to PDP container

Login to PDP container
kubectl exec -ti --namespace onap policy-pdp-0 bash

2.Create policy model

Policy Creation
curl -k -v --user 'healthcheck:zb!XztG34' -X POST -H 'Content-Type:application/json' -H 'Accept:application/json' -d @policy_type.json https://policy-api:6969/policy/api/v1/policytypes curl -v -k --silent --user 'healthcheck:zb!XztG34' -X POST "https://policy-api:6969/policy/api/v1/policytypes/onap.policies.monitoring.docker.sonhandler.app/versions/1.0.0/policies" -H "Accept: application/json" -H "Content-Type: application/json" -d @son_policy.json

3.Push policy model

Push Policy
curl --silent -k --user 'healthcheck:zb!XztG34' -X POST "https://policy-pap:6969/policy/pap/v1/pdps/policies" -H "Accept: application/json" -H "Content-Type: application/json" -d @push.json

4.Verify config policy is present

Fetch policy
curl -v -k --silent --user 'healthcheck:zb!XztG34' -X GET "https://policy-api:6969/policy/api/v1/policytypes/onap.policies.monitoring.docker.sonhandler.app/versions/1.0.0/policies" -H "Accept: application/json"

ConfigDB set-up:

The steps to set up configDB can be found in the attachment below.

Deployment Steps

  • Copy sonhms-blueprint and inputs file to /blueprints directory in DCAE bootstrap POD

  • Log-in to the DCAE bootstrap POD's main container

  • validate and deploy component

    Deploy component

    cfy install -b sonhms -d sonhms -i <inputsFilePath> <blueprintFilePath>



  • undeploy

    Undeploy component

    cfy uninstall sonhms





  • delete blueprint

    Delete blueprint

    cfy blueprints delete sonhms



Validation

The status of service can be checked after the deployment using command "kubectl get pods --namespace onap |grep son"

Verify
ubuntu@k8s-dcae-1:~$ kubectl get pods --namespace onap | grep son dep-s37e79facdf9b4820a933852d56127cd9-dcae-sonhms-59897b86qmv5p 1/1 Running 0 5h

Logs can be checked for dmaap connectivity

Logs
05:50:40.444 [pool-1-thread-2] INFO c.a.n.c.c.impl.CambriaConsumerImpl - UEB GET /events/PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO/sonhms-cg/sonhms-cid 05:50:40.445 [pool-1-thread-2] INFO c.att.nsa.apiClient.http.HttpClient - GET http://message-router:3904/events/PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO/sonhms-cg/sonhms-cid (anonymous) 05:50:40.448 [pool-2-thread-2] INFO c.a.n.c.c.impl.CambriaConsumerImpl - UEB GET /events/unauthenticated.SEC_FAULT_OUTPUT/sonhms-cg/sonhms-cid 05:50:40.448 [pool-2-thread-2] INFO c.att.nsa.apiClient.http.HttpClient - GET http://message-router:3904/events/unauthenticated.SEC_FAULT_OUTPUT/sonhms-cg/sonhms-cid (anonymous) 05:50:40.450 [pool-3-thread-2] INFO c.a.n.c.c.impl.CambriaConsumerImpl - UEB GET /events/unauthenticated.SEC_MEASUREMENT_OUTPUT/sonhms-cg/sonhms-cid 05:50:40.450 [pool-3-thread-2] INFO c.att.nsa.apiClient.http.HttpClient - GET http://message-router:3904/events/unauthenticated.SEC_MEASUREMENT_OUTPUT/sonhms-cg/sonhms-cid (anonymous) 05:50:50.622 [pool-1-thread-2] INFO c.att.nsa.apiClient.http.HttpClient ---> HTTP/1.1 200 OK 05:50:50.622 [pool-3-thread-2] INFO c.att.nsa.apiClient.http.HttpClient ---> HTTP/1.1 200 OK 05:50:50.622 [pool-2-thread-2] INFO c.att.nsa.apiClient.http.HttpClient ---> HTTP/1.1 200 OK



Functional Tests

Following are the functional tests based on three input events - FM data, PM data and Neighbor-list-change

1.FM data

 



To post the above message to DMAAP topic :

Post FM event
curl -i -H "Content-Type: application/json" -X POST -d @Fault.json http://<DMAAPHost>:<DMAAPPort>/events/unauthenticated.SEC_FAULT_OUTPUT

Logs which shows the fault event being handled by MS

Logs
12:05:41.872 [Thread-7] INFO o.o.d.services.sonhms.MainThread - newFmNotification has come 12:05:42.294 [Thread-7] INFO o.o.d.s.s.FaultNotificationComponent - get fault notifications method 12:05:42.319 [Thread-7] INFO o.o.d.s.s.FaultNotificationComponent - Parsing FM notification 12:05:42.319 [Thread-7] INFO o.o.d.services.sonhms.MainThread - Buffered FM cell Chn0001 12:05:42.319 [Thread-7] INFO o.o.d.services.sonhms.MainThread - fmNotificationFaultEvent [event=Event [commonEventHeader=CommonEventHeader [version=4.0.1, vesEventListenerVersion=7.0.1, domain=fault, eventName=Fault_RansimAgent-Wipro_RanPCIProblem, eventId=fault000001, sequence=1, priority=High, reportingEntityId=de305d54-75b4-431b-adb2-eb6b9e546014, reportingEntityName=ncserver1, sourceId=cc305d54-75b4-431b-adb2-eb6b9e541234, sourceName=Chn0001, startEpochMicrosec=1451772223000000, lastEpochMicrosec=1451772403000000, timeZoneOffset=UTC-05:30, nfNamingCode=RansimAgent, nfVendorName=Wipro], faultFields=FaultFields [faultFieldsVersion=4.0, alarmCondition=RanPciCollisionConfusionOccurred, eventSourceType=other, specificProblem=Collision, eventSeverity=CRITICAL, vfStatus=Active, alarmAdditionalInformation=org.onap.dcaegen2.services.sonhms.AlarmAdditionalInformation@1c393144]]] 12:05:42.320 [Thread-7] INFO o.o.d.services.sonhms.MainThread - bufferedFMNotificationCells before staring timer [Chn0001] 12:05:42.320 [Thread-7] INFO o.o.d.services.sonhms.MainThread - Buffered FM cell [Chn0001] 12:05:42.320 [Thread-7] INFO o.o.d.services.sonhms.MainThread - newFmNotification has come 12:05:42.322 [Thread-7] INFO o.o.d.s.s.FaultNotificationComponent - get fault notifications method 12:05:47.321 [Thread-7] INFO o.o.d.services.sonhms.MainThread - FM handling difference > 5000 12:05:47.321 [Thread-7] INFO o.o.d.services.sonhms.MainThread - FM bufferedFMNotificationCells [FaultEvent [event=Event [commonEventHeader=CommonEventHeader [version=4.0.1, vesEventListenerVersion=7.0.1, domain=fault, eventName=Fault_RansimAgent-Wipro_RanPCIProblem, eventId=fault000001, sequence=1, priority=High, reportingEntityId=de305d54-75b4-431b-adb2-eb6b9e546014, reportingEntityName=ncserver1, sourceId=cc305d54-75b4-431b-adb2-eb6b9e541234, sourceName=Chn0001, startEpochMicrosec=1451772223000000, lastEpochMicrosec=1451772403000000, timeZoneOffset=UTC-05:30, nfNamingCode=RansimAgent, nfVendorName=Wipro], faultFields=FaultFields [faultFieldsVersion=4.0, alarmCondition=RanPciCollisionConfusionOccurred, eventSourceType=other, specificProblem=Collision, eventSeverity=CRITICAL, vfStatus=Active, alarmAdditionalInformation=org.onap.dcaegen2.services.sonhms.AlarmAdditionalInformation@1c393144]]]] 12:05:47.321 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - Handling Fault notification 12:05:47.321 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - fm notification [FaultEvent [event=Event [commonEventHeader=CommonEventHeader [version=4.0.1, vesEventListenerVersion=7.0.1, domain=fault, eventName=Fault_RansimAgent-Wipro_RanPCIProblem, eventId=fault000001, sequence=1, priority=High, reportingEntityId=de305d54-75b4-431b-adb2-eb6b9e546014, reportingEntityName=ncserver1, sourceId=cc305d54-75b4-431b-adb2-eb6b9e541234, sourceName=Chn0001, startEpochMicrosec=1451772223000000, lastEpochMicrosec=1451772403000000, timeZoneOffset=UTC-05:30, nfNamingCode=RansimAgent, nfVendorName=Wipro], faultFields=FaultFields [faultFieldsVersion=4.0, alarmCondition=RanPciCollisionConfusionOccurred, eventSourceType=other, specificProblem=Collision, eventSeverity=CRITICAL, vfStatus=Active, alarmAdditionalInformation=org.onap.dcaegen2.services.sonhms.AlarmAdditionalInformation@1c393144]]]] 12:05:47.497 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - Handle Unmatching cells for FM notificatins newCells[Chn0001] 12:05:47.497 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - Handle Unmatching cells for FM notificatins,collisionConfusionCount[2, 0]

To check the output message that has been posted to DCAE_CL_OUTPUT:

Get DCAE_CL_OUTPUT
curl http://<DMAAPHost>:<DMAAPPort>/events/DCAE_CL_OUTPUT/cg/cid

Output message from DCAE_CL_OUTPUT:

Response message
["{\"closedLoopControlName\":\"ControlLoop-vPCI-fb41f388-a5f2-11e8-98d0-529269fb1459\",\"closedLoopAlarmStart\":1556539638298,\"closedLoopEventClient\":\"microservice.PCI\",\"closedLoopEventStatus\":\"ONSET\",\"target\":\"generic-vnf.vnf-id\",\"from\":\"PCIMS\",\"version\":\"1.0.2\",\"payload\":\"{\\\"Configurations\\\":[{\\\"data\\\":{\\\"FAPService\\\":{\\\"alias\\\":\\\"Chn0001\\\",\\\"X0005b9Lte\\\":{\\\"phyCellIdInUse\\\":160,\\\"pnfName\\\":\\\"ncserver1\\\"},\\\"CellConfig\\\":{\\\"LTE\\\":{\\\"RAN\\\":{\\\"Common\\\":{\\\"CellIdentity\\\":\\\"Chn0001\\\"}}}}}}}]}\",\"target_type\":\"VNF\",\"requestID\":\"1d13e1c8-e296-4332-b3ec-4e8b7180e3f5\",\"AAI\":{\"generic-vnf.prov-status\":\"ACTIVE\",\"generic-vnf.is-closed-loop-disabled\":\"false\",\"generic-vnf.vnf-id\":\"ncserver1\"},\"Action\":\"ModifyConfig\"}","{\"closedLoopControlName\":\"ControlLoop-vPCI-fb41f388-a5f2-11e8-98d0-529269fb1459\",\"closedLoopAlarmStart\":1556539648633,\"closedLoopEventClient\":\"microservice.PCI\",\"closedLoopEventStatus\":\"ONSET\",\"target\":\"generic-vnf.vnf-id\",\"from\":\"PCIMS\",\"version\":\"1.0.2\",\"payload\":\"{\\\"Configurations\\\":[{\\\"data\\\":{\\\"FAPService\\\":{\\\"alias\\\":\\\"Chn0001\\\",\\\"CellConfig\\\":{\\\"LTE\\\":{\\\"RAN\\\":{\\\"Common\\\":{\\\"CellIdentity\\\":\\\"Chn0001\\\"},\\\"NeighborListInUse\\\":{\\\"LTECell\\\":[{\\\"PNFName\\\":\\\"ncserver1\\\",\\\"PLMNID\\\":\\\"NTWK005\\\",\\\"CID\\\":\\\"Chn0013\\\",\\\"PhyCellID\\\":1,\\\"Blacklisted\\\":\\\"true\\\"}],\\\"LTECellNumberOfEntries\\\":\\\"1\\\"}}}}}}}]}\",\"target_type\":\"VNF\",\"requestID\":\"11f9a3ef-ffd4-4cf3-8d53-a076f8a92b6e\",\"AAI\":{\"generic-vnf.prov-status\":\"ACTIVE\",\"generic-vnf.is-closed-loop-disabled\":\"false\",\"generic-vnf.vnf-id\":\"ncserver1\"},\"Action\":\"ModifyConfigANR\"}"]



2.PM data

To post the above message to DMAAP topic :

Post PM event
curl -i -H "Content-Type: application/json" -X POST -d @Measurements.json http://<DMAAPHost>:<DMAAPPort>/events/unauthenticated.SEC_MEASUREMENT_OUTPUT

Logs which shows the performance event being handled by MS

Logs
12:00:02.375 [pool-3-thread-6] INFO o.o.d.s.s.d.PmNotificationCallback - Performance notification org.onap.dcaegen2.services.sonhms.entity.PerformanceNotifications@124c7070 12:00:02.797 [pool-3-thread-6] INFO o.o.d.s.s.d.PmNotificationCallback - Performance notification org.onap.dcaegen2.services.sonhms.entity.PerformanceNotifications@581b43ba 12:00:03.235 [Thread-8] INFO o.o.d.services.sonhms.PmThread - New PM notification from Dmaap 12:00:03.396 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - bad neighbor Chn0014 12:00:03.396 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - not bad neighbor Chn0013 12:00:03.396 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - triggering policy to remove bad neighbors 12:00:03.398 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - payload : Payload [configuration=[org.onap.dcaegen2.services.sonhms.model.Configurations@5fa7afd6]] 12:00:03.420 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - Policy Notification: {"closedLoopControlName":"ControlLoop-vPCI-fb41f388-a5f2-11e8-98d0-529269fb1459","closedLoopAlarmStart":1556539203411,"closedLoopEventClient":"microservice.PCI","closedLoopEventStatus":"ONSET","target":"generic-vnf.vnf-id","from":"PCIMS","version":"1.0.2","payload":"{\"Configurations\":[{\"data\":{\"FAPService\":{\"alias\":\"Chn0001\",\"CellConfig\":{\"LTE\":{\"RAN\":{\"Common\":{\"CellIdentity\":\"Chn0001\"},\"NeighborListInUse\":{\"LTECell\":[{\"PNFName\":\"ncserver1\",\"PLMNID\":\"plmnid1\",\"CID\":\"Chn0014\",\"PhyCellID\":0,\"Blacklisted\":\"true\"}],\"LTECellNumberOfEntries\":\"1\"}}}}}}}]}","target_type":"VNF","requestID":"e8608ba4-3790-4c3e-8e77-9f0dd574afd4","AAI":{"generic-vnf.prov-status":"ACTIVE","generic-vnf.is-closed-loop-disabled":"false","generic-vnf.vnf-id":"ncserver1"},"Action":"ModifyConfigANR"} 12:00:03.438 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - send notification to policy result true 12:00:03.442 [Thread-8] INFO c.a.n.c.c.impl.CambriaConsumerImpl - UEB GET /events/DCAE_CL_RSP/sonhms-cg/sonhms-cid 12:00:03.443 [Thread-8] INFO c.att.nsa.apiClient.http.HttpClient - GET http://message-router:3904/events/DCAE_CL_RSP/sonhms-cg/sonhms-cid (anonymous) ... 12:00:03.537 [pool-6-thread-1] INFO c.a.n.c.c.i.CambriaSimplerBatchPublisher - sending 1 msgs to /events/DCAE_CL_OUTPUT. Oldest: 99 ms 12:00:03.538 [pool-6-thread-1] INFO c.att.nsa.apiClient.http.HttpClient - POST http://message-router:3904/events/DCAE_CL_OUTPUT (anonymous) ... 12:00:03.551 [pool-6-thread-1] INFO c.att.nsa.apiClient.http.HttpClient - --> HTTP/1.1 200 OK 12:00:03.551 [pool-6-thread-1] INFO c.a.n.c.c.i.CambriaSimplerBatchPublisher - cambria reply ok (13 ms):{"serverTimeMs":1,"count":1} 12:00:09.482 [pool-1-thread-10] INFO c.att.nsa.apiClient.http.HttpClient - --> HTTP/1.1 200 OK 12:00:09.482 [pool-2-thread-2] INFO c.att.nsa.apiClient.http.HttpClient - --> HTTP/1.1 200 OK 12:00:13.657 [Thread-8] INFO c.att.nsa.apiClient.http.HttpClient - --> HTTP/1.1 200 OK 12:00:13.657 [Thread-8] INFO o.o.d.s.s.dmaap.PolicyDmaapClient - no response from policy, timer expired 12:00:13.657 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - handled policy response 12:00:13.657 [Thread-8] INFO o.o.d.s.sonhms.PmNotificationHandler - Sent ANR update to policy true

To check the output message that has been posted to DCAE_CL_OUTPUT:

Get DCAE_CL_OUTPUT
curl http://<DMAAPHost>:<DMAAPPort>/events/DCAE_CL_OUTPUT/cg/cid

Output message from DCAE_CL_OUTPUT:

Response message
["{\"closedLoopControlName\":\"ControlLoop-vPCI-fb41f388-a5f2-11e8-98d0-529269fb1459\",\"closedLoopAlarmStart\":1556539382476,\"closedLoopEventClient\":\"microservice.PCI\",\"closedLoopEventStatus\":\"ONSET\",\"target\":\"generic-vnf.vnf-id\",\"from\":\"PCIMS\",\"version\":\"1.0.2\",\"payload\":\"{\\\"Configurations\\\":[{\\\"data\\\":{\\\"FAPService\\\":{\\\"alias\\\":\\\"Chn0001\\\",\\\"CellConfig\\\":{\\\"LTE\\\":{\\\"RAN\\\":{\\\"Common\\\":{\\\"CellIdentity\\\":\\\"Chn0001\\\"},\\\"NeighborListInUse\\\":{\\\"LTECell\\\":[{\\\"PNFName\\\":\\\"ncserver1\\\",\\\"PLMNID\\\":\\\"plmnid1\\\",\\\"CID\\\":\\\"Chn0014\\\",\\\"PhyCellID\\\":0,\\\"Blacklisted\\\":\\\"true\\\"}],\\\"LTECellNumberOfEntries\\\":\\\"1\\\"}}}}}}}]}\",\"target_type\":\"VNF\",\"requestID\":\"130779d1-2369-4e4e-af85-982a963efe6b\",\"AAI\":{\"generic-vnf.prov-status\":\"ACTIVE\",\"generic-vnf.is-closed-loop-disabled\":\"false\",\"generic-vnf.vnf-id\":\"ncserver1\"},\"Action\":\"ModifyConfigANR\"}"]



3.Neighbor-list change notification

To post the above message to DMAAP topic :

Post sdnr-notification
curl -i -H "Content-Type: application/json" -X POST -d @sdnr_notif.json http://<DMAAPHost>:<DMAAPPort>/events/PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO

Logs which shows the sdnr notification being handled by MS

Logs
12:09:41.557 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - Handling SDNR notification 12:09:41.561 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - handling matching cells 12:09:41.561 [Thread-7] INFO o.o.d.services.sonhms.EventHandler - handling unmatched cells 12:09:41.562 [Thread-7] INFO o.o.d.s.sonhms.utils.ClusterUtils - indide clusterMap 12:09:41.578 [Thread-7] INFO o.o.d.s.sonhms.utils.ClusterUtils - clusterMap{CellPciPair [cellId=Chn0025, physicalCellId=10]=[CellPciPair [cellId=Chn0011, physicalCellId=0], CellPciPair [cellId=Chn0012, physicalCellId=1], CellPciPair [cellId=Chn0017, physicalCellId=2], CellPciPair [cellId=Chn0018, physicalCellId=3], CellPciPair [cellId=Chn0019, physicalCellId=4], CellPciPair [cellId=Chn0020, physicalCellId=5], CellPciPair [cellId=Chn0021, physicalCellId=6], CellPciPair [cellId=Chn0022, physicalCellId=7], CellPciPair [cellId=Chn0023, physicalCellId=8], CellPciPair [cellId=Chn0024, physicalCellId=9], CellPciPair [cellId=Chn0033, physicalCellId=18]]} 12:09:41.624 [Thread-7] INFO o.o.d.s.sonhms.utils.ThreadUtils - Creating new child thread 12:09:41.633 [pool-4-thread-2] INFO o.o.d.s.sonhms.child.ChildThread - Starting child thread 12:09:41.633 [pool-4-thread-2] INFO o.o.d.s.sonhms.child.Detection - Finding collision and confusion count

To check the output message that has been posted to DCAE_CL_OUTPUT:

Get DCAE_CL_OUTPUT
curl http://<DMAAPHost>:<DMAAPPort>/events/DCAE_CL_OUTPUT/cg/cid

Output message from DCAE_CL_OUTPUT:

Response message
["{\"closedLoopControlName\":\"ControlLoop-vPCI-fb41f388-a5f2-11e8-98d0-529269fb1459\",\"closedLoopAlarmStart\":1556539872424,\"closedLoopEventClient\":\"microservice.PCI\",\"closedLoopEventStatus\":\"ONSET\",\"target\":\"generic-vnf.vnf-id\",\"from\":\"PCIMS\",\"version\":\"1.0.2\",\"payload\":\"{\\\"Configurations\\\":[{\\\"data\\\":{\\\"FAPService\\\":{\\\"alias\\\":\\\"Chn0025\\\",\\\"X0005b9Lte\\\":{\\\"phyCellIdInUse\\\":234,\\\"pnfName\\\":\\\"ncserver2\\\"},\\\"CellConfig\\\":{\\\"LTE\\\":{\\\"RAN\\\":{\\\"Common\\\":{\\\"CellIdentity\\\":\\\"Chn0025\\\"}}}}}}}]}\",\"target_type\":\"VNF\",\"requestID\":\"8e52364b-ed9d-4daa-8c8e-7ed55d061ef8\",\"AAI\":{\"generic-vnf.prov-status\":\"ACTIVE\",\"generic-vnf.is-closed-loop-disabled\":\"false\",\"generic-vnf.vnf-id\":\"ncserver2\"},\"Action\":\"ModifyConfig\"}"]