CCVPN Closed Loop Diagram
- SDC/CLAMP Portal design and activate policy.
- SDC/CLAMP config and activate the policy.
- SDC/CLAMP distribute the DCAE config.
- SDC/CLMAP distribute the alarm correlation rules to Holmes.
- 3rd party SOTN controller report link down alarm to DCAE
- DCAE will do data cleaning and filtering for the alarms
- DCAEk keep track the datas.
- Holmes do analysis for the alarms.
- Holmes notify the reroute event.
- Policy matching the reroute rules.
- Policy call SO to delete the old services and create the new services. For the creation flow, a variable route will be recalculated.
CLAMP
Currently collector’s are not deployed on-demand, they are pre-deployed in DCAE and used by all the mS and all control loops. So the
collector’s, currently, are not deployed via CLAMP and there is no cloudify blueprint available to do so anyway. The collector box in CLAMP
GUI is just a virtual box to show the control loop flow, in reality there is no configuration made by CLAMP for the collector (at least at this
stage of ONAP, it might change in the future).
This raises the following requirements driven by CCVPN use case towards to design of control loop/CLAMP to accomplish micro-service
deployed on demand at CL instantiation.
- Collector on demand per control loop instance
- Or, common APIs of collector for updating collector at run time per control loop instantiation
- One instance of Holmes engine per control loop
- Configurable policy/actor/recipe mapping at PK
DCAE
DCAE Diagram
The APIs between SOTN Controller and the alarm notification collector following IETF-RESTCONF/YANG-PUSH notification standard:
- RestConf Collector (RC)subscribes for remote failure alarm to SOTN Controller (SC)
- RC requests to set up a persistent connection with the 3rd party SC.
- As the connection is standing, SC pushes service route status data to the collector
- RC receives alarm data, converts it into JSON format and publishes on DMAAP with topic of ROUTE_ALARM_OUTPUT
- UVA consumes the alarm message
- UVA requests the RestConf2VES mapping from the mapper. As of CCVPN use case proposal, the RestConf2VES.xml is manually uploaded to the mapper.
- Mapper response back the mapping
- UVA converts json alarm into VES event
- UVA publishes the VES event on DMAAP for further correlation
APIs to SOTN Controller (Restconf)
Subscribe Notification
Functionality
Collector (Client) and SOTN Controller establishes subscription relationship.
Method
POST
Request-URL
/restconf/operations/ietf-subscribed-notifications:establish-subscription
Request-Body
{
"ietf-subscribed-notifications:input": {
"encoding": "encode-json"
}
}
Response-Body
{
"ietf-subscribed-notifications:output": {
"identifier": "1"
}
}
Establish Long Term Connectivity
Functionality
Collector establishes long term connectivity with SOTN Controller and the controller will continuously Push the subscribed notification over.
Method
GET
Request-URL
/restconf/streams/yang-push-json
Request-Body
Response-Body
{
"ietf-notification:notification": {
"eventTime": {eventTime},
"ietf-yang-push: push-change-update ": {
"subscription-id ": {subscription-id },
" datastore-changes ": {
"ietf-yang-patch:yang-patch": {
"patch-id": {patch-id},
"edit": [
{
"edit-id" : {edit-id },
"operation" : {operation },
"target" : {target },
"value": {value}
}
]
}
}
}
}
}
Restconf Notification
This is the Route/LinkId sttatus notification example from SOTN Controller.
"ietf-restconf:notification": {
"eventTime": "2018-07-28T09:15:03.924Z",
"ietf-yang-push:push-change-update": {
"subscription-id": 1,
"datastore-changes": {
"ietf-yang-patch:yang-patch": {
"patch-id": "d1d08ce8-b24d-4efb-a0e7-7b835642f2f1",
"edit": [
{
"edit-id": "0",
"operation": "merge",
"target": "/network=providerId%2F5555%2FclientId%2F6666%2FtopologyId%2F11/node=example-node/ietf-network-topology:termination-point=14",
"value": {
"ietf-network-topology:termination-point": [
{
"supporting-termination-point": [
{
"network-ref": "providerId/5555/clientId/6666/topologyId/33",
"node-ref": "node-ref-example",
"tp-ref": "33488898"
}
],
"ietf-eth-te-topology:svc": {
"client-facing": "true",
"supported-classification": {
"transparent": "true"
}
},
"ietf-te-topology:te": {
"admin-status": "up",
"interface-switching-capability": [
{
"encoding": "lsp-encoding-ethernet",
"max-lsp-bandwidth": [
{
"priority": "7",
"te-bandwidth": {
"ietf-eth-te-topology:eth-bandwidth": "1000000"
}
}
],
"switching-capability": "switching-l2sc"
}
],
"oper-status": "up",
"inter-domain-plug-id": "51000"
},
"ietf-te-topology:te-tp-id": "1",
"tp-id": "1"
}
]
}
}
]
}
}
}
}
Route Alarm Message
This is the message definition at output of Restconf collector to DMAAP with topic of RESTCONF_ALARM_TOPIC.
{
"notify_oid" : "example-mappingfile-id.x.1",
"ietf-restconf:notification": {
"eventTime": "2018-07-28T09:15:03.924Z",
"ietf-yang-push:push-change-update": {
"subscription-id": 1,
"datastore-changes": {
"ietf-yang-patch:yang-patch": {
"patch-id": "d1d08ce8-b24d-4efb-a0e7-7b835642f2f1",
"edit": [
{
"edit-id": "0",
"operation": "merge",
"target": "/network=providerId%2F5555%2FclientId%2F6666%2FtopologyId%2F11/node=example-node/ietf-network-topology:termination-point=14",
"value": {
"ietf-network-topology:termination-point": [
{
"supporting-termination-point": [
{
"network-ref": "providerId/5555/clientId/6666/topologyId/33",
"node-ref": "node-ref-example",
"tp-ref": "33488898"
}
],
"ietf-eth-te-topology:svc": {
"client-facing": "true",
"supported-classification": {
"transparent": "true"
}
},
"ietf-te-topology:te": {
"admin-status": "up",
"interface-switching-capability": [
{
"encoding": "lsp-encoding-ethernet",
"max-lsp-bandwidth": [
{
"priority": "7",
"te-bandwidth": {
"ietf-eth-te-topology:eth-bandwidth": "1000000"
}
}
],
"switching-capability": "switching-l2sc"
}
],
"oper-status": "up",
"inter-domain-plug-id": "51000"
},
"ietf-te-topology:te-tp-id": "1",
"tp-id": "1"
}
]
}
}
]
}
}
}
}
}
VES MAPPER
The Restconf2VES.xml could be uploaded to DCAE at the closed loop deployment from DCAE Designer. For now, it will be manually uploaded to the mapper at the closed loop instantiation.
Mapping XML
UVA subscribes RESTCONF_ALARM_TOPIC on DMAAP.
Route Status Alarm
{
"event": {
"commonEventHeader": {
"sourceId": "example-target", // 'target' from the restconf notification.
"startEpochMicrosec": 1413378172000000,
"eventId": "ab305d54-85b4-a31b-7db2-fb6b977766",
"sequence": 0,
"domain": "fault",
"lastEpochMicrosec": 1413378172000033,
"eventName": "Fault_Route_Status",
"sourceName": "example-target", // 'Target' from the restconf notification.
"priority": "High",
"version": 3.0,
"reportingEntityName": "Domain_Contorller"
},
"faultFields": {
"eventSeverity": "CRITICAL",
"alarmCondition": "Route_Down",
"faultFieldsVersion": 2.0,
"specificProblem": "Fault_SOTN_Service_Status",
"alarmAdditionalInformation": [
{
"name": "linkId",
"value": "example-linkId"
},
{
"name": "nodeId",
"value": "example-nodeId"
},
{
"name": "topologyId",
"value": "example-topologyId"
},
{
"name": "tp-id",
"value": "example-tp-id"
}
]
"eventSourceType": "other",
"vfStatus": "Active"
}
}
}
These alarms are published to the unauthenticated.SEC_FAULT_OUTPUT topic of DMaaP.
DCAE Blueprints
The initial/default configuration of restconf collector and Holmes are specified in the following micro-service blueprints.
Blueprint for Restconf Collector
Blueprint for Holmes Rule Manager
HOLMES (WIP)
CCVPN Close Loop requires Holmes to correlate route down alarms from SOTN Controllers from different sites. (Refer to this page on Holmes installation,)
In Phase 1, for the minimum, two of the above defined Route_Down_Alarm will be correlated within time window of 15 - 30, for instance, milliseconds. (Refer to this page as examples.)
Rule Creation
{
"ruleName": "CCVPN",
"loopControlName": "ControlLoop-CCVPN-2179b738-fd36-4843-a71a-a8c24c70c55b",
"description": "This rule is designed for the correlation analysis for the CCVPN use case.",
"content": "package org.onap.holmes.droolsRule;\n\nimport org.onap.holmes.common.dmaap.DmaapService;\nimport org.onap.holmes.common.api.stat.VesAlarm;\nimport org.onap.holmes.common.aai.CorrelationUtil;\nimport org.onap.holmes.common.dmaap.entity.PolicyMsg;\nimport org.onap.holmes.common.dropwizard.ioc.utils.ServiceLocatorHolder;\nimport org.onap.holmes.common.utils.DroolsLog;\n \n\nrule \"Relation_analysis_Rule\"\nsalience 200\nno-loop true\n when\n $root : VesAlarm(alarmIsCleared == 0,\n $sourceId: sourceId, sourceId != null && !sourceId.equals(\"\"),\n\t\t\t$sourceName: sourceName, sourceName != null && !sourceName.equals(\"\"),\n\t\t\t$startEpochMicrosec: startEpochMicrosec,\n specificProblem in (\"Fault_SOTN_Service_Failure\"),\n $eventId: eventId)\n $child : VesAlarm( eventId != $eventId, parentId == null,\n specificProblem in (\"Fault_SOTN_Service_Failure\"),\n startEpochMicrosec < $startEpochMicrosec + 60000 && startEpochMicrosec > $startEpochMicrosec - 60000 )\n then\n\t\t$child.setParentId($root.getEventId());\n\t\tupdate($child);\n\t\t\nend\n\nrule \"root_has_child_handle_Rule\"\nsalience 150\nno-loop true\n\twhen\n\t\t$root : VesAlarm(alarmIsCleared == 0, rootFlag == 0, $eventId: eventId)\n\t\t$child : VesAlarm(eventId != $eventId, parentId == $eventId)\n\tthen\n\t\tDmaapService dmaapService = ServiceLocatorHolder.getLocator().getService(DmaapService.class);\n\t\tPolicyMsg policyMsg = dmaapService.getPolicyMsg($root, $child, \"org.onap.holmes.droolsRule\");\n dmaapService.publishPolicyMsg(policyMsg, \"unauthenticated.DCAE_CL_OUTPUT\");\n\t\t$root.setRootFlag(1);\n\t\tupdate($root);\nend\n\nrule \"root_no_child_handle_Rule\"\nsalience 100\nno-loop true\n when\n $root : VesAlarm(alarmIsCleared == 0, rootFlag == 0,\n sourceId != null && !sourceId.equals(\"\"),\n\t\t\tsourceName != null && !sourceName.equals(\"\"),\n specificProblem in (\"Fault_SOTN_Service_Failure\"))\n then\n\t\tDmaapService dmaapService = ServiceLocatorHolder.getLocator().getService(DmaapService.class);\n\t\tPolicyMsg policyMsg = dmaapService.getPolicyMsg($root, null, \"org.onap.holmes.droolsRule\");\n dmaapService.publishPolicyMsg(policyMsg, \"unauthenticated.DCAE_CL_OUTPUT\");\n\t\t$root.setRootFlag(1);\n\t\tupdate($root);\nend\n\nrule \"root_cleared_handle_Rule\"\nsalience 100\nno-loop true\n when\n $root : VesAlarm(alarmIsCleared == 1, rootFlag == 1)\n then\n\t\tDmaapService dmaapService = ServiceLocatorHolder.getLocator().getService(DmaapService.class);\n\t\tPolicyMsg policyMsg = dmaapService.getPolicyMsg($root, null, \"org.onap.holmes.droolsRule\");\n dmaapService.publishPolicyMsg(policyMsg, \"unauthenticated.DCAE_CL_OUTPUT\");\n\t\tretract($root);\nend\n\nrule \"child_handle_Rule\"\nsalience 100\nno-loop true\n when\n $child : VesAlarm(alarmIsCleared == 1, rootFlag == 0)\n then\n\t\tretract($child);\nend",
"enabled": 1
}
To illustrate, the content field in the above rule is presented as readable format in the following. We need to convert it to valid json string when uploading to Holmes.
package org.onap.holmes.droolsRule;
import org.onap.holmes.common.dmaap.DmaapService;
import org.onap.holmes.common.api.stat.VesAlarm;
import org.onap.holmes.common.aai.CorrelationUtil;
import org.onap.holmes.common.dmaap.entity.PolicyMsg;
import org.onap.holmes.common.dropwizard.ioc.utils.ServiceLocatorHolder;
import org.onap.holmes.common.utils.DroolsLog;
rule "Relation_analysis_Rule"
salience 200
no-loop true
when
$root : VesAlarm(alarmIsCleared == 0,
$sourceId: sourceId, sourceId != null && !sourceId.equals(""),
$sourceName: sourceName, sourceName != null && !sourceName.equals(""),
$startEpochMicrosec: startEpochMicrosec,
specificProblem in ("Fault_SOTN_Service_Failure"),
$eventId: eventId)
$child : VesAlarm( eventId != $eventId, parentId == null,
specificProblem in ("Fault_SOTN_Service_Failure"),
startEpochMicrosec < $startEpochMicrosec + 60000 && startEpochMicrosec > $startEpochMicrosec - 60000 )
then
$child.setParentId($root.getEventId());
update($child);
end
rule "root_has_child_handle_Rule"
salience 150
no-loop true
when
$root : VesAlarm(alarmIsCleared == 0, rootFlag == 0, $eventId: eventId)
$child : VesAlarm(eventId != $eventId, parentId == $eventId)
then
DmaapService dmaapService = ServiceLocatorHolder.getLocator().getService(DmaapService.class);
PolicyMsg policyMsg = dmaapService.getPolicyMsg($root, $child, "org.onap.holmes.droolsRule");
dmaapService.publishPolicyMsg(policyMsg, "unauthenticated.DCAE_CL_OUTPUT");
$root.setRootFlag(1);
update($root);
end
rule "root_no_child_handle_Rule"
salience 100
no-loop true
when
$root : VesAlarm(alarmIsCleared == 0, rootFlag == 0,
sourceId != null && !sourceId.equals(""),
sourceName != null && !sourceName.equals(""),
specificProblem in ("Fault_SOTN_Service_Failure"))
then
DmaapService dmaapService = ServiceLocatorHolder.getLocator().getService(DmaapService.class);
PolicyMsg policyMsg = dmaapService.getPolicyMsg($root, null, "org.onap.holmes.droolsRule");
dmaapService.publishPolicyMsg(policyMsg, "unauthenticated.DCAE_CL_OUTPUT");
$root.setRootFlag(1);
update($root);
nend
rule "root_cleared_handle_Rule"
salience 100
no-loop true
when
$root : VesAlarm(alarmIsCleared == 1, rootFlag == 1)
then
DmaapService dmaapService = ServiceLocatorHolder.getLocator().getService(DmaapService.class);
PolicyMsg policyMsg = dmaapService.getPolicyMsg($root, null, "org.onap.holmes.droolsRule");
dmaapService.publishPolicyMsg(policyMsg, "unauthenticated.DCAE_CL_OUTPUT");
retract($root);
end
rule "child_handle_Rule"
salience 100
no-loop true
when
$child : VesAlarm(alarmIsCleared == 1, rootFlag == 0)
then
retract($child);
end
Rule Execution
After the correlation is done successfully, there should be a corresponding control loop event defined in the following section published on the unauthenticated.DCAE_CL_OUTPUT topic of DMaaP.
POLICY
Policy Creation
controlLoop:
version: 2.0.0
controlLoopName: ControlLoop-CCVPN-2179b738-fd36-4843-a71a-a8c24c70c55b
trigger_policy: unique-policy-id-16-deleteE2EService
timeout: 3600
abatement: false
policies:
- id: unique-policy-id-16-deleteE2EService
name: Delete E2E Service
description:
actor: SO
recipe: DeleteE2EService
target:
type: VM
retry: 3
timeout: 1200
success: unique-policy-id-17-createE2EService
failure: final_failure
failure_timeout: final_failure_timeout
failure_retries: final_failure_retries
failure_exception: final_failure_exception
failure_guard: final_failure_guard
- id: unique-policy-id-17-createE2EService
name: Create E2E Service
description:
actor: SO
recipe: CreateE2EService
target:
type: VM
retry: 3
timeout: 1200
success: final_success
failure: final_failure
failure_timeout: final_failure_timeout
failure_retries: final_failure_retries
failure_exception: final_failure_exception
failure_guard: final_failure_guard
{
"closedLoopEventClient": "DCAE.HolmesInstance",
"policyVersion": "1.0.0.5",
"policyName": "CCVPN",
"policyScope": "service=SOTNService,type=SampleType,closedLoopControlName=CL-CCVPN-d925ed73-8231-4d02-9545-db4e101f88f8",
"target_type": "VM",
"AAI": {
"service-instance.service-instance-id" : "TBD"
},
"closedLoopAlarmStart": 1484677482204798,
"closedLoopEventStatus": "ONSET",
"closedLoopControlName": "ControlLoop-CCVPN-2179b738-fd36-4843-a71a-a8c24c70c55b",
"version": "1.0.2",
"target": "vserver.vserver-name",
"requestID": "97964e10-686e-4790-8c45-bdfa61df770f",
"from": "DCAE"
}
Policy Engine subscribes the unauthenticated.DCAE_CL_OUTPUT on DMAAP.
AAI Enrichment APIs
DCAE VES event A&AI enrichment for previous use cases are defined here as reference. The following describe those supporting CCVPN Closed Loop,
URL: https://<AAI host>:<AAI port>/aai/v14/network/connectivities/connectivity/{connectivityId}
Method: Get
Response Body:
{
...
"access-ltp-id":"ltp id",
...
"relationship-list":{
"relationship" : [
{
"related-to" : "service-instance",
"related-link" : "url of service-instance",
"relationship-data": [
"relationship-key" : "service-instance.service-instance-id",
"relationship-value": "some id"
]
},
{
"related-to" : "",vpn-binding
"related-link" : "url of vpn-binding",
"relationship-data": [
"relationship-key" : "vpn-binding.vpn-binding-id",
"relationship-value": "some id"
]
}
]
}
}
For updating TP status.
URL: https://<AAI host>:<AAI port>/aai/v14/network/pnfs/pnf/{pnfName}/p-interfaces/p-interface/{tp-id}
Method: PUT
Request Body:
{
"interface-name":"{tp-id}",
"network-ref" : "some network",
"transparent" : "some value",
"speed-value" : "eth bandwidth",
"operational-status" : "some status",
"relationship-list" : {
"relationship" : [
{
"related-to" : "vpn-binding",
"related-link" : "url of vpn-binding",
"relationship-data" : [
"relationship-key" : "vpn-binding.vpn-id",
"relationship-value" : "some-id"
]
}
]
}
}
For query by TP & Status:
URL: https://<AAI host>:<AAI port>/aai/v14/network/pnfs/pnf/{pnfName}/p-interfaces/p-interface/{tp-id}
Method: GET
Request Body :
{
}
Response body:
{
"interface-name":"{tp-id}",
"network-ref" : "some network",
"transparent" : "some value",
"speed-value" : "eth bandwidth",
"operational-status" : "some status",
"relationship-list" : {
"relationship" : [
{
"related-to" : "vpn-binding",
"related-link" : "url of vpn-binding",
"relationship-data" : [
"relationship-key" : "vpn-binding.vpn-id",
"relationship-value" : "some-id"
]
}
]
}
}
URL: https://<AAI host>:<AAI port>/aai/v14/business/customers/customer/{global-custoner-id}/service-subscriptions/service-subscription/{service-type}/service-instances?service-instance-id={serviceId}
Method: GET
Request Body:
{
}
Response Body:
{
}
URL: https://<AAI host>:<AAI port>/aai/v14/network/logical-links?link-name={linkName}&operational-status={status}
Method: GET
{
}
Response Body:
{
"results": [
"logic-links" : {
"link-name" : "{linkName}",
"operational-status": "{status}",
"model-invariant-id": "some invariant",
"model-version-id" : "some version",
"link-id":"some id",
"relationship-list" : [
{
"relationship" : [
{
"related-to": "p-interface",
"related-link": "url of p-interface 1",
"relationship-data":[
"relationship-key" : "p-interface.interface-name",
"relationship-value" : "some name 1"
]
}
]
}
]
}
URL: https://<AAI host>:<AAI port>/aai/v14/network/pnfs/pnf/{pnfName}/p-interfaces?interface-name={ifName}&operational-status={status}
Method: GET
Resquest Body:
{
}
Response Body:
{
"results" : [
{
"p-interface" : {
"interface-name":{ifName}",
"network-ref": "some ref",
"transparent": "some blue",
"operational-status":"some speed",
"speed-value" : "some speed",
"relationship-list":
"relationship" : [
{
"related-to" : "logic-link",
"related-link" :'url of logical-link",
"relationship-data":[
"relationship-key" : "logical-link.link.name",
"relationship-value"; "some name"
]
},
{
"related-to" : "pnf",
"related-link" :'url of pnf",
"relationship-data":[
"relationship-key" : "pnf.pnf.name",
"relationship-value"; "some name"
]
}
]
}
]
}