DCAE R6 PMSH MicroService

Overview

The PM Subscription Handler (PMSH) is a micro service written in Python, which allows for the definition and unlocking of PM subscriptions on one or more network function (NF) instances.

Architecture Diagram

Artifacts

PMSH blueprint	PMSH image: 1.0.2

Deployment Prerequisite/dependencies

The following components must be up and running to successfully deploy PMSH
1. DCAE Platform
2. SDC
3. CLAMP
4. POLICY
5. A&AI (Should have at least 1 PNF registered)

Deployment Steps

In the Frankfurt release, there is a workaround required in order to deploy the PM Subscription Handler. The monitoring policy for the microservice needs to be manually inserted into CONSUL. This requires two steps:

1. The CONSUL service is no longer exposed in the Frankfurt release, this needs to be exposed in order to be accessible outside the k8s cluster.

Expose CONSUL service

kubectl expose svc -n onap consul-server --name=x-consul-server --type=NodePort

2. Insert the monitoring policy into CONSUL

PUT policy into CONSUL

curl -X PUT \
  http://<k8s-node-ip>:<consul-port>/v1/kv/dcae-pmsh:policy \
  -H 'Content-Type: application/json' \
  -d '{
    "subscription": {
        "subscriptionName": "test-subscription",
        "administrativeState": "LOCKED",
        "fileBasedGP": 15,
        "fileLocation": "/pm/pm.xml",
        "nfFilter": {
            "swVersions": [
                "1.0.0"
            ],
            "nfNames": [
                "^pnf.*"
            ]
        },
        "measurementGroups": [{
                "measurementGroup": {
                    "measurementTypes": [{
                            "measurementType": "countera"
                        },
                        {
                            "measurementType": "counterb"
                        }
                    ],
                    "managedObjectDNsBasic": [{
                            "DN": "dna"
                        },
                        {
                            "DN": "dnb"
                        }
                    ]
                }
            },
            {
                "measurementGroup": {
                    "measurementTypes": [{
                            "measurementType": "counterc"
                        },
                        {
                            "measurementType": "counterd"
                        }
                    ],
                    "managedObjectDNsBasic": [{
                            "DN": "dnc"
                        },
                        {
                            "DN": "dnd"
                        }
                    ]
                }
            }
        ]
    }
}'

Deployment of PMSH can then be done using Dashboard UI/Cloudify UI or via CLI. Below steps are based on CLI.

Transfer blueprint component file in DCAE bootstrap POD under /blueprints directory
Transfer blueprint component inputs file in DCAE bootstrap POD under / directory
Log-in to the DCAE bootstrap POD's main container

Validate blueprint

Validate Blueprint

cfy blueprints validate /blueprints/k8s-pmsh.yaml

Verify Plugin versions in target Cloudify instance match to blueprint imports
Verify Plugin version
```
cfy plugins list
```
If the version of plugin used are different, update the blueprint import to match.

Deploy Service

Upload and deploy blueprint

cfy install -b pmsh -d pmsh -i /k8s-pmsh-inputs.yaml /blueprints/k8s-pmsh.yaml

To un-deploy

Uninstall running component and delete deployment
Uninstall component
```
cfy uninstall pmsh
```
Delete blueprint
Delete blueprint
```
cfy blueprints delete pmsh
```

Initial Validation

After deployment, verify if PMSH pod is running correctly

Verify Heartbeat is running

root@k8s-rancher:~# kubectl get pods -n onap | egrep "pmsh"

dep-dcae-pmsh-6b58cbc7f5-lm92g                                2/2     Running            0          149m

We can curl message router to verify that PMSH has published an event to it's configured topic. By default this is unauthenticated.DCAE_CL_OUTPUT

Get from MR

curl -k https://message-router:3905/events/unauthenticated.DCAE_CL_OUTPUT/1/1

If PMSH is working correctly we should see an event published to message router

Response from MR

["{\"nfName\":\"pnf500\",\"policyName\":\"pmsh-operational-policy\",\"changeType\":\"CREATE\",\"subscription\":{\"administrativeState\":\"LOCKED\",\"subscriptionName\":\"test-subscription\",\"measurementGroups\":[{\"measurementGroup\":{\"measurementTypes\":[{\"measurementType\":\"countera\"},{\"measurementType\":\"counterb\"}],\"managedObjectDNsBasic\":[{\"DN\":\"dna\"},{\"DN\":\"dnb\"}]}},{\"measurementGroup\":{\"measurementTypes\":[{\"measurementType\":\"counterc\"},{\"measurementType\":\"counterd\"}],\"managedObjectDNsBasic\":[{\"DN\":\"dnc\"},{\"DN\":\"dnd\"}]}}],\"fileBasedGP\":15,\"fileLocation\":\"/pm/pm.xml\"},\"closedLoopControlName\":\"pmsh-control-loop\"}"]

This can also be verified by checking the logs to verify PMSH can connect to A&AI and DMaaP

Verify PMSH logs

pmsh@dcae-pmsh:/opt/app/pmsh$ tail -f /var/log/ONAP/dcaegen2/services/pmsh/debug.log 
2020-03-11T08:40:12.363092|PMSH Configuration from Configbinding Service: {'config': {'aaf_identity': 'dcae@dcae.onap.org', 'control_loop_name': 'pmsh-control-loop', 'operational_policy_name': 'pmsh-operational-policy', 'key_path': '/opt/app/pmsh/etc/certs/key.pem', 'streams_publishes': {'policy_pm_publisher': {'type': 'message_router', 'dmaap_info': {'topic_url': 'https://message-router:3905/events/unauthenticated.DCAE_CL_OUTPUT'}}}, 'cert_path': '/opt/app/pmsh/etc/certs/cert.pem', 'streams_subscribes': {'policy_pm_subscriber': {'type': 'message_router', 'dmaap_info': {'topic_url': 'https://message-router:3905/events/unauthenticated.PMSH_CL_INPUT'}}, 'aai_subscriber': {'type': 'message_router', 'dmaap_info': {'topic_url': 'https://message-router:3905/events/AAI-EVENT'}}}, 'aaf_password': 'demo123456!', 'ca_cert_path': '/opt/app/pmsh/etc/certs/cacert.pem'}, 'policy': {'subscription': {'subscriptionName': 'test-subscription', 'administrativeState': 'LOCKED', 'fileBasedGP': 15, 'fileLocation': '/pm/pm.xml', 'nfFilter': {'swVersions': ['1.0.0', '1.0.1'], 'nfNames': ['^pnf.*', '^vnf.*']}, 'measurementGroups': [{'measurementGroup': {'measurementTypes': [{'measurementType': 'countera'}, {'measurementType': 'counterb'}], 'managedObjectDNsBasic': [{'DN': 'dna'}, {'DN': 'dnb'}]}}, {'measurementGroup': {'measurementTypes': [{'measurementType': 'counterc'}, {'measurementType': 'counterd'}], 'managedObjectDNsBasic': [{'DN': 'dnc'}, {'DN': 'dnd'}]}}]}}}
2020-03-11T08:40:12.363617|Administrative State did not change in the Config
2020-03-11T08:40:42.432892|PMSH Configuration from Configbinding Service: {'config': {'aaf_identity': 'dcae@dcae.onap.org', 'control_loop_name': 'pmsh-control-loop', 'operational_policy_name': 'pmsh-operational-policy', 'key_path': '/opt/app/pmsh/etc/certs/key.pem', 'streams_publishes': {'policy_pm_publisher': {'type': 'message_router', 'dmaap_info': {'topic_url': 'https://message-router:3905/events/unauthenticated.DCAE_CL_OUTPUT'}}}, 'cert_path': '/opt/app/pmsh/etc/certs/cert.pem', 'streams_subscribes': {'policy_pm_subscriber': {'type': 'message_router', 'dmaap_info': {'topic_url': 'https://message-router:3905/events/unauthenticated.PMSH_CL_INPUT'}}, 'aai_subscriber': {'type': 'message_router', 'dmaap_info': {'topic_url': 'https://message-router:3905/events/AAI-EVENT'}}}, 'aaf_password': 'demo123456!', 'ca_cert_path': '/opt/app/pmsh/etc/certs/cacert.pem'}, 'policy': {'subscription': {'subscriptionName': 'test-subscription', 'administrativeState': 'UNLOCKED', 'fileBasedGP': 15, 'fileLocation': '/pm/pm.xml', 'nfFilter': {'swVersions': ['1.0.0', '1.0.1'], 'nfNames': ['^pnf.*', '^vnf.*']}, 'measurementGroups': [{'measurementGroup': {'measurementTypes': [{'measurementType': 'countera'}, {'measurementType': 'counterb'}], 'managedObjectDNsBasic': [{'DN': 'dna'}, {'DN': 'dnb'}]}}, {'measurementGroup': {'measurementTypes': [{'measurementType': 'counterc'}, {'measurementType': 'counterd'}], 'managedObjectDNsBasic': [{'DN': 'dnc'}, {'DN': 'dnd'}]}}]}}}
2020-03-11T08:40:42.433357|Administrative State changed from "LOCKED" "to "UNLOCKED".
2020-03-11T08:40:42.941068|Publishing Event to Activate Sub: test-subscription for the nf: pnf500
2020-03-11T08:40:42.998828|Adding network functions to subscription test-subscription
2020-03-11T08:40:43.088201|NetworkFunctionModel: pnf500, Active
2020-03-11T08:40:43.180609|Listening to AAI-EVENT topic in MR.
2020-03-11T08:40:47.321604|Request sent to MR topic: https://message-router:3905/events/unauthenticated.PMSH_CL_INPUT

Functional tests

Given that the steps above were used to configure the PMSH monitoring policy with administrativeState = LOCKED. The subscription has not yet been "activated" on the network function.

Activate an inactive subscription

If the subscription is in a LOCKED state (administrativeState=LOCKED) we can reconfigure PMSH to activate the subscription:

Activate Subscription

curl -X PUT \
  http://<k8s-node-ip>:<consul-port>/v1/kv/dcae-pmsh:policy \
  -H 'Content-Type: application/json' \
  -d '{
    "subscription": {
        "subscriptionName": "test-subscription",
        "administrativeState": "UNLOCKED",
        "fileBasedGP": 15,
        "fileLocation": "/pm/pm.xml",
        "nfFilter": {
            "swVersions": [
                "1.0.0"
            ],
            "nfNames": [
                "^pnf.*"
            ]
        },
        "measurementGroups": [{
                "measurementGroup": {
                    "measurementTypes": [{
                            "measurementType": "countera"
                        },
                        {
                            "measurementType": "counterb"
                        }
                    ],
                    "managedObjectDNsBasic": [{
                            "DN": "dna"
                        },
                        {
                            "DN": "dnb"
                        }
                    ]
                }
            },
            {
                "measurementGroup": {
                    "measurementTypes": [{
                            "measurementType": "counterc"
                        },
                        {
                            "measurementType": "counterd"
                        }
                    ],
                    "managedObjectDNsBasic": [{
                            "DN": "dnc"
                        },
                        {
                            "DN": "dnd"
                        }
                    ]
                }
            }
        ]
    }
}'

We should then be able to see a CREATE event sent to message router:

Get from MR

curl -k https://message-router:3905/events/unauthenticated.DCAE_CL_OUTPUT/1/1

If PMSH is working correctly we should see a CREATE event sent to message router:

Response from MR

["{\"nfName\":\"pnf500\",\"policyName\":\"pmsh-operational-policy\",\"changeType\":\"CREATE\",\"subscription\":{\"administrativeState\":\"LOCKED\",\"subscriptionName\":\"test-subscription\",\"measurementGroups\":[{\"measurementGroup\":{\"measurementTypes\":[{\"measurementType\":\"countera\"},{\"measurementType\":\"counterb\"}],\"managedObjectDNsBasic\":[{\"DN\":\"dna\"},{\"DN\":\"dnb\"}]}},{\"measurementGroup\":{\"measurementTypes\":[{\"measurementType\":\"counterc\"},{\"measurementType\":\"counterd\"}],\"managedObjectDNsBasic\":[{\"DN\":\"dnc\"},{\"DN\":\"dnd\"}]}}],\"fileBasedGP\":15,\"fileLocation\":\"/pm/pm.xml\"},\"closedLoopControlName\":\"pmsh-control-loop\"}"]

Deactivate an active subscription

If the subscription is in an UNLOCKED state (administrativeState=UNLOCKED) we can reconfigure PMSH to deactivate the subscription:

Deactivate Subscription

curl -X PUT \
  http://<k8s-node-ip>:<consul-port>/v1/kv/dcae-pmsh:policy \
  -H 'Content-Type: application/json' \
  -d '{
    "subscription": {
        "subscriptionName": "test-subscription",
        "administrativeState": "LOCKED",
        "fileBasedGP": 15,
        "fileLocation": "/pm/pm.xml",
        "nfFilter": {
            "swVersions": [
                "1.0.0"
            ],
            "nfNames": [
                "^pnf.*"
            ]
        },
        "measurementGroups": [{
                "measurementGroup": {
                    "measurementTypes": [{
                            "measurementType": "countera"
                        },
                        {
                            "measurementType": "counterb"
                        }
                    ],
                    "managedObjectDNsBasic": [{
                            "DN": "dna"
                        },
                        {
                            "DN": "dnb"
                        }
                    ]
                }
            },
            {
                "measurementGroup": {
                    "measurementTypes": [{
                            "measurementType": "counterc"
                        },
                        {
                            "measurementType": "counterd"
                        }
                    ],
                    "managedObjectDNsBasic": [{
                            "DN": "dnc"
                        },
                        {
                            "DN": "dnd"
                        }
                    ]
                }
            }
        ]
    }
}'

We should then be able to see a DELETE event sent to message router:

Get from MR

curl -k https://message-router:3905/events/unauthenticated.DCAE_CL_OUTPUT/1/1

If PMSH is working correctly we should see a DELETE event sent to message router:

Response from MR

["{\"nfName\":\"pnf500\",\"policyName\":\"pmsh-operational-policy\",\"changeType\":\"DELETE\",\"subscription\":{\"administrativeState\":\"LOCKED\",\"subscriptionName\":\"test-subscription\",\"measurementGroups\":[{\"measurementGroup\":{\"measurementTypes\":[{\"measurementType\":\"countera\"},{\"measurementType\":\"counterb\"}],\"managedObjectDNsBasic\":[{\"DN\":\"dna\"},{\"DN\":\"dnb\"}]}},{\"measurementGroup\":{\"measurementTypes\":[{\"measurementType\":\"counterc\"},{\"measurementType\":\"counterd\"}],\"managedObjectDNsBasic\":[{\"DN\":\"dnc\"},{\"DN\":\"dnd\"}]}}],\"fileBasedGP\":15,\"fileLocation\":\"/pm/pm.xml\"},\"closedLoopControlName\":\"pmsh-control-loop\"}"]

Dynamic Configuration Update

As the PMSH service periodically polls Consul KV using configbindingService API's - the run time configuration of PMSH service can be updated dynamically without having to redeploy/restart the service. The updates to the configuration can be triggered either from Policy (or CLAMP) or made directly in Consul.

Locate the service name by executing into PMSH Service pod and getting env HOSTNAME value

ServiceName

root@k8s-rancher:~# kubectl exec -it -n onap dep-s78f36f2daf0843518f2e25184769eb8b-dcae-pmsh-servithzx2 /bin/bash
Defaulting container name to s78f36f2daf0843518f2e25184769eb8b-dcae-pmsh-service.
Use 'kubectl describe pod/dep-s78f36f2daf0843518f2e25184769eb8b-dcae-pmsh-servithzx2 -n onap' to see all of the containers in this pod.

misshtbt@s78f36f2daf0843518f2e25184769eb8b-dcae-pmsh-service:~/bin$ env | grep HOSTNAME
HOSTNAME=s78f36f2daf0843518f2e25184769eb8b-dcae-pmsh-service

Change the configuration for Service in KV-store through UI

Consul URL

http://<k8s-node-ip>:<consul-ip>/ui/#/dc1/kv/

Consul Snapshot <>