Introduction and Flow

This page details how to execute the demo that was recorded on the 14th of April 2021 at the DCAE meeting. The demo illustrates the flow of the control loop participant architecture and how it will create and manage services. Specifically, this demo shows the use of the control loop runtime's commissioning and instantiation API to trigger the deployment of the PMSH policy through the policy participant and the instantiation of the PMSH service through the DCAE participant. The DCAE participant uses the existing CLAMP backend to send requests and relay responses to the other participants and the runtime. There are several references to the states of the control loop and the control loop elements in this demo. Detail on the specifics of these states can be found here: The ONAP Policy Framework. This demonstration is not to be considered a reflection of the final implemented flow but merely as an indication of the currently implemented progress. The flow of the demo is shown in the image below.

Prerequisites

Local environment running Ubuntu. Tested with Ubuntu 20.04
Access to a Kubernetes Cluster running an ONAP deployment (version 7 or 8).
Docker and Docker Compose installed locally.
Java 11.
Maven 3.6.3
A running CLAMP backend (connected to the Kubernetes cluster) with a local MariaDB database. This is running from commit ebc6907968050189688a0b8ccc0e0c42419a9202 on the master branch.
npm 7.9.0
node 10.9.0
Angular 11
Google Chrome
POSTMAN

Setup CLAMP and MariaDB

Alterations to CLAMP

Some files need to be added to CLAMP in order for the demo to run successfully. These files include configuration files for Springboot and scripts to pre-populate the database.

Add the PMSH template file to "extra/sql/dump/k8s-pmsh-bp.yaml". The file is available here k8s-pmsh-bp.yaml.
There should be an SQL script in "extra/sql/dump/poc-test-data.sql". The file is available here poc-test-data.sql.
The file at "extra/sql/dump/load-fake-data.sh" needs to be altered to refer to "poc-test-data.sql". The altered version is here load-fake-data.sh.
An SQL script responsible for creating DBs is located here: "extra/sql/bulkload/create-db.sql". This should be replaced with the altered version here create-db.sql.
For conveneince, one can alter the docker-compose file at "extra/docker/clamp/docker-compose.yml" to match the one provided here docker-compose.yml. This is not strictly needed but makes it easier to delete volumes during the docker container deployment.
The Springboot properties file at "src/main/resources/application-noaaf.properties" should be altered to reflect to addresses and ports of your own Kubernetes cluster. By default, this properties file does not point to any cluster. For our case, we altered our local "/etc/hosts" file to associate hostnames with ip addresses - we then manually entered the relevant NodePorts into the properties file. An example is here application-noaaf.properties.

Build CLAMP

Run the following from the command line in the CLAMP root directory.

mvn clean install

Or for a faster build and if you encounter build errors

mvn clean install -DskipITs

Run and Pre-populate MariaDB

cd extra/bin-for-dev
./start-db.sh test

This should start and pre-populate the database. You can check the database with a suitable client to make sure if you wish. You can make sure the DB is running with:

docker ps

That's all for the database.

Start the CLAMP Backend

cd extra/bin-for-dev
./start-backend.sh

Note that you may need to run this with sudo.

Setup and Run DMAAP Simulator

DMAAP is required for communication between the runtime and participants. Here we outline how to setup and run the simulator. This should be done on another terminal.

Clone, Build and Run DMAAP Simulator

git clone "ssh://YOUR_USER@gerrit.nordix.org:29418/onap/policy/models"
cd models
mvn clean install

Then to run it

cd models-sim/models-sim-dmaap
mvn exec:java -Dexec.mainClass=org.onap.policy.models.sim.dmaap.startstop.Main -Dexec.args="-c src/main/resources/etc/DefaultConfig.json"

Setup and Run Policy API

The Policy Api is required locally by the Policy Participant. Carry out these actions on another terminal.

Clone, Build and Run the Policy API

git clone "ssh://YOUR_USER@gerrit.nordix.org:29418/onap/policy/api"
cd api
mvn clean install

The PolicyAPIConfig.json file for running the policy API for the demo is present in the clamp tosca-poc-nordix branch. The new configuration has also been provided here PolicyAPIConfig.json.

Then we can run the Policy API.

cd main
mvn exec:java -Dexec.mainClass=org.onap.policy.api.main.startstop.Main -Dexec.args=" -c /YOUR_PATH_TO_CLAMP/clamp/tosca-controlloop/common/src/test/resources/demo/config/PolicyAPIConfig.json"

Setup and Run Participant Runtime

The participant runtime is where the deployment of PMSH will be triggered. It contains the commissioning and instantiation APIs referred to in the diagram above.

If you haven't already done so, get a fresh clone of the tosca-poc-nordix branch of CLAMP. These actions should be carried out in a new terminal

git clone -b tosca-poc-nordix "ssh://YOUR_USER@gerrit.nordix.org:29418/onap/policy/clamp"

Build the project from the "tosca-controlloop" folder

cd clamp/tosca-controlloop
mvn clean install

This builds the runtime but also build the participant modules that we will be running later.

Run the participant runtime.

cd runtime
mvn exec:java -Dexec.mainClass=org.onap.policy.clamp.controlloop.runtime.main.startstop.Main -Dexec.args=" -c ../common/src/test/resources/demo/config/RuntimeConfig.json"

This should now be running

Setup and Run Policy Participant

The policy participant interacts with the policy api to create policies and policy types. On a separate terminal, you should run the policy participant.

cd clamp/tosca-controlloop/participant/participant-impl/participant-impl-policy
mvn exec:java -Dexec.mainClass=org.onap.policy.clamp.controlloop.participant.policy.main.startstop.Main -Dexec.args=" -c ../../../common/src/test/resources/demo/config/PolicyParticipantConfig.json"

This should now be running.

Setup and Run the DCAE Participant

The DCAE participant will interact with DCAE through the CLAMP backend. Setup and run as below in a new terminal.

cd clamp/tosca-controlloop/participant/participant-impl/participant-impl-dcae
mvn exec:java  -Dexec.mainClass=org.onap.policy.clamp.controlloop.participant.dcae.main.startstop.Main -Dexec.args=" -c src/main/resources/config/DCAEParticipantConfig.json"

The DCAE Participant should now be running.

Setup and Start the Angular UI

A simple user interface has been designed to monitor the status of the control loops and the control loop elements. It interacts with the the runtime to get status information and display it to the user. Do the following in a new terminal.

git clone https://github.com/brunomilitzer/TOSCA_ControlLoop.git
cd TOSCA_ControlLoop
npm install

The "npm install" will install the dependencies of the Angular UI. Then we can run it.

ng serve

The UI should now be available at http://localhost:4200. However, to avoid CORS errors we will need to run Google Chrome with disabled security. The below command should be executed in a new terminal.

mkdir custom-chrome
cd custom-chrome
google-chrome --disable-web-security --user-data-dir="/YOUR_PATH/custom-chrome"

Then you can navigate to http://localhost:4200 in the new chrome window. The UI should be displayed.

Starting and Monitoring the Deployment

There is a POSTMAN collection that is used for sending calls to the commissioning and instantiation API. It should be present in the tosca-poc-nordix branch here: "tosca-controlloop/common/src/test/resources/demo/config/DEMO.postman_collection.json". Is is also provided here for convenience: DEMO.postman_collection.json. This collection should be imported into your local POSTMAN. Once that is done, we should able to start issuing calls to the running components.

First, we send the Tosca service template to the commissioning API. Find the below query in the POSTMAN collection and send. The response should be a 200OK.

Next, we can make a call to create an instantiation of the control loop and elements. A successful call will mean that we are ready to move the loops and elements into the PASSIVE state. Find the blow query in the POSTMAN collection and send.

This again should return a 200OK from the instantiation API. Before you move the control loop into the PASSIVE state, you can look at the result of the GET instantiation call provided in the collection. The output should be something like the below:

Control Loops Unitialized

{
    "controlLoopList": [
        {
            "definition": {
                "name": "org.onap.domain.pmsh.PMSHControlLoopDefinition",
                "version": "1.2.3"
            },
            "state": "UNINITIALISED",
            "orderedState": "UNINITIALISED",
            "elements": {
                "709c62b3-8918-41b9-a747-d21eb79c6c22": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.policy.controlloop.PolicyControlLoopParticipant",
                        "version": "2.3.1"
                    },
                    "participantId": {
                        "name": "NULL",
                        "version": "0.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "Operational Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                "709c62b3-8918-41b9-a747-d21eb79c6c21": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.policy.controlloop.PolicyControlLoopParticipant",
                        "version": "2.3.1"
                    },
                    "participantId": {
                        "name": "NULL",
                        "version": "0.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "Monitoring Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                "709c62b3-8918-41b9-a747-d21eb79c6c20": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c20",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_DCAEMicroservice",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.dcae.controlloop.DCAEMicroserviceControlLoopParticipant",
                        "version": "2.3.4"
                    },
                    "participantId": {
                        "name": "NULL",
                        "version": "0.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "DCAE Control Loop Element for the PMSH instance 0 control loop"
                }
            },
            "name": "PMSHInstance0",
            "version": "1.0.1",
            "description": "PMSH control loop instance 0"
        }
    ]
}

We can see from the output that the control loop elements are in UNINITIALISED state. At this stage, we should look at the Angular UI to see the status of the control loop and elements reflected. Example below. At this stage the loop and elements are grey, which indicates an UNITIALISED state. Again, we can navigate to http://localhost:4200 on our custom google chrome window to get this.

To trigger deployment of our PMSH instance, we need to set the control loop elements into the PASSIVE state. Find the PUT call below in the POSTMAN collection and send. Make sure to set the state in the body of the request to PASSIVE.

This call should return a 202 Accepted status. Once this call is made, we can make a GET call to the Instantiation endpoint to see the instantiated control loops. This call is included in the POSTMAN collection. If everything worked correctly, the states should be in passive and we should see the PMSH instance start to come up on the Kubernetes cluster. Lets look at the get:

Control Loop Elements Moved to PASSIVE state

{
    "controlLoopList": [
        {
            "definition": {
                "name": "org.onap.domain.pmsh.PMSHControlLoopDefinition",
                "version": "1.2.3"
            },
            "state": "UNINITIALISED2PASSIVE",
            "orderedState": "PASSIVE",
            "elements": {
                "709c62b3-8918-41b9-a747-d21eb79c6c22": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.policy.controlloop.PolicyControlLoopParticipant",
                        "version": "2.3.1"
                    },
                    "participantId": {
                        "name": "org.onap.PM_Policy",
                        "version": "1.0.0"
                    },
                    "state": "PASSIVE",
                    "orderedState": "PASSIVE",
                    "description": "Operational Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                "709c62b3-8918-41b9-a747-d21eb79c6c21": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.policy.controlloop.PolicyControlLoopParticipant",
                        "version": "2.3.1"
                    },
                    "participantId": {
                        "name": "org.onap.PM_Policy",
                        "version": "1.0.0"
                    },
                    "state": "PASSIVE",
                    "orderedState": "PASSIVE",
                    "description": "Monitoring Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                "709c62b3-8918-41b9-a747-d21eb79c6c20": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c20",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_DCAEMicroservice",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.dcae.controlloop.DCAEMicroserviceControlLoopParticipant",
                        "version": "2.3.4"
                    },
                    "participantId": {
                        "name": "NULL",
                        "version": "0.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "PASSIVE",
                    "description": "DCAE Control Loop Element for the PMSH instance 0 control loop"
                }
            },
            "name": "PMSHInstance0",
            "version": "1.0.1",
            "description": "PMSH control loop instance 0"
        }
    ]
}

If we look at each of the elements, we can see that the "state" and "orderedState" have been moved in to PASSIVE. All except for the DCAE element, which has state="UNINITIALISED" and orderedState="PASSIVE". That is because it takes time for the PMSH service to come up and report back that it has come up. We can check on the Kubernetes cluster to see how the install is progressing.

Then, we will see this change if we take a look at the GET instantiation call.

PASSIVE Control Loop

{
    "controlLoopList": [
        {
            "definition": {
                "name": "org.onap.domain.pmsh.PMSHControlLoopDefinition",
                "version": "1.2.3"
            },
            "state": "PASSIVE",
            "orderedState": "PASSIVE",
            "elements": {
                "709c62b3-8918-41b9-a747-d21eb79c6c22": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.policy.controlloop.PolicyControlLoopParticipant",
                        "version": "2.3.1"
                    },
                    "participantId": {
                        "name": "org.onap.PM_Policy",
                        "version": "1.0.0"
                    },
                    "state": "PASSIVE",
                    "orderedState": "PASSIVE",
                    "description": "Operational Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                "709c62b3-8918-41b9-a747-d21eb79c6c21": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.policy.controlloop.PolicyControlLoopParticipant",
                        "version": "2.3.1"
                    },
                    "participantId": {
                        "name": "org.onap.PM_Policy",
                        "version": "1.0.0"
                    },
                    "state": "PASSIVE",
                    "orderedState": "PASSIVE",
                    "description": "Monitoring Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                "709c62b3-8918-41b9-a747-d21eb79c6c20": {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c20",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_DCAEMicroservice",
                        "version": "1.2.3"
                    },
                    "participantType": {
                        "name": "org.onap.dcae.controlloop.DCAEMicroserviceControlLoopParticipant",
                        "version": "2.3.4"
                    },
                    "participantId": {
                        "name": "DCAEParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "PASSIVE",
                    "orderedState": "PASSIVE",
                    "description": "DCAE Control Loop Element for the PMSH instance 0 control loop"
                }
            },
            "name": "PMSHInstance0",
            "version": "1.0.1",
            "description": "PMSH control loop instance 0"
        }
    ]
}

Also notice that the control loop itself is gone to PASSIVE state, because all of its elements are in PASSIVE state. Once again, we can see these changes reflected in the Angular UI. The yellow state means PASSIVE.

Also, it is possible to retrieve statistics about the different control loop elements using the monitoring API. An example call for our instantiation is provided below.

This should return JSON of a "clElementStatistics" array. An example is provided below.

clElementStatistics Example

{
    "clElementStatistics": [
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
            "participantId": {
                "name": "org.onap.PM_Policy",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:51:18.026Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
            "participantId": {
                "name": "org.onap.PM_Policy",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:51:19.779Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
            "participantId": {
                "name": "org.onap.PM_Policy",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:51:20.439Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
            "participantId": {
                "name": "org.onap.PM_Policy",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:51:18.026Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
            "participantId": {
                "name": "org.onap.PM_Policy",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:51:19.779Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
            "participantId": {
                "name": "org.onap.PM_Policy",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:51:20.439Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c20",
            "participantId": {
                "name": "DCAEParticipant0",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:52:32.861Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        },
        {
            "id": "709c62b3-8918-41b9-a747-d21eb79c6c20",
            "participantId": {
                "name": "DCAEParticipant0",
                "version": "1.0.0"
            },
            "timeStamp": "2021-04-23T11:52:32.895Z",
            "controlLoopState": "PASSIVE",
            "clElementUptime": 0
        }
    ]
}

These statistics can also be accessed in the UI by clicking on an individual control loop element.

We can use POSTMAN to set the state of the loop and elements into state RUNNING in a similar way to before.

Make sure to include the "orderedState": "RUNNING" in the body of the request and then send. This should change all of the states into RUNNING, which should be reflected in the UI as below.

Summary

Setup of all components required for Tosca Control Loop-based PMSH instance deployment.
- CLAMP
- MariaDB
- DMAAP
- Policy API
- Control Loop Runtime
- DCAE and Policy Participants
- Angular UI
Used POSTMAN to:
- Send to tosca service template to commissioning endpoint.
- Created a control loop and control loop elements in JSON and sent to the instantion API.
- Change the state of the control loop and elements to triogger deployment.
Used the Angular UI to:
- Observe state changes in the control loop and elements.
- View the statistics associated with the control loop elements.

Accompanying Demo Video Link

https://web.microsoftstream.com/video/1cec563b-1ecb-43f1-aacf-2477f1c83992

Participant Full Pipeline PMSH Deployment Demo - How to run...