Integration Stability Testing has two main components:
- Running "ete stability72hr" Robot suite hourly. This test suite verifies that ONAP can instantiate vDNS, vFWCL, and VVG.
- Set up vFW Closed Loop to remain running, then check hourly that the closed loop functionality is still working.
Stability72hr Test Suite
From the Rancher VM, log in as root, and run:
/root/oom/kubernetes/robot/ete-k8s.sh onap stability72hr
Known issues
On the Wind River lab, openstack will occasionally fail to deploy the stacks due to error like "missing credentials". This appears to be an infrastructure issue.
vFWCL Stability Testing
First, set up vFW Close Loop. Assuming that you have a freshly deployed ONAP, follow the instructions below in sequence:
Create the demo models and customers:
# demo init /root/oom/kubernetes/robot/demo-k8s.sh onap init
ONAP by default does not pre-load any policies, but the robot test suite that creates the vFWCL VMs expects that sample policies already exist. So, load the sample policies first:
# push sample vFWCL policies PAP_POD=$(kubectl --namespace onap get pods | grep policy-pap | sed 's/ .*//') kubectl exec -it $PAP_POD -n onap -c pap -- bash -c 'export PRELOAD_POLICIES=true; /tmp/policy-install/config/push-policies.sh'
Spin up the Demo vFWCL stacks and VMs:
# ete instantiateDemoVFWC /root/oom/kubernetes/robot/ete-k8s.sh onap instantiateDemoVFWCL
This will create 3 VMs: FW, Sink, and PNG (Packet Generator). These VMs will remain running for the duration of the stack. Note the public IP address of the PNG; we will need this later.
The above test suite automatically creates the VMs using VID, does the SDNC Preload, updates the policy with the PNG model-invariant-id, and creates the APPC mount point.
Now we need to restart policy drools so that it will pick up the new policies:
# restart drools kubectl exec -it dev-policy-drools-0 -- bash -c 'source /opt/app/policy/etc/profile.d/env.sh; policy stop' kubectl exec -it dev-policy-drools-0 -- bash -c 'source /opt/app/policy/etc/profile.d/env.sh; policy start'
The update policies could take 15 to 20 minutes to kick in. So, wait a while:
# wait for policy to kick in sleep 20m
Now, to verify that the vFW Closed Loop is working; be sure to pass in the public IP address as the variable $PNG_IP:
# demo vfwclosedloop /root/oom/kubernetes/robot/demo-k8s.sh onap vfwclosedloop $PNG_IP
You can watch the packets by going to the Sink VM IP port 667 using a browser.
Known Issues
The VPP stack in the Packet Generator will freeze after running for about a day or so. This can be fixed by rebooting the Packet Generator VM. For long-running tests, it would be advisable to reboot the Packet Generator every 12 hours or so.