...

1. Running "ete stability72hr" Robot suite hourly.  This test suite verifies that ONAP can instantiate vDNS, vFWCL, and VVG.
2. Set up vFW Closed Loop to remain running, then check hourly that the closed loop functionality is still working.

Stability72hr Test Suite

From the Rancher VM, log in as root, and go to ~run:

/root/oom/kubernetes/robot/ete-k8s.sh onap stability72hr

Known issues

On the Wind River lab, openstack will occasionally fail to deploy the stacks due to error like "missing credentials".  This appears to be an infrastructure issue.

vFWCL Stability Testing

First, set up vFW Close Loop.  Assuming that you have a freshly deployed ONAP, follow the instructions below in sequence:

Create the demo models and customers:

# demo init
/root/oom/kubernetes/robot/demo-k8s.

...

Run 

./etesh onap init

ONAP by default does not pre-load any policies, but the robot test suite that creates the vFWCL VMs expects that sample policies already exist.  So, load the sample policies first:

# push sample vFWCL policies
PAP_POD=$(kubectl --namespace onap get pods | grep policy-pap | sed 's/ .*//')
kubectl exec -it $PAP_POD -n onap -c pap -- bash -c 'export PRELOAD_POLICIES=true; /tmp/policy-install/config/push-policies.sh'

Spin up the Demo vFWCL stacks and VMs:

# ete instantiateDemoVFWC
/root/oom/kubernetes/robot/ete-k8s.sh onap instantiateDemoVFWCL

This will create 3 VMs: FW, Sink, and PNG (Packet Generator).  These VMs will remain running for the duration of the stack.  Note the public IP address of the PNG; we will need this later.

The above test suite automatically creates the VMs using VID, does the SDNC Preload, updates the policy with the PNG model-invariant-id, and creates the APPC mount point.

Now we need to restart policy drools so that it will pick up the new policies:

# restart drools
kubectl delete pod dev-policy-drools-0 -n onap

The update policies could take 15 to 20 minutes to kick in.  So, wait a while:

# wait for policy to kick in
sleep 20m

Now, to verify that the vFW Closed Loop is working; be sure to pass in the public IP address of the Packet Generator as the variable $PNG_IP:

# demo vfwclosedloop
/root/oom/kubernetes/robot/demo-k8s.sh onap instantiatevfwclosedloop $PNG_IP

You can watch the packets by going to the Sink VM IP port 667 using a browser.

Known Issues

The VPP stack in the Packet Generator will freeze after running for about a day or so.  This can be fixed by rebooting the Packet Generator VM.  For long-running tests, it would be advisable to reboot the Packet Generator every 12 hours or so.