APPC Generic_Restart Test

1 Verify VNF Data in A&AI
- 1.1 Typical Output of verify.sh
2 Reset Data
- 2.1 Output of verify.sh - VNF in "Error" status
3 Update APPC DB
- 3.1 VNF_STATE_MANAGEMENT List
4 Login to API Doc Explorer
5 Create VNF Restart Request
6 Submit Your Request
7 Look for Request Accepted
8 Monitor the Karaf Log
9 Monitor SvcLogic Log
10 Look for Success in SvcLogic or Karaf Log
11 Verify Restart in Cloud Dashboard

Verify VNF Data in A&AI

Using the verify.sh script validate the VNF data is in A&AI as well as the Orchestration Status of the VNF (Should be Running).
More details are available at Verify VNF Data is in A&AI.

Typical Output of verify.sh

{
    "inventory-response-item": [
        {
            "extra-properties": {},
            "generic-vnf": {
                "in-maint": false,
                "is-closed-loop-disabled": false,
                "model-invariant-id": "vCPE_Intrastructure_Metro_vGMUX",
                "model-version-id": "vCPE",
                "orchestration-status": "Running",
                "resource-version": "1510104391195",
                "service-id": "vFirewall_demo_app",
                "vnf-id": "vCPE_Infrastructure_vGMUX_demo_app",
                "vnf-name": "vCPE_Infrastructure_vGMUX_demo_app",
                "vnf-type": "vCPE"
            },
            "inventory-response-items": {
                "inventory-response-item": [
                    {
                        "extra-properties": {},
                        "inventory-response-items": {
                            "inventory-response-item": [
                                {
                                    "extra-properties": {},
                                    "vnfc": {
                                        "in-maint": false,
                                        "is-closed-loop-disabled": false,
                                        "nfc-function": "vGMUX",
                                        "nfc-naming-code": "vGMUXCODE",
                                        "orchestration-status": "Running",
                                        "prov-status": "Running",
                                        "resource-version": "1509725780164",
                                        "vnfc-name": "VNFC_vGMUX_demo_app"
                                    }
                                },
                                {
                                    "extra-properties": {},
                                    "inventory-response-items": {
                                        "inventory-response-item": [
                                            {
                                                "cloud-region": {
                                                    "cloud-owner": "pod25",
                                                    "cloud-region-id": "RegionOne",
                                                    "cloud-region-version": "titanium_cloud",
                                                    "cloud-type": "openstack",
                                                    "cloud-zone": "cloud zone",
                                                    "complex-name": "complex name",
                                                    "identity-url": "http://10.0.14.1:9005/api/multicloud-titanium_cloud/v0/pod25_RegionOne/identity/v2.0",
                                                    "owner-defined-type": "owner-defined-type",
                                                    "resource-version": "1509725696928",
                                                    "sriov-automation": true
                                                },
                                                "extra-properties": {},
                                                "inventory-response-items": {
                                                    "inventory-response-item": [
                                                        {
                                                            "esr-system-info": {
                                                                "cloud-domain": "Default",
                                                                "default-tenant": "Integration",
                                                                "esr-system-info-id": "432ac032-e996-41f2-84ed-9c7a1766eb29",
                                                                "ip-address": "example-ip-address-val-44431",
                                                                "password": "onapdemo",
                                                                "port": "example-port-val-93234",
                                                                "resource-version": "1509725697438",
                                                                "service-url": "http://10.12.25.2:5000/v3",
                                                                "ssl-cacert": "example-ssl-cacert-val-75021",
                                                                "ssl-insecure": true,
                                                                "system-name": "example-system-name-val-29070",
                                                                "system-type": "VIM",
                                                                "type": "example-type-val-85254",
                                                                "user-name": "demo",
                                                                "vendor": "example-vendor-val-94515",
                                                                "version": "example-version-val-71880"
                                                            },
                                                            "extra-properties": {}
                                                        }
                                                    ]
                                                }
                                            }
                                        ]
                                    },
                                    "tenant": {
                                        "resource-version": "1509725696999",
                                        "tenant-id": "466979b815b5415ba14ada713e6e1846",
                                        "tenant-name": "Integration"
                                    }
                                }
                            ]
                        },
                        "vserver": {
                            "in-maint": false,
                            "is-closed-loop-disabled": false,
                            "prov-status": "Running",
                            "resource-version": "1509725717054",
                            "vserver-id": "e8020a03-5cd9-4f8c-b2d8-9972a256cf6a",
                            "vserver-name": "zdcpe1cpe01mux01",
                            "vserver-name2": "zdcpe1cpe01mux01",
                            "vserver-selflink": "http://10.0.14.1:80/api/multicloud-titanium_cloud/v0/pod25_RegionOne/compute/v2.1/466979b815b5415ba14ada713e6e1846/servers/e8020a03-5cd9-4f8c-b2d8-9972a256cf6a"
                        }
                    }
                ]
            },
            "model-name": "vCPE"
        }
    ]
}

Reset Data

Update the VNF Orchestration Status using the update.sh script.
More details are available at Resetting A&AI VNF Orchestration Status.
If the VNF Orchestration Status is "Error" it will need to be reset following the process in the link above for resetting the orchestration status.

Output of verify.sh - VNF in "Error" status

{
    "inventory-response-item": [
        {
            "extra-properties": {},
            "generic-vnf": {
                "in-maint": false,
                "is-closed-loop-disabled": false,
                "model-invariant-id": "vCPE_Intrastructure_Metro_vGMUX",
                "model-version-id": "vCPE",
                "orchestration-status": "Error",
 ===========>   "resource-version": "1510104391195",  <====================
                "service-id": "vFirewall_demo_app",
                "vnf-id": "vCPE_Infrastructure_vGMUX_demo_app",
                "vnf-name": "vCPE_Infrastructure_vGMUX_demo_app",
                "vnf-type": "vCPE"
            },
            "inventory-response-items": {
.
.
.

The resource-version can be obtained from the verify.sh then used to update the udpate-vnf-status.json prior to executing the update.sh script. For me details go to Resetting A&AI VNF Orchestration Status.

$ update.sh aai1

Update APPC DB

Remove any VNF - VNF_LOCK_MANAGEMENT or VNF - VNF_STATE_MANAGEMENT entries from any prior executed tests.

During test execution sometimes the action fails and the VNF's state is not stable and the VNF might still be locked. In that case, you may remove the State and Lock info for the VNF.

VNF_STATE_MANAGEMENT List

select * from VNF_STATE_MANAGEMENT;

Sample Results:
+------------------------------------+----------+-----------------------------------------------------------+---------------+-----+
| VNF_ID                             | STATE    | OWNER_ID                                                  | UPDATED       | VER |
+------------------------------------+----------+-----------------------------------------------------------+---------------+-----+
| APPC_Test_VM                       | UNSTABLE | VNF-Restart-APPC_Test_VM@VNF-DMaaP-Restart-APPC_Test_VM@2 | 1509747445000 |   1 |
+------------------------------------+----------+-----------------------------------------------------------+---------------+-----+
delete from VNF_STATE_MANAGEMENT where VNF_ID="APPC_Test_VM";

The SQL for the VNF_LOCK_MANAGEMENT will be similar to the example of the VNF_STATE_MANAGEMENT above.

Login to API Doc Explorer

The API Doc Explorer can be found at http://appc-address:8282/apidoc/explorer/index.html.

Create VNF Restart Request

Create your request with the VNF info as well as a current date/time.

Sample API Doc Explorer VNF Restart
{ "input": { "common-header": { "timestamp": "2017-10-25T11:10:04.244Z", "api-ver": "2.00", "originator-id": "664be3d2-6c12-4f4b-a3e7-c349acced200", "request-id": "664be3d2-6c12-4f4b-a3e7-c349acced200", "sub-request-id": "1", "flags": { "force" : "TRUE", "ttl" : 60000 } }, "action": "Restart", "action-identifiers": { "vnf-id": "vCPE_Infrastructure_vGMUX_demo_app" } } }

Submit Your Request

Submit your request making sure you're submitting the request to the /opterations/appc-provider-lcm:restart

Look for Request Accepted

In the response body you should see "ACCEPTED - request accepted"

Monitor the Karaf Log

Monitor the karaf.log in the /opt/opendaylight/current/data/log directory using "tail -f karaf.log" or view the file itself. You should observe "os-stop" and "os-start" in the log.
os-stop/os-start example from karaf.log
2017-11-03 13:47:23,406 | INFO | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | 4 * Sending client request on thread appc-dispatcher-1 4 > POST http://10.0.14.1:80/api/multicloud-titanium_cloud/v0/pod25_RegionOne/compute/v2/cf142fb96ce24770946c0f7117c0aeb3/servers/60be808f-5d45-4a60-a578-9c0c6ee6be08/action 4 > Accept: application/json 4 > Content-Type: application/json 4 > X-Auth-Token: e33cb6556d414414a4df96267b454192 { "os-stop" : { } } 2017-11-03 13:47:23,406 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | sun.net.www.MessageHeader@5778469e10 pairs: {POST /api/multicloud-titanium_cloud/v0/pod25_RegionOne/compute/v2/cf142fb96ce24770946c0f7117c0aeb3/servers/60be808f-5d45-4a60-a578-9c0c6ee6be08/action HTTP/1.1: null}{Accept: application/json}{X-Auth-Token: e33cb6556d414414a4df96267b454192}{Content-Type: application/json}{User-Agent: Jersey/2.25.1 (HttpUrlConnection 1.8.0_131)}{Cache-Control: no-cache}{Pragma: no-cache}{Host: 10.0.14.1}{Connection: keep-alive}{Content-Length: 21} 2017-11-03 13:47:24,366 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | sun.net.www.MessageHeader@2cc678bf10 pairs: {null: HTTP/1.1 202 Accepted}{Server: openresty}{Date: Fri, 03 Nov 2017 13:47:31 GMT}{Content-Type: text/html}{Transfer-Encoding: chunked}{Connection: keep-alive}{X-Subject-Token: e33cb6556d414414a4df96267b454192}{X-Frame-Options: SAMEORIGIN}{Vary: Cookie}{Allow: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS} 2017-11-03 13:47:24,366 | INFO | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | 4 * Client response received on thread appc-dispatcher-1 4 < 202 4 < Allow: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS 4 < Connection: keep-alive 4 < Content-Type: text/html 4 < Date: Fri, 03 Nov 2017 13:47:31 GMT 4 < Server: openresty 4 < Transfer-Encoding: chunked 4 < Vary: Cookie 4 < X-Frame-Options: SAMEORIGIN 4 < X-Subject-Token: e33cb6556d414414a4df96267b454192 2017-11-03 13:47:46,530 | INFO | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | 7 * Sending client request on thread appc-dispatcher-1 7 > POST http://10.0.14.1:80/api/multicloud-titanium_cloud/v0/pod25_RegionOne/compute/v2/cf142fb96ce24770946c0f7117c0aeb3/servers/60be808f-5d45-4a60-a578-9c0c6ee6be08/action 7 > Accept: application/json 7 > Content-Type: application/json 7 > X-Auth-Token: e33cb6556d414414a4df96267b454192 { "os-start" : { } } 2017-11-03 13:47:46,530 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | sun.net.www.MessageHeader@79d7c76110 pairs: {POST /api/multicloud-titanium_cloud/v0/pod25_RegionOne/compute/v2/cf142fb96ce24770946c0f7117c0aeb3/servers/60be808f-5d45-4a60-a578-9c0c6ee6be08/action HTTP/1.1: null}{Accept: application/json}{X-Auth-Token: e33cb6556d414414a4df96267b454192}{Content-Type: application/json}{User-Agent: Jersey/2.25.1 (HttpUrlConnection 1.8.0_131)}{Cache-Control: no-cache}{Pragma: no-cache}{Host: 10.0.14.1}{Connection: keep-alive}{Content-Length: 22} 2017-11-03 13:47:47,513 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | sun.net.www.MessageHeader@4c0b92710 pairs: {null: HTTP/1.1 202 Accepted}{Server: openresty}{Date: Fri, 03 Nov 2017 13:47:54 GMT}{Content-Type: text/html}{Transfer-Encoding: chunked}{Connection: keep-alive}{X-Subject-Token: e33cb6556d414414a4df96267b454192}{X-Frame-Options: SAMEORIGIN}{Vary: Cookie}{Allow: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS} 2017-11-03 13:47:47,514 | INFO | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | 7 * Client response received on thread appc-dispatcher-1 7 < 202 7 < Allow: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS 7 < Connection: keep-alive 7 < Content-Type: text/html 7 < Date: Fri, 03 Nov 2017 13:47:54 GMT 7 < Server: openresty 7 < Transfer-Encoding: chunked 7 < Vary: Cookie 7 < X-Frame-Options: SAMEORIGIN 7 < X-Subject-Token: e33cb6556d414414a4df96267b454192

Monitor SvcLogic Log

The SvcLogic log is in the /opt/opendaylight/current/data/log "SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0].log"
Sample SvcLogic for Generic_Restart
Example of DG Node Execution - "Executing node 16" for example: 2017-11-03 14:50:49,739 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Executing node 16 2017-11-03 14:50:49,739 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Executing node executor for node type switch - org.onap.ccsdk.sli.core.sli.provider.SwitchNodeExecutor 2017-11-03 14:50:49,739 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Executing switch node 2017-11-03 14:50:49,739 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | test expression ($vnfInfo.inventory-response-item[0].generic-vnf.vnf-id) evaluates to APPC_Test_VM

Look for Success in SvcLogic or Karaf Log

In the Generic_Restart SVCLogic log you should observe "2017-11-03 14:51:40,592 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Returning status success" once the Restart has occured for all VMs in the VNF.

Sample SvcLogic for Generic_Restart Success
Example of a successfully executed DG: 2017-11-03 14:51:40,591 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Executing node 60 2017-11-03 14:51:40,592 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Executing node executor for node type return - org.onap.ccsdk.sli.core.sli.provider.ReturnNodeExecutor 2017-11-03 14:51:40,592 | DEBUG | SvcLogicGraph [module=APPC, rpc=Generic_Restart, mode=sync, version=3.0.0] | Returning status success

Verify Restart in Cloud Dashboard

Verify the VNF/VMs were restart in the Cloud Provider dashboard. Currently, that is OpenStack and the status can be viewed in the Horizon dashboard. Look at the Action Log observing a stop then a start.