Beijing Specific Tests
Test No. | Category (Stability, Resiliency, ConfigScaleOut, Scalability) | Description | Status (Not Started, In-Progress, Pass, Fail) | Notes | |||
|---|---|---|---|---|---|---|---|
| 1 | Stability | Execute Restart Test for 72 hour period | Pass | Tested against Amsterdam code due to defect APPC-658 which prevents DMaaP messages from being received or sent. Test results and notes can be found at: ONAP APPC 72 Hour Stability Test Results | |||
| 2 | Stability | Execute Rebuild Test for 72 hour period | Pass | Tested against Amsterdam code due to defect APPC-658 which prevents DMaaP messages from being received or sent. Test results and notes can be found at: ONAP APPC 72 Hour Stability Test Results | |||
| 3 | Stability | Execute Start Test for 72 hour period | Pass | Tested against Amsterdam code due to defect APPC-658 which prevents DMaaP messages from being received or sent. Test results and notes can be found at: ONAP APPC 72 Hour Stability Test Results | |||
| 4 | Stability | Execute Stop Test for 72 hour period | Pass | Tested against Amsterdam code due to defect APPC-658 which prevents DMaaP messages from being received or sent. Test results and notes can be found at: ONAP APPC 72 Hour Stability Test Results | |||
| 5 | Stability (Heat) | Execute 72 hour test with steady stream of Restarts, Rebuilds, Start, and Stop actions in an Heat deploy APPC environment using Beijing running on Nitrogen | Pass | APPC-658 issue was fixed for this test. | |||
| 6 | Stability (OOM) | Execute 72 hour test with steady stream of Restarts, Rebuilds, Start, and Stop actions in an OOM clustered APPC environment using Beijing running on Nitrogen | Pass | APPC-658 issue was fixed for this test. | |||
| 7 | Resiliency | Verify that loss of one MYSQL slave instance does not impact DB availability. | Pass (on Carbon ODL) Pass (on Nitrogen ODL) | Since MariaDB and Galera support is not being introduced in this release, this test involves terminating the one MySQL instance and verifying that Kubernetes brings the instance back up successfully. | |||
| 68 | Resiliency | Verify that loss of one APPC controller instance does not impact application availability | Pass (on Carbon ODL) Pass (on Nitrogen ODL) | Loss of one APPC controller instance does not impact application availability. | |||
| 79 | Scalability | Scale (up or down) MySQL dynamically. | Fail | Dynamic scaling for the database is scheduled to be supported with the introduction of MariaDB and Galera in a future release. | |||
| 810 | Scalability | Scale (up or down) APPC controller dynamically. | Pass with issue (on Carbon ODL) Pass with issue (on Nitrogen ODL) | The APPC controller can be dynamically scaled; however, the ODL cluster is not configured properly after the scaling operation. Based on this issue, it is recommended to use static scaling such that the cluster size is set appropriately at the time of installation to ensure a properly functioning ODL cluster. | |||
| 11 | ConfigScaleOut via apidoc | vLB/vDNS used. Scenario: new vDNS instantiated and IP of new vDNS instance must be made known (i.e., configured) to vLB via ConfigScaleOut request | Pass | Additional information: APPC ConfigScaleOut Testing with vDNS/vLoadBalancer | |||
| 12 | ConfigScaleOut via DMaaP | vLB/vDNS used. | 9 | ConfigScaleOut | Scenario: new vDNS instantiated and IP of new vDNS instance must be made known (i.e., configured) to vLB via ConfigScaleOut request | Pass | Additional information: APPC ConfigScaleOut Testing with vDNS/vLoadBalancer |
| 13 | CDT | CDT connect to APPC runtime | Pass | ||||
| 14 | CDT | CDT distribute artifacts to APP runtime | Pass | Creating the vLoadBalancer in the CDT | |||
Regression Tests
TC No. | Action | Heat | OOM | LCM API Provider | APPC Provider | Via DGOrchestrator | Direct to OpenStack | Thru MultiVIM | Query AAI | NetConf | Via DMaaP | Status | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ModifyConfig | Yes | Yes* | N/A | Yes | Yes | Pass | vFW/pgsteam change regression use case | |||||
| 2 | Healthcheck | Yes | Yes | Need jar containing simulator with a service running in order to test. | |||||||||
| 3 | Restart | Yes | Yes* | Yes | Yes | Yes | Yes | Yes | Pass | ||||
| 4 | Restart | Yes | Yes | Yes | Yes | Yes | Pass | This test is using the Generic_Restart DG designed to allow for VIM. | |||||
| 5 | Restart | Yes | Yes* | Yes | Yes | No | No | Pass | |||||
| 6 | Restart | Yes* | Yes | Yes | Pass | ||||||||
| 7 | Stop | Yes | Yes* | Yes | Yes | Yes | Yes | Yes | Pass | ||||
| 8 | Start | Yes | Yes* | Yes | Yes | Yes | Yes | Yes | Pass | ||||
| 9 | Rebuild | Yes | Yes* | Yes | Yes | Yes | Yes | Yes | Pass | ||||
| 10 | Rebuild | Yes | Yes* | Yes | Yes | No | Pass | 2018-04-26 14:04:12,445 | INFO | p1257028015-1480 | eelf | 401 - appc-common - 1.3.0.SNAPSHOT - - | APPC0120I Rebuild '1' finished with status SUCCESS. Reason: Success | |||||
| 11 | Snapshot | Yes | Yes* | Yes | Yes | No | No | Partial | Snapshot created successfully but the image verification failed due to APPC-869. | ||||
| *For OOM - APPC MySQL sql_mode differs from the standard OOM MySQL deployment due to restrictions when sql_mode has "only_full_group_by" enabled. APPC-878 | |||||||||||||