The TSC has decided to launch a task to define how to improve Pair-Wise Testing for Dublin as defined in https://jira.onap.org/browse/TSC-42
Current situation
For Beijing Release at Santa-Clara event (Dec, 2017), it was decided to perform pair-wise tests to achieve S3P under Heat and OOM installations.
...
- Dependencies problem. To realize pair-wise testing, the two paired components must be ready. For Casablanca, some components were available late impacting the time to perform pair-wise tests.
- No formal explanation to explain what to test
- No formal way to describe the tests
- No test storage
- No consistent tests: Some projects perform tests in both ways, but there is not a global consistency (ie i.e. NBI tested SO - NBI, while SO did not test SO - NBI)
What to be test?
Each ONAP component includes a large number of Docker containers. Each project must be responsible to perform unitary test to validate the interaction between these Docker containers and to validate the API offered by the components (eg e.g. SO is responsible to test the various offered API).
The main objective is to test that both components are interacting correctly (eg e.g. using the correct ports, protocol, API version...).
...
- Define formally the component dependencies for each project (cf c.f. NBI proposal) by project
- Run CSIT on OOM installation by project
- Develop a tool to verify that pair-wise tests are defined based on component dependencies by integration
- Define pair-wise tests for RC0 by M4 by project
- Run pair-wise tests under Integration project on daily basis after M4 (eg e.g. deploy a full fresh ONAP and run the pair-wise tests) by integration
- Store tests in a DB by integration
...