...
The testsuites usually include several tests(e.g. onap_vnf include basic_ubunutu, freearius_nbi and clearwater_vims, security includes 6 tests). Tests can be run sequentially or in parallel.
To be integrated in CI, you must integrate your tests in of of the defined test categories (see Integration categories & testsuites discussions):
- infrastructure-healthcheck
- healthcheck
- smoke-usecases
- candidate-usecases
- security
- benchmark
The testing part
1) Your tests must be fully automated...
Your tests must be fully automated. It means that it shall take into account the env on which it is executed, setup the resources, run the tests, delete the resources created for the test
if you are using non open source VNF, proprietary third components/equipments you will not be able to fully automate.
...
You can develop your test using any language (python, bash, robotframework, go..) and any framework.
Ideally use case owner shall be able to provide the automatic procedure at this stage.
2) ...and integrated in a docker based on Xtesting
However your As any framework is allowed, at the end we may have a huge diversity of tests, test frameworks, test artifacts. In order to simplify the integration of the tests in any CI chains, we decided to leverage the OPNFV xtesting framexork.
Whatever your tests, they will be launched always the same way and generate consistent artifacts. As they will be dockerized, they will be portable in any CI/CD system.
Your test suites must then be "xtestingized", it means they must be embedded in a docker file leveraging Xtesting.
This task can be done by the integration team - the most difficult is the test itself...
Xtesting is a light framework aiming to harmonize test inputs and outputs, which is very helpful for integration.It has been developped in OPNFV and already proposed several infrastructure test dockers for Openstack and kubernetes, leveraging upstream tests (https://wiki.opnfv.org/pages/viewpage.action?pageId=13211751).
...
*: note that Xtesting is open source so if your favorite framework is missing you can add it
The CI part
1) Your tests must be declared in the DB...
for that you need to have access to the common test DB used by OPNFV and ONAP: testresults.opnfv.org
...
it is obviously possible to update, delete using usual mongo commands.
2) ... and in the CI
Once everything is declared in the DB, you can add the test in the CI chain.
...