...
Identified Need | Description | Driver | Dependencies | Project fit |
Test Suite | robot test suites executed | Jenkins Job Flow | Autorelease | Trigger daily autorelease job to build all projects together from source |
Project proposals.
...
•Code and tools for automatic system testing and continuous integration test flows across ONAP projects •Common guidelines, templates, and best practices to help project developers to write unit and system test code •Framework and tools for security testing •robot test suites executed. Consumer driven/broker driven testing framework | •Automate the building artifacts/binaries to minimize human errors and reduce engineering costs •Ensure that changes in one project will not break the functionality of other projects •Assure that the entire ONAP project/product functions correctly in the case of continual change in subprojects •Ensure consistency in unit and system testing methodology across all the ONAP projects •Capture security issues | Robot Framework, Project Specific Contributions of Tests | integration | |
CI Builder | •Scripts and definitions for build pipelines and CI jobs in Jenkins, as well as for VM and docker images required for CI tests | •Required to support the executing of CI jobs (e.g. for Jenkins) | Jenkins | integration |
Autorelease | •Scripts to build the artifacts/binaries (e.g. zip/targz files) that are used in the release candidates and final release from scratch | •Detect cyclical dependencies •Generate release candidates and final release •It generates dependency lists / graph automatically | Jenkins | integration |
Distribution | •Scripts to be used by the end user for setup and execution of the ONAP project modules •Default/sample configuration files, README files | •Ease of industry adoption of ONAP by providing scripts and other information for setup/installation/configuration | integration | |
Packaging | •Scripts and package definitions for deb, RPM, etc. installer packages for various Linux or other OS distributions •Sample VM or docker images | •An industry standard installer could help anyone to try out ONAP easier •Industry adaption | Jenkins, Nexus | integration |
S3P | •Test cases for performance, scalability, resilience/stress testing, longevity •Benchmarking and performance whitepapers | •Define standard S3P testing metrics •Provide and publish benchmarking results | Jenkins, | integration |
Infrastructure Specification | •Develop the specifications for the “ONAP compliant” deployment and test environment | •Assist the planning and procurement of the necessary hardware and infrastructure for setting up ONAP environments | integration | |
Bootstrap | •A framework to automatically install and test a set of base infrastructure components for new developer | •Reduce the barrier of entry to allow new ONAP developers to ramp up onto active development quickly •Reduce the cost to the community in responding to simple environment setup questions faced by new developers | integration | |
VF Compliant and Verification Testing | •Create automatic test cases and script for VF testing •Perform VF compliant testing and verification using tools provided by ONAP •Delivery the testing reports and whitepaper | •Assist define the testing metrics •Reduce adoption risks for end-users | Jenkins, Labs | integration |
Community Integration Lab | •Scripts and definitions for setting up a POC sample deployment of use cases in lab settings •Provisioning, installation, and setup of all the telco equipment such as switches, routers, and gateways to enable end to end testing •Allow remote access to the lab environment for interoperability testing •Automatic updates of code in lab environment from future releases | •Support the needs of consistent, reproducible lab setup for demo and POC purposes •Promote easy interoperability testing with different hardware devices, SDN controllers, etc. •Automate the process of keeping the lab code up to date with the latest changes | Labs | integration |
| Template | Templates for test suites | integration |
Project proposals.
Project 1:
Project ID: integration
Project Name:- System Integration and Testing
Project Description:- Provide
- Responsible for ONAP cross-projects system integration and all related testing, such as VNF compliant & verification testing, necessary for the successful delivery and industry adaption of the ONAP project as a whole. Provide integration tools and processes for CI/CD of source master and release branches. Consolidation of common test suites.
- Test
- CI Builder
- Autorelease
- Distribution
- Packaging
- S3P
- Infrastruture Specification
- Bootstrap
- VF Compliant and Verification Testing
- Community Integration Lab
- Test Templates
Scope:
- Quickly identify scope, consider: documentation, APIs, models, testing, integration, functionality.
- Note of any particular deliverables to highlight.
- If anything is out of scope, not it down
Other:
- Identify baseline code (if any)
Project 2:
Project Name:
- Provide a brief project name.
Project Description
- Provide a high level description of the project
Scope:
- Quickly identify scope, consider: documentation, APIs, models, testing, integration, functionality.
- Note of any particular deliverables to highlight.
- If anything is out of scope, not it down
Other:
...
- Provides all the cross-project infrastructure framework and DevOp toolchain, code and scripts, best practice guidance and benchmark, testing reports and white paper related to:
- Cross projects Continuous System Integration Testing (CSIT)
- VNF compliant and verification testing leveraging ONAP projects
- Release delivering of the ONAP project
- PoC: building and maintenance community integration labs
- Continuous Distribution (CD) to ONAP community integration labs
Other:
- Leveraging existing set up already in ONAP/ECOMP and OPEN-O
Meeting Notes:
#info
...
helen
...
says
...
presents
...
Slidedeck:
...
System
...
Integration
...
and
...
Testing
#info
...
helen
...
says
...
integration
...
team
...
is
...
a
...
helper
...
project.
...
Not
...
have
...
authority
...
to
...
be
...
gatekeeper.
#info
...
helen says we
...
use
...
nexus,
...
docker,
...
slack,
...
bitgeria,
...
swagger, etc.
#info helen says for open-o, created many test cases
...
#question will there be docker level test? looking Looking at the docker logs?
- We have the base infrastructure for it. We do not scan the logs for strings. The robot test framework will check against the signature of the api call.
#question Will there be dynamic testing?
- We trigger unit level testing triggered automatically by Jenkins at build time
#question In the microservice, the broker based testing is getting more popular.
- consumer driven, broker driven
#question who builds the test?
- The developer will write the test and the integration team provides the framework, but it will be up to individual projects
#question do we test both expected and unexpected inputs?
- We should test both.
#question can we consider a consolidation of the test?
- That's the goal of the common integration.
#question do we have a way to leverage localize test environments?
- Requires additional work with LF
#question Can we have test templates?
- Yes
#question We need a project to address security concerns?
- The TSC needs to own that and create a committee for that
Presentations
System Integration and Testing.pdf