| Table of Contents |
|---|
CPS Team Testing Strategy
CPS Test Overview
CPS testing strategy follows best in class quality principles and practices; based on the Testing Pyramid as recommended by Martin Fowler.
credit: https://martinfowler.com/articles/practical-test-pyramid.html
This Diagram shows an overview of the different | type os types of test and their scope related to the architecture layers | ImplementationColor Coding | |||||||||||||||||||||
| Integration Source Code | ||||||||||||||||||||||
Unit Test | |||||||||||||||||||||||
| Unit Test | ||||||||||||||||||||||
Integration Test |
Over time we have developed an extensive set of tests:
| 1 |
(UI) Integration test (since we don't have a UI these start at the REST layer of our components)
| 2 | Service Test (or Semi-Integration Test) using Spring Boot Test Containers for |
Postgres and Kafka and Mocked/Stubbed Services for external components like DMI and Policy Executor. These test are written in Groovy & Spock
|
| 3 |
(UI) Integration test (since we don't have a UI these start at the REST layer of our components)
| |
| 4 | Contract Testing (not visible in diagram) |
Integration Test
CSIT Integration Test
- CPS Integration Test Plan
- CSIT Testing for CPS
- Local Setup Using CSIT
- CPS-188 CSIT Testing for CPS
- Project-specific CSIT structure (non CPS page)
K6 Test Overview
<TBD>
Service / Semi-Integration Test
Persistence Testing
- CPS Persistence layer (semi-integration) testing
- Liquibase Change Log (sets) Testing (test DB Upgrade)
Contract Testing
- Contract Testing.pptx
- CPS has developed stubs for clients.
- Link to Gerrit patch: https://gerrit.onap.org/r/c/cps/+/129256
- Jira Link:
Jira Legacy server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-919
Performance Testing
Automated Performance Test
Our K6 test now included includes automated performance testing for some (NCMP) Test Cases (KPIs). See K6 Test Overview
Performance Test Stubs
NCMP Performance Test With Wiremock (not in use amymore)- TBC DMI Stub
- TBC Policy Executor Stub
Performance Test Results
This table is to track once-off performance test exercises/results, preferrable on a child page here.
| Date | Jira | Report | Notes | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| TBD |
| CM Data Notification Subscription Create Performance | |||||||||
| TBD |
| CM Data Notification Forwarding Performance | |||||||||
| Jul 2024 | NCMP: Read data operation resource API Performance for multiple cm-handles | ||||||||||
| Mar 2024 | Performance test for updating YANG schema set API | ||||||||||
| Feb 2024 |
| Alternate ID performance testing | |||||||||
| Sep 2023 |
| Performance plots for CPS in Jenkins | |||||||||
| Aug 2023 | Montreal Read/Write Performance | ||||||||||
| Apr 2023 | CPS-Core Read & Write (large load) Test Results | Used Data Schema and Samples provides by Wipro | |||||||||
| Feb 2023 | CPS Analyse Heap-Memory use | looks like some attachments got lost in ONAP Wiki Cleanup | |||||||||
| Jan 2023 | NCMP De-Register API Performance | NCMP De-registration Performance test guide | |||||||||
| Jan 2023 | Performance: disable CPS-temporal notifications | ||||||||||
| Sep 2022 | NCMP Performance Test With Wiremock | ||||||||||
| Nov 2021 |
| CPS Basic Performance (Load) Test |
Testing Frameworks
Groovy & Spock
- Groovy & Spock Test Code Conventions
- OneSummit Groovy & Spock Workshop Nov 2022 (includes sample project & exercises!)
- Additional resources:
Other Test / Verification Tools
Read the Docs (RTD)
Prerequisites
- Linux Environment (WSL works too)
- Python
- Tox (see https://tox.wiki/en/3.14.1/install.html)
Steps
- (temporary) Modify docs/tox.ini (for example https://gerrit.onap.org/r/c/cps/+/135892/2/docs/tox.ini ) do NOT merge these changes into master!
- to use YOUR python version
- If you have a lot of new documentation you can (temporary) remove
-Wflag from the first twosphinx-buildcommands (sections; tox and docs-linkcheck) so it doesn't stop on the first error
(note there are always a lot of spelling 'warnings' so the third build command should always be run without the -W flag (section; docs-spellcheck)
- cd into
<your project location>docs - run
tox - tip: First time run can take a few minutes! use -V or -VV for more verbose output to see what it is doing
Alternate checks
- Windows Subsystem (WSL):
doc8 *.rst --ignore D002 --ignore D004
(need to ignore warnings about CR, trailing whitespace and line-length because of different filesystem) - Note. If there are unexpected issues with Linelength ie. errors locally that are not causing problems in CI then these can be ignored using
--ignore D001Thomas Kulik suspects this might have to do with doc8 tool version and he will report back to us on this issue
Reference Material
- Ref. Martin Fowler Practical-test-pyramid
- CSIT References