Versions Compared

Old Version 68

changes.mady.by.user Toine Siebelink

Saved on

compared with

New Version 69

changes.mady.by.user Toine Siebelink

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents









CPS Team Testing Strategy

  1. Test Strategy.pptx
  2. Current Tests with Vertical Slice Example

CPS Test Overview

CPS testing strategy follows best in class quality principles and practices; based on the Testing Pyramid as recommended by Martin Fowler.


credit: https://martinfowler.com/articles/practical-test-pyramid.html


This Diagram shows an overview of the different type os test and their scope related to the architecture layers

Implementation

Drawio
bordertrue
diagramNameCPS Test Overview
simpleViewerfalse
width400
linksauto
tbstyletop
lboxtrue
diagramWidth1621
revision2

Integration Test
Semi Integration
Unit Test


Over time we have developed an extensive set of tests:

1

(UI) Integration test (since we don't have a UI these start at the REST layer of our components)

    1. CSIT Component System Integration Test.  CPS utilizes Robot framework for CSIT tests..
    2. K6 Integration Test
    3. K6 Performance Tests
      1. Performance Graphs

2

Service Test (or Semi-Integration Test) using Spring Boot Test Containers for Postgress and Kafka and Mocked/Stubbed Services for external components like DMI and Policy Executor. These test are written in Groovy & Spock

    1. Functional
    2. Performance
      1. Performance Graphs

3

Unit Tests. CPS uses Groovy & Spock for unit testing


4

Contract Testing (not visible in diagram)


Integration Test

CSIT Integration Test

K6 Test Overview

<TBD>

Service / Semi-Integration Test

Persistence Testing

  1. CPS Persistence layer (semi-integration) testing
  2. Liquibase Change Log (sets) Testing (test DB Upgrade)

Contract Testing

  1. Contract Testing.pptx
  2. CPS has developed stubs for clients. 
    1. Link to Gerrit patch:  https://gerrit.onap.org/r/c/cps/+/129256
    2. Jira Link:
      Jira Legacy
      serverSystem Jira
      serverId4733707d-2057-3a0f-ae5e-4fd8aff50176
      keyCPS-919

Performance Testing

Automated Performance Test

Our K6 test now included automated performance testing for some (NCMP) Test Cases (KPIs)

Performance Test Stubs

  1. NCMP Performance Test With Wiremock (not in use amymore)

Performance Test Results

  1. CPS Basic Performance (Load) TestClosed issues related to CPS performance testing

Testing Frameworks

Groovy & Spock

  1. Groovy & Spock Test Code Conventions
  2. OneSummit Groovy & Spock Workshop Nov 2022 (includes sample project & exercises!)
  3. Additional resources:
    1. https://www.baeldung.com/groovy-spock
    2. http://spockframework.org/spock/docs
    3. https://github.com/spockframework/spock-example
    4. http://groovy-lang.org/documentation.html

Other Test / Verification Tools

Read the Docs (RTD)

Prerequisites

  1. Linux Environment (WSL works too)
  2. Python
  3. Tox  (see https://tox.wiki/en/3.14.1/install.html)

Steps

  1. (temporary) Modify docs/tox.ini (for example https://gerrit.onap.org/r/c/cps/+/135892/2/docs/tox.ini ) do NOT merge these changes into master!
    1. to use YOUR python version
    2. If you have a lot of new documentation you can (temporary) remove -W flag from the first two  sphinx-build commands (sections; tox and docs-linkcheck) so it doesn't stop on the first error
      (note there are always a lot of spelling 'warnings' so the third build command should always be run without the -W flag (section; docs-spellcheck) 
  2. cd into <your project location>docs
  3. run tox
  4. tip: First time run can take a few minutes! use -V or -VV for more verbose output to see what it is doing

Alternate checks

  1. Windows Subsystem (WSL):  doc8 *.rst --ignore D002 --ignore D004
    (need to ignore warnings about CR, trailing whitespace and line-length because of different filesystem)
  2. Note. If there are unexpected issues with Linelength ie. errors locally that are not causing problems in CI then these can be ignored using --ignore D001
    Thomas Kulik suspects this might have to do with doc8 tool version and he will report back to us on this issue

Reference Material

  1. Ref. Martin Fowler Practical-test-pyramid
    1. conclusion-summary (PPT)
  2. CSIT References
    1. Local Setup Using CSIT 
    2. Maximizing Benefits of CSIT in ONAP Development
    3. https://robotframework.org/