Introduction

CSIT ("Continuous System and Integration Testing", even though you could also read that as "Component-Specific Integration Testing") cases in ONAP exist typically for verifying functionalities of a single or few ONAP components in limited testing environments (suitable for lightweight installation in local development environment and Jenkins) and often rely on simulators. Currently these tests are executed once per day on images from master branch (as well as on maintenance branches of selected previous releases) or whenever the test cases themselves are committed for review. This means that by ONAP's current automated verification procedure any new bugs that could be caught by the existing CSIT cases are not discovered until the bugs are already merged to master. In addition, any failures in daily CSIT Jenkins are not sending any kind of notifications or raising tickets or alarms, so the teams are not even getting any automated feedback from the failures that can therefore go days or in worst cases weeks without notice (much less action).

...

• Project teams should check their CSIT Jenkins jobs every day and treat any failures as showstoppers
  • It is critical that failures are at least analysed within 30 days, because that's how long Jenkins stores the results. After 30 days you can no longer find out via Jenkins what changes broke the build!
  • integration team could also benefit from checking these and making sure teams are alerted to any failures - in longer term this should save final integration phase troubles and keep integration team generally up to date on the overall state of the CSIT suites and therefore (at least ideally) of the ONAP project
• The importance of manual CSIT execution by developers before committing (and all actions making that as easy as possible) is emphasized since it is currently the only gatekeeper before code merges