Motivation
...
- CSIT suites that test components from multiple project repositories at the same time
- such CSIT tests may have to be separated using additional simulators, or
- project repository structures themselves may have to be reconsidered
- In any case, the division between the images under test and images that are just necessary dependencies should be clearly made and documented
- the images under test should be coming from the commit branch
- in the case of necessary dependencies it must be decided whether they should be provided as simulators or real components
- if provided as real components, they should be referred to with a fixed, unchanging version number and should be stable and mature enough to develop on
- ONAP's (unintentional?) practice of allowing the same versioned image to be changed might prove problematic
- if provided as real components, they should be referred to with a fixed, unchanging version number and should be stable and mature enough to develop on
- project-specific simulators are trivial case from dependency handling point of view, but if common simulators are in use, dependency considerations for them are the same as with real components
- Jenkins templates may have to be redesigned to support unified approach for triggering review branch-specific docker build, CSIT execution and voting chain as part of review verification
- What is the most elegant and effortless solution for ensuring that the docker images built from commit branch are taken in use in the CSIT (both in Jenkins and in local development environment)?
- CSITs would become blockers for merging code
- local pre-commit verification should be supported better by common CSIT tools
are all projects and suites mature enough to deal with that?
- Docker image production practices will have to be unified (see Docker Image Build Guidelines and also Independent Versioning and Release Process)
- To which extent are the current guidelines actually followed?
- Are the guidelines good enough for CSIT development as they are now if they were followed?
...
| Projects with CSIT | CSIT jobs | CSIT job triggers | Tested docker images and repositories | Docker build jobs that create the tested images | Dependencies | Comments | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aaf (disabled) | aaf-frankfurt-csit-certservice Frankfurt branch only? The tests triggered from here do not even exist in master (nor are there any jobs for still remaining aafapi and sms-test-plan?) | aaf-certservice-frankfurt-merge-java - this job neither builds a docker image nor triggers another job that does it, so why trigger CSIT? | nexus3.onap.org:10001/onap/ org.onap.aaf.certservice.aaf-certservice-api:frankfurt-latest (hardcoded) The source repository is aaf/certservice | aaf-certservice-maven-docker-stage-frankfurt (note that aaf-certservice-release-verify and aaf-certservice-release-merge also build aaf-certservice-api image, but not with frankfurt-latest tag) | primekey/ejbca-ce:6.15.2.5 (hardcoded, real image used) | In maintenance mode, CSIT disabled:
| ||||||||||||||||||||||||||||||||||||||||
| appc (disabled) | appc-master-csit-healthcheck | - | nexus3.onap.org:10001/onap/appc-image:1.7.2-SNAPSHOT-latest nexus3.onap.org:10001/onap/appc-cdt-image:1.7.2-SNAPSHOT-latest (hardcoded) docker-compose.yml as well as appc-image and appc-cdt-image come from appc/deployment | No job is producing these images at the moment! appc-deployment-master-docker-java-daily produces 1.8.0-SNAPSHOT-latest and appc-deployment-frankfurt-docker-java-daily is producing 1.7.3-SNAPSHOT-latest 1.7.2-SNAPSHOT-latest images have not been created since May 27 | mariadb:10.4.3 nexus3.onap.org:10001/onap/ccsdk-dgbuilder-image:0.6.0 (no longer produced, last update to Nexus on Aug 30 2019) nexus3.onap.org:10001/onap/ccsdk-ansible-server-image:0.4.4 (no longer produced, last update to Nexus on Jun 06 2019) (hardcoded, real images used) | In maintenance mode, CSIT disabled:
| ||||||||||||||||||||||||||||||||||||||||
ccsdk (temporarily disabled, see
| See
| |||||||||||||||||||||||||||||||||||||||||||||
| ccsdk-oran | ccsdk-oran/job/ccsdk-oran-maven-merge-master
|
The source repository is ccsdk/oran | Same as triggers | nexus3.onap.org:10003/onap/sdnc-image (2.1.0-STAGING-latest for master, nexus3.onap.org:10003/onap/sdnc-image:2.0.1-STAGING-latest for Guilin) nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-controlpanel:2.0.0 | These tests are currently cloned from ORAN repository and they are essentially plain shell scripts that are just wrapped into Robot In Honolulu the test cases should be brought under ONAP repository and rewritten with Robot and ONAP CSIT scripts as much as possible | |||||||||||||||||||||||||||||||||||||||||
| clamp | Invalid trigger jobs removed in
| nexus3.onap.org:10001/onap/clamp-backend:4.1-STAGING-latest (hardcoded) The docker-compose.yml and source of the images is clamp | clamp-maven-docker-stage-elalto
| mariadb:10.5.4 python:2-slim | The versions are a mess; master docker builds are producing 5.1 versions, frankfurt branch 5.0 versions, and el-alto 4.1 versions. However, CSIT master branch is testing versions 4.1 and CSIT does not even have el-alto branch? | |||||||||||||||||||||||||||||||||||||||||
| dcaegen2 | dcaegen2-services-son-handler-master-csit-testsuites dcaegen2-services-pmsh-master-csit-testsuite dcaegen2-services-bbs-event-processor-master-csit-bbs-testsuites dcaegen2-pmmapper-master-csit-pmmapper dcaegen2-master-csit-testsuites dcaegen2-master-csit-prh-testsuites dcaegen2-collectors-restconf-master-csit-testsuites dcaegen2-collectors-hv-ves-master-csit-testsuites dcaegen2-collectors-datafile-master-csit-ManagementInterface-suite | dcaegen2-services-son-handler-master-merge-java dcaegen2-services-pmsh-docker-merge-master dcaegen2-services-master-bbs-event-processor-merge-java (pmmapper has no trigger job that actually exists) dcaegen2-collectors-ves-master-merge-java dcaegen2-services-prh-master-merge-java dcaegen2-collectors-restconf-master-merge-java dcaegen2-collectors-hv-ves-master-merge-java (datafile CSITs have no trigger jobs that actually exist)
| nexus3.onap.org:10003/onap/org.onap.dcaegen2.services.son-handler:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pmsh:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.components.bbs-event-processor:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.pm-mapper:latest nexus3.onap.org:10003/onap/org.onap.dcaegen2.collectors.ves.vescollector:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.services.prh.prh-app-server:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.restconfcollector:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:latest nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:latest (all hardcoded in various places in the tests) TBD: most or all of these are in their own dedicated repositories, TBD: are the repo-image-CSIT relations straightforward 1:1 in all cases (or any case)? | Some of the triggering jobs produce the tested images, others do not produce any image; requires a deeper look | A lot of simulators used, TBD | |||||||||||||||||||||||||||||||||||||||||
| dmaap | dmaap-buscontroller-master-csit-ssl dmaap-buscontroller-master-csit-with_dr dmaap-buscontroller-master-csit-with_mr dmaap-datarouter-master-csit-ssl-dr-suite | dmaap-datarouter-maven-docker-stage-master
| nexus3.onap.org:10001/onap/dmaap/buscontroller nexus3.onap.org:10001/onap/dmaap/datarouter-prov nexus3.onap.org:10001/onap/dmaap/datarouter-node nexus3.onap.org:10001/onap/dmaap/datarouter-subscriber (all hardcoded, version not given so latest by default) At least three different source repositories: dmaap/messagerouter/messageservice | There are no jobs producing onap/dmaap/buscontroller - the latest image in Nexus has been updated Thu Feb 21 2019 and belongs to Casablanca (?!) - ticket raised:
dmaap-datarouter-maven-docker-stage-master (same as job trigger) | TBD | It's unclear in some of these suites if datarouter and messagerouter are dependencies for buscontroller or components under test? We seem to have multi-repository problem here that needs to be solved somehow | ||||||||||||||||||||||||||||||||||||||||
| multicloud | multicloud-master-csit-functionality1 |
| nexus3.onap.org:10001/onap/multicloud/vio:1.4.1 nexus3.onap.org:10001/onap/multicloud/framework:1.4.1 nexus3.onap.org:10001/onap/multicloud/openstack-starlingx nexus3.onap.org:10001/onap/multicloud/vio (all hardcoded, default latest for those that do not define version explicitly) | Latest in Nexus: onap/multicloud/vio:1.4.1 is from Sep 12 2019 onap/multicloud/framework:1.4.1 is from Sep 06 2019 ...these seem to be some early El-Alto versions? Ticker raised: https://jenkins.onap.org/view/multicloud/job/multicloud-openstack-maven-docker-stage-elalto produces version 1.4.3 of onap/multicloud/openstack-starlingx https://jenkins.onap.org/view/multicloud/job/multicloud-framework-maven-docker-stage-master produces version 1.6.0 (and also latest) of onap/multicloud/framework multicloud-openstack-maven-docker-stage-master produces version 1.5.6 (and also latest) of onap/multicloud/openstack-starlingx multicloud-openstack-vmware-maven-docker-stage-master produces version 1.4.2 (and also latest) of onap/multicloud/vio | - | |||||||||||||||||||||||||||||||||||||||||
| music (disabled) | music-master-verify-csit-music-distributed-kv-store-test-plan | music-distributed-kv-store-master-docker-golang-shell-daily | nexus3.onap.org:10001/onap/music/distributed-kv-store (hardcoded, latest by default) | music-distributed-kv-store-master-docker-golang-shell-daily | - | In maintenance mode, CSIT disabled: | ||||||||||||||||||||||||||||||||||||||||
| oom-platform-cert-service | oom-platform-cert-service-master-merge-java (triggers two CSITs but does not produce or trigger docker image? Ticket raised:
| nexus3.onap.org:10001/onap/org.onap.oom.platform.cert-service.oom-certservice-api:latest nexus3.onap.org:10001/onap/org.onap.oom.platform.cert-service.oom-certservice-post-processor:latest (but started in Robot test and defined in Robot variable so it would be theoretically overridable) | oom-platform-cert-service-maven-docker-stage-master (for both) | primekey/ejbca-ce:6.15.2.5 | ||||||||||||||||||||||||||||||||||||||||||
| optf |
| nexus3.onap.org:10001/onap/optf-cmso-service:latest nexus3.onap.org:10001/onap/optf-cmso-dbinit:latest nexus3.onap.org:10001/onap/optf-cmso-topology:latest nexus3.onap.org:10001/onap/optf-cmso-ticketmgt:latest nexus3.onap.org:10001/onap/optf-cmso-optimizer:latest nexus3.onap.org:10001/onap/optf-cmso-robot:latest (!! The actual Robot tests run here in setup phase instead of as separate Robot tests from CSIT repo) nexus3.onap.org:10001/onap/optf-has:2.0.2-SNAPSHOT-latest nexus3.onap.org:10001/onap/optf-osdf:2.0.3-SNAPSHOT-latest Repositories: | optf-cmso-maven-docker-stage-master optf-has-maven-docker-stage-master optf-osdf-maven-docker-stage-master (word of warning: the frankfurt versions of the above jobs are also pushing the same images with "latest"! Probably worth a ticket) | |||||||||||||||||||||||||||||||||||||||||||
| policy | policy-master-csit-distribution policy-master-csit-drools-applications (+frankfurt branches for each) | policy-apex-pdp-master-merge-java policy-distribution-master-merge-java policy-drools-applications-master-merge-java policy-drools-pdp-master-merge-java policy-xacml-pdp-master-merge-java (+frankfurt branches for each - note that these also produce the same images with latest tag; this is not critical though since CSITs do not refer to those) | Dynamic image version resolution based on GERRIT_BRANCH and pom.xmls in the referred branches of the related repositories amended with "SNAPSHOT-latest"
| Same as triggers | Policy components have a lot of dependencies between each other and distinction between what's under test and what's just required is not clear | Policy CSITs and their triggers and image references are currently almost as valid as they can be under current circumstances (the only uncertainty is related to the distinction between images under test and dependency images) | ||||||||||||||||||||||||||||||||||||||||
| sdc (disabled) |
| SDC has moved its legacy TestNG tests under SDC repo already and incorporated them into maven jobs. (where they test the docker image(s) generated in the same build) This needs to be evaluated in Honolulu. | ||||||||||||||||||||||||||||||||||||||||||||
| sdnc | sdnc-master-csit-sdnc_netconf_tls_post_deploy |
| nexus3.onap.org:10001/onap/sdnc-image/1.8-STAGING-latest nexus3.onap.org:10001/onap/sdnc-ansible-server-image/1.8-STAGING-latest nexus3.onap.org:10001/onap/admportal-sdnc-image/1.8-STAGING-latest nexus3.onap.org:10001/onap/sdnc-ueb-listener-image/1.8-STAGING-latest nexus3.onap.org:10001/onap/sdnc-dmaap-listener-image/1.8-STAGING-latest nexus3.onap.org:10001/onap/ccsdk-dgbuilder-image/0.7-STAGING-latest --- nexus3.onap.org:10001/onap/sdnc-image/1.8.3-STAGING-latest The source repo is sdnc/oam | SDNC images are going at version 2.1 in master )sdnc-oam-maven-docker-stage-master) and at 2.0 in Guilin (https://jenkins.onap.org/view/sdnc/job/sdnc-oam-maven-docker-stage-guilin) Latest 1.8-STAGING-latest images are from Aug 31 2020 and 1.8.3-STAGING-latest from Jun 07 2020. These images are from Frankfurt and have no longer any jobs to generate them. Ticket created: | primekey/ejbca-ce:6.15.2.5 nexus3.onap.org:10001/onap/org.onap.aaf.certservice.aaf-certservice-api:latest --- nexus3.onap.org:10001/mariadb:10.1.11 | |||||||||||||||||||||||||||||||||||||||||
| so | so-master-csit-integration-etsi-testing so-master-csit-integration-testing (and the same for frankfurt and elalto; see
|
| nexus3.onap.org:10001/onap/so/catalog-db-adapter:1.3.1 nexus3.onap.org:10001/onap/so/request-db-adapter:1.3.1 nexus3.onap.org:10001/onap/so/sdnc-adapter:1.3.1 nexus3.onap.org:10001/onap/so/openstack-adapter:1.3.1 nexus3.onap.org:10001/onap/so/vfc-adapter::1.3.1 nexus3.onap.org:10001/onap/so/sdc-controller:1.3.1 nexus3.onap.org:10001/onap/so/bpmn-infra:1.3.1 nexus3.onap.org:10001/onap/so/api-handler-infra:1.3.1 --- 1.6.0 tag for integration-etsi-testing (TBD: images to be added) Source repo seems to be so for all of these (to be verified) | 1.3.1 goes all the way back to early Casablanca - these have not been updated since Nov 16 2018! 1.6.0 are early Frankfurt versions (no updates since April 2020)
so-maven-docker-stage-master is at 1.7.3 so-maven-docker-stage-frankfurt is at 1.6.4 so-maven-docker-stage-elalto is at 1.5.4 | TBD - is distinction between images under test and image dependencies clear? | integration-testing suite hasn't been relevant for two years | ||||||||||||||||||||||||||||||||||||||||
| usecases (disabled? but how?) | ||||||||||||||||||||||||||||||||||||||||||||||
| vfc | vfc-gvnfm-vnflcm-master-csit-sanity-check vfc-gvnfm-vnfres-master-csit-sanity-check vfc-nfvo-driver-vnfm-gvnfm-master-csit-sanity-check vfc-nfvo-lcm-master-csit-sanity-check | vfc-gvnfm-vnflcm-master-docker-java-version-shell-daily vfc-gvnfm-vnfres-master-docker-java-version-shell-daily vfc-nfvo-driver-vnfm-gvnfm-master-docker-java-version-shell-daily | nexus3.onap.org:10001/onap/vfc/vnflcm nexus3.onap.org:10001/onap/vfc/vnfres nexus3.onap.org:10001/onap/vfc/gvnfmdriver nexus3.onap.org:10001/onap/vfc/nslcm (all latest by default) Source repositories: | Same as triggers | consul:0.9.3 nexus3.onap.org:10001/onap/msb/msb_discovery nexus3.onap.org:10001/onap/msb/msb_a nexus3.onap.org:10001/library/mariadb redis (!?) It's not clear in all cases which VFC images are under test and which are just necessary dependencies | |||||||||||||||||||||||||||||||||||||||||
| vid |
| nexus3.onap.org:10001/onap/vid:6.0-STAGING-latest Source repo is vid | vid-maven-docker-stage-frankfurt vid-maven-docker-stage-master produces 7.0.0 versions except for STAGING-latest, which is somehow still 6.0 - however, this is problematic as Frankfurt also produces 6.0-STAGING-latest images. Ticket raised:
| nexus3.onap.org:10001/library/mariadb:10 so-simulator (specific to this suite) aai-simulator (specific to this suite) | ||||||||||||||||||||||||||||||||||||||||||
| vfsdk | vnfsdk-refrepo-master-merge-java (for both CSITs; however, this job does not produce or trigger any docker image? Ticket raised:
| nexus3.onap.org:10001/onap/vnfsdk/refrepo/postgres:latest nexus3.onap.org:10001/onap/vnfsdk/refrepo:latest Source repo is vnfsdk/refrepo | vnfsdk-refrepo-maven-docker-stage-master Where is onap/vnfsdk/refrepo/postgres coming from? It hasn't been produced since Jul 09 2019? | What's the relevance of the marketplace CSIT now (given that it's testing postgres image that hasn't been built for over a year)? |
...