...

• The project must define a review verification job based on "{project-name}-unified-review-{stream}-PoC" template (which is defined in ci-management/jjb/integration/integration-templates-csit.yaml)
• The following JJB variables must be defined in the project definition that defines the review verification job:
  • project-name (project identifier that can be freely chosen)
  • project (should correspond project's name in Gerrit)
  • stream (list of supported branches)
  • artifact-type (should match the type of docker snapshot PoC template)
  • java-version
  • mvn-version
  • mvn-settings
  • mvn-global-settings
• The project must define a job for building snapshot docker images from artifacts of given artifact-type based on any template following naming convention "{project-name}-{stream}-{artifact-type}-docker-snapshot-PoC"
  • only golang available at the moment! New templates for java, python etc. should be added to support more projects
  • The docker job definition must specify the project-specific script or method that
    • builds the artifacts
    • puts the artifacts into docker image(s)
    • tags the docker image(s) with <version>-SNAPSHOT-<unique docker tag> where UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) is used for the last part
    • pushes the docker image(s) to ONAP snapshot Nexus
• The project must define a job for project-specific CSIT based on "{project-name}-{stream}-project-csit-PoC" template defined in ci-management/jjb/integration/integration-templates-csit.yaml
  • The project must have csit/run-project-csit.sh that is responsible of the entire CSIT execution
  • run-project-csit.sh should use the UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) to specify the exact docker images to pull and run in startup phase
  • Following the integration/csit subdirectory structure and using its run-csit.sh (by cloning the integration/csit and copying the common scripts to workspace) to execute the actual tests is recommended (at least for existing Robot CSITs) but not mandatory
• The project should define a merge verification job based on "{project-name}-unified-merge-{stream}-PoC" template (which is defined in ci-management/jjb/integration/integration-templates-csit.yaml)
• The project should define a job for tagging verified snapshot docker images as STAGING based on "{project-name}-{stream}-{artifact-type}-docker-staging-PoC" template (in ci-management/jjb/global-templates-docker.yaml)
  • The docker job definition must specify the project-specific script or method that
    • pulls the verified snapshot docker images identified by the UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) from ONAP snapshot Nexus
    • tags the docker image(s) with <version>-STAGING-latest
    • pushes the docker image(s) to ONAP snapshot Nexus

Example

...

projects

Music PoC

PoC implementation of the above design is available in music project for reference - note though that the CSIT is currently disabled and that artifact build jobs are not yet supported by the templates

• All the related job and template changes done to ci-management repository are available in master, see
  • ci-management/jjb/
    • global-templates-docker.yaml
    • global-templates-golang.yaml
  • ci-management/jjb/integration/
    • integration-macros.yaml
    • integration/integration-templates-csit.yaml
  •  ci-management/jjb/music/
    • music-distributed-kv-store.yaml
    • music-distributed-kv-store-csit.yaml
    • build-music-distributed-kv-store-image.sh
    • tag-music-distributed-kv-store-image-staging.sh
• The changes to music repository itself are available in open Gerrit review in https://gerrit.onap.org/r/admin/crepos/music/distributed-kv-store/+/114637
  
• The entire csit subdirectory
changes to docker

• the entire csit subdirectory
• changes to docker scripts to support unique tagging and staging

CCSDK/distribution

First actively used example of this design (excluding artifact build jobs) is now available

• All the related job and template changes done to ci-management repository are available in master
  • ci-management/jjb/integration/
    • integration-maven-jobs.yaml
    • integration/integration-templates-csit.yaml
  •  ci-management/jjb/ccsdk/
    • distribution.yaml
    • ccsdk-csit.yaml
    • tag-ccsdk-distribution-images-staging.sh
• The changes to ccsdk/distribution repository itself are available under https://gerrit.onap.org/r/admin/repos/ccsdk/distribution
  
  • the entire csit subdirectory
  • scripts to support unique tagging and staging

Open questions and remaining tasks

• The There are only two snapshot docker build template templates currently available is for golang
  • templates for java, python etc. should be added if no common template is feasible
  Do we need umbrella jobs for review and merge verification, or should we trigger docker jobs and chain CSIT to them directly?
  Making changes to the overall structure might be more complex without umbrella jobs
  • maven
  • golang
• Is BUILD_ID the best possible identifier to be used for unique docker tag?
  • Most docker builds already use unique timestamp to identify the images, but using it between jobs is more complex (especially in case of umbrella jobs) 
• This proposal replaces plain Maven/Makefile/other artifact build jobs completely
  • Sonar builds seem to be completely separate scheduled builds anyway, and unsupported by any templates?
  • Ideally all reviews should start with Sonar build and fail the review on violations