...

1. Developer pushes a commit to Gerrit
2. Gerrit triggers review verification job in Jenkins
   • The template templates for review verification job is are in ci-management/jjb/integration/integration-templates-csit.yaml and currently , named as "{project-name}-unified-reviewreview-verification-<artifact type>-{stream}-PoC""
     • The only currently supported artifact type in review verification templates is maven 
   • The trigger is "gerrit-trigger-patch-submitted" and it is activated by any file change in the project
   Review verification build triggers
   • the job starts with artifact build
   jobDocker build job builds snapshot docker image(s) and pushes them to ONAP Snapshot Nexus3 (nexus3.onap.org)
   • • Artifacts should be created from the patch under review
     , unit tests executed and Sonar analysis performed here
3. Optionally, the artifacts may be pushed to Onap Nexus (nexus.onap.org)
4. Review verification job triggers docker build job
   • The value of BUILD_ID Jenkins variable is passed to the docker build job as UNIQUE_DOCKER_TAG (along with PROJECT, BRANCH and GERRIT_REFSPEC)
   • The triggered build is defined as  "{project-name}-{stream}-{artifact-type}-docker-snapshot-PoC" in the review verification job template
   • The build is blocking and fails the entire review verification job before moving it moves on to the next step on failure
   • The only currently existing docker build job template that follows the expected naming pattern is defined in ci-management/jjb/global-templates-golang.yaml as "{project-name}-{stream}-golang-docker-snapshot-PoC"
5. Optionally, the artifacts to be used can be pulled from ONAP Nexus (nexus.onap.org)
6. • • Currently the first step just compiles the artifacts locally (with unit tests), but additional operations - for example sonar analysis - should be considered here
7. Docker build is started
8. Docker build step builds docker image(s) to local temporary docker repository
   • The docker build scripts and the created artifacts (either pulled from artifact Nexus in the previous step or created here on the fly) are completely based on the submitted patch in Gerrit
   • The actual build script is specific to the project and has to be specified in the specific docker build job definition
   • The build should tag the relevant docker image(s) with <version>-SNAPSHOT-<unique docker tag> using the UNIQUE_DOCKER_TAG value given as parameter
   Review verification job triggers CSIT job
   • The value of BUILD_ID Jenkins variable is passed to the docker build job as UNIQUE_DOCKER_TAG (along with PROJECT, BRANCH and GERRIT_REFSPEC)
   • The triggered build is defined as "{project-name}-{stream}-project-csit-PoC" in the review verification job template
   • The build is blocking and fails the entire review verification job on failure
   • The CSIT job template is defined in ci-management/jjb/integration/integration-templates-csit.yaml as "{project-name}-{stream}-project-csit-PoC"the template differentiate local image from any that could be found from Nexus3
     • Missing repository prefix in the image name is an obvious hint 
9. CSIT job is started
   
   • this step calls csit/run-project-csit.sh under the project's root directory via integration-run-project-test builder defined in ci-management/integration/integration-macros.yaml
   • run-project-csit.sh is project-specific, but can take advantage of common scripts in integration/csit if considered useful
   • run-project-csit.sh can and should run all the test plans of the project in one go and provide the results under csit/archives directory
10. CSIT job pulls the docker images created in step 74. (by using project-specific setup scripts) and tests them
    • The test cases to be executed are also from the patch under review
    • the setup script responsible of pulling the required images should ensure that the images are local
      • To be considered: should CSIT ever be able to pull the images under test from Nexus?
11. Review verification job gives +1 or -1 verify vote to the review based on the overall success of the verification
12. Once the commit is approved (+2'd) and submitted, Gerrit triggers merge verification job in Jenkins
    • The template for merge verification job is in ci-management/jjb/integration/integration-templates-csit.yaml and currently named as "{project-name}-merge-unifiedverification-merge<artifact type>-{stream}-PoC"
    • The only currently supported artifact type in merge verification templates is maven 
    • The trigger is "gerrit-trigger-patch-merged" and it is activated by any file change in the project
    Merge verification build triggers
    • the job starts with artifact build
    jobDocker build job builds snapshot
    • • Artifacts should be created
      from the master, unit tests executed and Sonar analysis performed here
13. Optionally, the artifacts may be pushed to Onap Nexus (nexus.onap.org)
14. Merge verification job triggers docker build job in exactly the same way as review verification job triggers it in step 7.
15. Optionally, the artifacts to be used can be pulled from ONAP Nexus (nexus.onap.org)
16. • • from master
      • Sonar analysis and other possible additional artifact-related steps should be considered here
17. Docker build is started
18. Docker build step builds docker image(s) and pushes them to ONAP Snapshot Nexus3 (nexus3.onap.org)The procedure is the same as in step 7. except for the fact that the to local temporary docker repository
    • The docker build scripts and the created artifacts are completely based on master
    Merge verification job triggers CSIT job
    • The actual build script is the same as in review verification and used in the same way
19. CSIT is started in exactly the same way as review verification job triggers it in step 8.
20. CSIT job pulls the docker images created in step 10. (by using project-specific setup scripts) and tests them
21. Merge verification job triggers docker image staging job
    • The value of BUILD_ID Jenkins variable is passed to the docker build job as UNIQUE_DOCKER_TAG (along with PROJECT, BRANCH and GERRIT_REFSPEC)
    • The triggered build is defined as "{project-name}-{stream}-docker-staging-PoC" in the merge verification job template
    • The staging job template is defined in ci-management/jjb/global-templates-docker.yaml as "{project-name}-{stream}-docker-staging-PoC"
      • The actual staging script is specific to the project and has to be specified in the specific docker build job definition
22. Docker image staging job pulls the docker image(s) built in step 10.
23. Docker image staging job Execution moves to staging steps after successful CSIT
24. Artifact staging step pushes the artifacts to Nexus2 staging repository
    • ...if there is any actual need for this step? What are the staged artifacts actually used for (and where and how)?
25. Docker image staging step tags the docker images(s) with <version>-STAGING-latest
26. ...and pushes them to ONAP Staging Nexus Nexus3
    • should that be staging or snapshot?

Requirements and recommendations

...

• The project must define a review verification job based on "{project-name}-unified-review-verification-<artifact type>-{stream}-PoC" template (which is defined in ci-management/jjb/integration/integration-templates-csit.yaml)
  • the only currently supported artifact type is maven - new templates must be created to support different artifact types (such as golang)
• The following JJB variables must be defined in the project definition that defines the review verification job:
  • project-name (project identifier that can be freely chosen)
  • project (should correspond project's name in Gerrit)
  • stream (list of supported branches)
  • artifact-type (should match the type of docker snapshot PoC template)
  • java-version
  • mvn-version
  • mvn-settings
  • mvn-global-settings
• The project must define a job for building snapshot docker images from artifacts of given artifact-type based on any template following naming convention "{project-name}-{stream}-{artifact-type}-docker-snapshot-PoC"
  • only golang available at the moment! New templates for java, python etc. should be added to support more projects
  • The docker job definition must specify the project-specific script or method that
    • builds the artifacts
    • puts the artifacts into docker image(s)
    • tags the docker image(s) with <version>-SNAPSHOT-<unique docker tag> where UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) is used for the last part
    • pushes the docker image(s) to ONAP snapshot Nexus
  The project must define a job for project-specific CSIT based on "{project-name}-{stream}-project-csit-PoC" template defined in ci-management/jjb/integration/integration-templates-csit.yamlThe project must support CSIT as follows:
  • The project must have csit/run-project-csit.sh that is responsible of the entire CSIT executionrun-project-csit.sh should use the UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) to specify the exact docker images to pull and run in startup phase
  • The setup scripts should be able to use locally built docker images
  • Following the integration/csit subdirectory structure and using its run-csit.sh (by cloning the integration/csit and copying the common scripts to workspace) to execute the actual tests is recommended (at least for existing Robot CSITs) is possible but not mandatory
• The project should define a merge verification job based on "{project-name}-unified-merge-{stream}-PoC" template (which is defined in ci-management/jjb/integration/integration-templates-csit.yaml)
• The project should define a job script for tagging verified snapshot local docker images as STAGING based on "{project-name}-{stream}-{artifact-type}-docker-staging-PoC" template (in ci-management/jjb/global-templates-docker.yaml)
  The docker job definition
  • The project-specific verification job definitions must specify the project-specific script or method that
    
    • pulls tags the verified snapshot docker images identified by the UNIQUE_DOCKER_TAG environment variable (passed down as Jenkins job input parameter) from ONAP snapshot Nexustags the local, verified docker image(s) with <version>-STAGING-latest
    • pushes the docker image(s) to ONAP snapshot NexusNexus3

Example

...

Music PoC

...

project

...

• ci-management/jjb/
  • global-templates-docker.yaml
  • global-templates-golang.yaml
• ci-management/jjb/integration/
  • integration-macros.yaml
  • integration/integration-templates-csit.yaml
•  ci-management/jjb/music/
  • music-distributed-kv-store.yaml
  • music-distributed-kv-store-csit.yaml
  • build-music-distributed-kv-store-image.sh
  • tag-music-distributed-kv-store-image-staging.sh

...

CCSDK/distribution

First actively used example of this design (excluding artifact build jobs) is now available

...

Open questions and remaining tasks

• There are only two snapshot docker build templates currently available
  • maven
  • golang
• Is BUILD_ID the best possible identifier to be used for unique docker tag?Most docker builds already use unique timestamp to identify the images, but using it between jobs is more complex (especially in case of umbrella jobs) Is there any need for SNAPSHOT docker images in Nexus3?
• Is there any need for staging artifacts in Nexus? Who needs them, where and how? How does the staging repository even work?
• Should CSIT be able to test docker images from Nexus? If so, under which circumstances and for what purpose?
• Currently there are verification templates available only for artifact type
  • maven
• This proposal replaces plain Maven/Makefile/other artifact build jobs completely
  • Sonar builds seem to be completely separate scheduled builds anyway, and unsupported by any templates?
  • Ideally all reviews should start with Sonar build and fail the review on violations