PyPi configuration in Nexus3

There are 5 PyPi repositories present in Nexus3:

Pypi (proxy) - This repository is used to proxy https://pypi.python.org/

https://nexus3.onap.org/repository/PyPi/

Pypi.public (group) - This group consolidates PyPi.release and PyPi. This group should be accessed for reading/pulling purposes.

https://nexus3.onap.org/repository/PyPi.public/

Pypi.release (hosted) - This repository hosts our official releases. This repo does not allow re-deployment

https://nexus3.onap.org/repository/PyPi.release/

Pypi.snapshot (hosted) - Used to host our snapshots meanwhile the release candidate is being worked on. This repo allows re-deployment.

https://nexus3.onap.org/repository/PyPi.snapshot/

Pypi.staging (hosted) - Used for our daily/staging non official publications. This repo allows re-deployment.

https://nexus3.onap.org/repository/PyPi.staging/

PyPi configuration in Jenkins and JJB

Projects can leverage this set of templates when they are a python project that needs to upload python packages to a PyPi repository.

These templates are located in the ci-management repo at "ci-management/jjb/onap-jjb/onap-pypi-template.yaml".

There are currently 4 templates available:

{project-name}-python-staging-{stream}
{project-name}-python-release-{stream}
{project-name}-{subproject}-python-staging-{stream}
{project-name}-{subproject}-python-release-{stream}

General Job Information

The "staging" jobs will trigger a Jenkins job that builds the python package and pushes the package to a staging repo located here https://nexus3.onap.org/repository/PyPi.staging/.

The staging jobs can be triggered by the following events:

A gerrit merge event
A comment on a gerrit patch of "pypi-remerge"

The "release" jobs will trigger a similar job that builds the python package and pushes it to the release repo located here https://nexus3.onap.org/repository/PyPi.release/.

The release jobs can be triggered by the following event:

A comment on a gerrit patch of "pypi-release"

Both jobs require the following variables to be defined:

project-name
stream
tox-dir (optional, default: '')
tox-envs (optional, default: '')
python-version

Optional params

tox-dir typically only need to be specified for subprojects. You can specify it though if your tox.ini is somewhere other than the root of your project. tox-envs are set to blank always so they do not overwrite your tox.ini; Setting them blank gives the option to set them if you manually kick off a job for troubleshooting purposes though.

Python projects with a subproject

Some projects have a structure where one repo contains several projects. LF has built templates to support them as well; you will use

these templates:

{project-name}-{subproject}-python-staging-{stream}
{project-name}-{subproject}-python-release-{stream}

Here is a concrete example of how to set up pypi jobs for a project with subprojects using dcaegen2/utils.

Example.yaml

---
- project:
    name: dcaegen2-utils
    project-name: 'dcaegen2-utils'
    python-version: python3
    project: 'dcaegen2/utils'
    stream:
      - 'master':
          branch: 'master'
    mvn-settings: 'dcaegen2-utils-settings'
    # due to a strange macro / variable translation problem this needs
    # to be passed as a string block to properly get the properties
    # correctly defined in the job
    maven-deploy-properties: |
      deployAtEnd=true
    files: '**'
    archive-artifacts: ''
    subproject:
      - 'dcaeapplib':
          tox-dir: dcaeapplib/
          tox-envs: ''
      - 'onap-dcae-dbs-docker-client':
          tox-dir: onap-dcae-dbs-docker-client/
          tox-envs: ''
      - 'onap-dcae-dcaepolicy-lib':
          tox-dir: onap-dcae-dcaepolicy-lib/
          tox-envs: ''
      - 'python-discovery-client':
          tox-dir: python-discovery-client/
          tox-envs: ''
      - 'python-dockering':
          tox-dir: python-dockering/
          tox-envs: ''

    jobs:
      - '{project-name}-{subproject}-python-staging-{stream}'
      - '{project-name}-{subproject}-python-release-{stream}'

   
    build-node: 'ubuntu1604-docker-8c-8g'

Note that the use of these templates implies a workflow where the staging artifact will be overwritten on each merge event. We are working on features that will allow immutable staging as well.

ONAP Wiki

How to use PyPi JJB Templates