PyPi configuration in Nexus3
There are 5 PyPi repositories present in Nexus3:
Pypi (proxy) - This repository is used to proxy https://pypi.python.org/
https://nexus3.onap.org/repository/PyPi/
Pypi.public (group) - This group consolidates PyPi.release and PyPi. This group should be accessed for reading/pulling purposes.
https://nexus3.onap.org/repository/PyPi.public/
Pypi.release (hosted) - This repository hosts our official releases. This repo does not allow re-deployment
https://nexus3.onap.org/repository/PyPi.release/
Pypi.snapshot (hosted) - Used to host our snapshots meanwhile the release candidate is being worked on. This repo allows re-deployment.
https://nexus3.onap.org/repository/PyPi.snapshot/
Pypi.staging (hosted) - Used for our daily/staging non official publications. This repo allows re-deployment.
https://nexus3.onap.org/repository/PyPi.staging/
PyPi configuration in Jenkins and JJB
Projects can leverage this set of templates when they are a python project that needs to upload python packages to a PyPi repository.
These templates are located in the ci-management repo at "ci-management/jjb/onap-jjb/onap-pypi-template.yaml".
There are currently 4 templates available:
- {project-name}-python-staging-{stream}
- {project-name}-python-release-{stream}
- {project-name}-{subproject}-python-staging-{stream}
- {project-name}-{subproject}-python-release-{stream}
General Job Information
The "staging" jobs will trigger a Jenkins job that builds the python package and pushes the package to a staging repo located here https://nexus3.onap.org/repository/PyPi.staging/.
The staging jobs can be triggered by the following events:
- A gerrit merge event
- A comment on a gerrit patch of "pypi-remerge"
The "release" jobs will trigger a similar job that builds the python package and pushes it to the release repo located here https://nexus3.onap.org/repository/PyPi.release/.
The release jobs can be triggered by the following event:
- A comment on a gerrit patch of "pypi-release"
Both jobs require the following variables to be defined:
- project-name
- stream
- tox-dir (optional, default: '')
- tox-envs (optional, default: '')
Optional params
tox-dir typically only need to be specified for subprojects. You can specify it though if your tox.ini is somewhere other than the root of your project. tox-envs are set to blank always so they do not overwrite your tox.ini; Setting them blank gives the option to set them if you manually kick off a job for troubleshooting purposes though.
Python projects with a subproject
Some projects have a structure where one repo contains several projects. LF has built templates to support them as well; you will use
these templates:
- {project-name}-{subproject}-python-staging-{stream}
- {project-name}-{subproject}-python-release-{stream}
Here is a concrete example of how to set up pypi jobs for a project with subprojects using dcaegen2/utils.
---
- project:
name: dcaegen2-utils
project-name: 'dcaegen2-utils'
python-version: python3
project: 'dcaegen2/utils'
stream:
- 'master':
branch: 'master'
mvn-settings: 'dcaegen2-utils-settings'
# due to a strange macro / variable translation problem this needs
# to be passed as a string block to properly get the properties
# correctly defined in the job
maven-deploy-properties: |
deployAtEnd=true
files: '**'
archive-artifacts: ''
subproject:
- 'dcaeapplib':
tox-dir: dcaeapplib/
tox-envs: ''
- 'onap-dcae-dbs-docker-client':
tox-dir: onap-dcae-dbs-docker-client/
tox-envs: ''
- 'onap-dcae-dcaepolicy-lib':
tox-dir: onap-dcae-dcaepolicy-lib/
tox-envs: ''
- 'python-discovery-client':
tox-dir: python-discovery-client/
tox-envs: ''
- 'python-dockering':
tox-dir: python-dockering/
tox-envs: ''
jobs:
- '{project-name}-{subproject}-python-staging-{stream}'
- '{project-name}-{subproject}-python-release-{stream}'
build-node: 'ubuntu1604-docker-8c-8g'
Related articles