This page describes steps taken and issues encountered in transitioning ONAP project to use small, multi-platform images
dmaap
datarouter
This section describes the migration of datarouter to small, Issues encountered:
1) docker-maven-plugin. Original spotify plugin not supported on Arm, changed to fabric8
2) docker pull/push repos default nexus/onap does not support multi-platform images.
The datarouter project generates 3 docker images:
datarouter-prov
datarouter-node
datarouter-subscriber
Each sub-project is structured similarly, therefore the changes to each to migrate to a small,
multi-platfrom image are similar and involve the following steps:
1. update pom to migrate from spotify docker-maven-plug to fabri8 docker-maven-plugin
2. create Dockerfile
#### Step 1: build and run on Intel (Mac Air)
> git clone ssh://sandovalfrOAM@gerrit.onap.org:29418/dmaap/datarouter && scp -p -P 29418 sandovalfrOAM@gerrit.onap.org:hooks/commit-msg datarouter/.git/hooks/
> cd datarouter
> mvn clean install -DskipTests -Pdocker
> docker images
nexus3.onap.org:10003/onap/dmaap/datarouter-subscriber latest b163a167f8c1 24 seconds ago 672MB
nexus3.onap.org:10003/onap/dmaap/datarouter-node latest eaadf6b93bcf 38 seconds ago 674MB
nexus3.onap.org:10003/onap/dmaap/datarouter-prov latest 8c9adc9a9851 About a minute ago 704MB
java 8 d23bdf5b1b1b 2 years ago 643MB
TEST
> cd datarouter-docker-compose/src/main/resources
- edit docker-compose, change nexus 0001 (remote pull repo) to 0003 (local build)
> docker-compose up
- terminal 2
> docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c193317ec860 nexus3.onap.org:10003/onap/dmaap/datarouter-node "sh startup.sh" About a minute ago Up About a minute 0.0.0.0:9090->8080/tcp, 0.0.0.0:9443->8443/tcp datarouter-node
e8dab741550e
a) fabric8 docker-maven-plugin allows setting from command line
b) or, build machine must set in ~/.m2/settings/xml
3) base image shell differences (alpine is not bash) impacts entrypoint and scripts
- for containers that rely heavily on scripts, e.g. dmaap/kafak11aaf add bash in image
4) docker-compose.yml must be configured for local dev
- refer to local images, not nexus
- includes dependencies that are not multi-platform
eg dmaap/buscontroller crunchydata/crunchy-postgres:centos7-10.4-2.0.0
5) Dockerfile defines docker USER root. Not recommended by ONAP
7) image naming is inconsistent among projects, some append registry to name, e.g. buscontroller
- appends <onap.nexus.dockerregistry.daily>, other's simply name with onap/<project><name> e.g
messageservice
- recommendation: use simple names and push registry definition to target push repo
datarouter
This section describes the migration of datarouter to small, multi-platform images.
The datarouter project generates 3 docker images:
datarouter-prov
datarouter-node
datarouter-subscriber
Each sub-project is structured similarly, therefore the changes to each to migrate to a small,
multi-platfrom image are similar and involve the following steps:
1. update pom to migrate from spotify docker-maven-plug to fabri8 docker-maven-plugin
2. create Dockerfile
#### Step 1: build and run on Intel (Mac Air)
> git clone ssh://sandovalfrOAM@gerrit.onap.org:29418/dmaap/datarouter && scp -p -P 29418 sandovalfrOAM@gerrit.onap.org:hooks/commit-msg datarouter/.git/hooks/
> cd datarouter
> mvn clean install -DskipTests -Pdocker
> docker images
nexus3.onap.org:10003/onap/dmaap/datarouter-subscriber latest b163a167f8c1 24 seconds ago 672MB
nexus3.onap.org:10003/onap/dmaap/datarouter-node latest eaadf6b93bcf 38 seconds ago 674MB
nexus3.onap.org:10003/onap/dmaap/datarouter-prov "sh startup.sh" About a minute ago Up About a minute (healthy) 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 0.0.0.0:443->8443/tcp datarouter-prov
cf0e996f0f31 latest 8c9adc9a9851 About a minute ago 704MB
java 8 d23bdf5b1b1b 2 years ago 643MB
TEST
> cd datarouter-docker-compose/src/main/resources
- edit docker-compose, change nexus 0001 (remote pull repo) to 0003 (local build)
> docker-compose up
- terminal 2
> docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c193317ec860 nexus3.onap.org:10003/onap/dmaap/datarouter-subscriber node "sh startup.sh" About a minute ago Up About a minute 8080/tcp, 0.0.0.0:70709090->7070>8080/tcp, 8443/tcp subscriber0.0.0.0:9443->8443/tcp datarouter-node
73affb6364f9 mariadb:10.2.14 "docker-entrypoint.s…e8dab741550e nexus3.onap.org:10003/onap/dmaap/datarouter-prov "sh startup.sh" About a minute ago Up About a minute (healthy) 0.0.0.0:33068080->3306/tcp mariadb> docker exec -it datarouter-node /bin/bash
# curl http://dmaap-dr-prov:8080/internal/prov
> docker exec -it datarouter-prov /bin/bash
# curl -v -X POST -H "Content-Type : application/vnd.dmaap-dr.feed" -H "X-DMAAP-DR-ON-BEHALF-OF: rs873m" --data-ascii @/opt/app/datartr/addFeed3.txt --post301 --location-trusted -k https>8080/tcp, 0.0.0.0:8443->8443/tcp, 0.0.0.0:443->8443/tcp datarouter-prov
cf0e996f0f31 nexus3.onap.org:10003/onap/dmaap/datarouter-subscriber "sh startup.sh" About a minute ago Up About a minute 8080/tcp, 0.0.0.0:7070->7070/tcp, 8443/tcp subscriber-node
73affb6364f9 mariadb:10.2.14 "docker-entrypoint.s…" About a minute ago Up About a minute (healthy) 0.0.0.0:3306->3306/tcp mariadb
> docker exec -it datarouter-node /bin/bash
# curl http://dmaap-dr-prov:8443
#### Step 2: migrate to alpine
- edit datarouter-[prov, node, subscriber]/pom.xml
+ <baseImage>openjdk:8-jre-alpine</baseImage>
+ <entryPoint>["sh", "/startup.sh"]</entryPoint
- note shell for openjdk:8-jre-alpine is sh, therefore entrypoint = ["sh", "startup.sh"]
> mvn clean install -DskipTests -Pdocker
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nexus3.onap.org:10003/onap/dmaap/datarouter-subscriber latest 03ea672a6d73 28 seconds ago 102MB
nexus3.onap.org:10003/onap/dmaap/datarouter-node latest 76345553e1ec 43 seconds ago 118MB
nexus3.onap.org:10003/onap/dmaap/datarouter-prov latest db1d6afe5c3f About a minute ago 148MB
nexus3.onap.org:10001/openjdk 8-jre-alpine 1b46cc2ba839 3 weeks ago 85MB
- Test as above, Note change to "/bin/sh" per alpine
> docker exec -it datarouter-node /bin/sh
# curl http://dmaap-dr-prov:8080/internal/prov
> docker exec -it datarouter-prov /bin/sh
# curl -v -X POST -H "Content-Type : application/vnd.dmaap-dr.feed" -H "X-DMAAP-DR-ON-BEHALF-OF: rs873m" --data-ascii @/opt/app/datartr/addFeed3.txt --post301 --location-trusted -k https://dmaap-dr-prov:8443
#### Step 3: migrate pom to fabric8 docker-maven-plugin
- Note: spotify docker-maven-8080/internal/prov
> docker exec -it datarouter-prov /bin/bash
# curl -v -X POST -H "Content-Type : application/vnd.dmaap-dr.feed" -H "X-DMAAP-DR-ON-BEHALF-OF: rs873m" --data-ascii @/opt/app/datartr/addFeed3.txt --post301 --location-trusted -k https://dmaap-dr-prov:8443
#### Step 2: migrate to alpine
- edit datarouter-[prov, node, subscriber]/pom.xml
+ <baseImage>openjdk:8-jre-alpine</baseImage>
+ <entryPoint>["sh", "/startup.sh"]</entryPoint
- note shell for openjdk:8-jre-alpine is sh, therefore entrypoint = ["sh", "startup.sh"]
> mvn clean install -DskipTests -Pdocker
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nexus3.onap.org:10003/onap/dmaap/datarouter-subscriber latest 03ea672a6d73 28 seconds ago 102MB
nexus3.onap.org:10003/onap/dmaap/datarouter-node latest 76345553e1ec 43 seconds ago 118MB
nexus3.onap.org:10003/onap/dmaap/datarouter-prov latest db1d6afe5c3f About a minute ago 148MB
nexus3.onap.org:10001/openjdk 8-jre-alpine 1b46cc2ba839 3 weeks ago 85MB
- Test as above, Note change to "/bin/sh" per alpine
> docker exec -it datarouter-node /bin/sh
# curl http://dmaap-dr-prov:8080/internal/prov
> docker exec -it datarouter-prov /bin/sh
# curl -v -X POST -H "Content-Type : application/vnd.dmaap-dr.feed" -H "X-DMAAP-DR-ON-BEHALF-OF: rs873m" --data-ascii @/opt/app/datartr/addFeed3.txt --post301 --location-trusted -k https://dmaap-dr-prov:8443
#### Step 3: migrate pom to fabric8 docker-maven-plugin
- Note: spotify docker-maven-plugin does not run on Arm
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:1.0.0:build (default-cli) on project datarouter-prov: Exception caught: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException:
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider: ExceptionInInitializerError: Can't overwrite cause with java.lang.UnsatisfiedLinkError: cannot determine CPU -> [Help 1]
- See: https://github.com/spotify/docker-maven-plugin/issues/233
- Note: spotify dockerfile-maven-plugin is alternate to fabric8, but other ONAP teams are using fabric8 so there
appears to be no compelling reason to stick w spotify
- Note: docker pull repo must be managed
- edit datarouter-[prov, node, subscriber]/pom.xml
Note that onap ~/.m2/settings.xml defines
<docker.pull.registry>nexus3.onap.org:10001</docker.pull.registry>
which forces an amd64 specific image.
to support other platforms, add to docker-maven-plugin configuration:
pullRegistry>${docker.pull.registry}</pullRegistry>
and invoke mvn with platfrom specific pull repo, e.g. on Arm
> mvn install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- create datarouter-[prov, node, subscriber]/src/main/resources/docker/Dockerfile
...
- throws 'broken pipe' error which is amisdirection to registry naming issue
- See https://github.com/spotify/dockerfile-maven/issues/95
- maven install error
- See: https://github.com/spotify/dockerfile-maven/issues/252
- upgrade Arm server to maven 3.6.0
- see: https://maven.apache.org/users/index.html
- requires multiple execution blocks for multiple tags
- see https://github.com/spotify/dockerfile-maven/issues/10://github.com/spotify/dockerfile-maven/issues/95
- maven install error
- See: https://github.com/spotify/dockerfile-maven/issues/252
- upgrade Arm server to maven 3.6.0
- see: https://maven.apache.org/users/index.html
- requires multiple execution blocks for multiple tags
- see https://github.com/spotify/dockerfile-maven/issues/10
DMAAP - messageservice
## dmaap-mr
Gerrit Project dmaap/messagerouter/messageservice
INT-910, INT-932
to test mr requires kafka11aaf
### Step 1: Build and run on local machine (Mac Air)
- in terminal
> git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice && (cd messageservice && curl -Lo `git rev-parse --git-dir`/hooks/commit-msg https://gerrit.onap.org/r/tools/hooks/commit-msg; chmod +x `git rev-parse --git-dir`/hooks/commit-msg)
> cd messageservice
- See docs/installation/installation.rst
- edit MsgRtrApi.properties
config.zk.servers=zookeeper
kafka.metadata.broker.list=kafka:9092
- docker-compose network maps service name(zookeeper, kafka) to IP
> cp bundleconfig-local/etc/appprops/MsgRtrApi.properties /var/tmp
- configure docker->properties->file sharing to include /var/tmp
> mvn clean install -DskipTests -Pdocker
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190304T195732Z ec37060b0ef0 4 minutes ago 782MB
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest ec37060b0ef0 4 minutes ago 782MB
onap/dmaap/dmaap-mr latest ec37060b0ef0 4 minutes ago 782MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190304T195732Z ec37060b0ef0 4 minutes ago 782MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest ec37060b0ef0 4 minutes ago 782MB
openjdk 8-jdk 5f8e49b0a018 3 weeks ago 624MB
- build kafka11aaf image (see separate guide)
> cd src/main/resources/docker-compose
- create docker-compose-local.yml
image: zookeeper
image: onap/dmaap/kafka01101:latest
image: onap/dmaap/dmaap-mr:latest
> docker-compose -f docker-compose-local.yml up -d
Test
- create sample file (any arbitrary content)
> curl -H "Content-Type:text/plain" -X POST -d @sample.txt http://localhost:3904/events/TestTopic1
> curl -H "Content-Type:text/plain" -X GET http://localhost:3904/events/TestTopic1/CG1/C1?timeout=1000
### Step 2: migrate to alpine
- edit messageservice/src/main/resources/Dockerfile
- from openjdk:8-jdk to openjdk:8-alpine
> mvn clean install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T122553Z e8fbf3bebd9b 46 seconds ago 244MB
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest e8fbf3bebd9b 46 seconds ago 244MB
onap/dmaap/dmaap-mr latest e8fbf3bebd9b 46 seconds ago 244MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T122553Z e8fbf3bebd9b 46 seconds ago 244MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest e8fbf3bebd9b 46 seconds ago 244MB
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T121402Z 62f0a2a98c62 12 minutes ago 244MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T121402Z 62f0a2a98c62 12 minutes ago 244MB
openjdk 8-jre-alpine 1b46cc2ba839 4 weeks ago 85MB
> docker-compose -f docker-compose-local.yml up
Test as step 1
#### Step 3: migrate pom to fabric8 docker-maven-plugin
- Note: spotify docker-maven-plugin does not run on Arm
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:1.0.0:build (default-cli) on project datarouter-prov: Exception caught: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException:
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider: ExceptionInInitializerError: Can't overwrite cause with java.lang.UnsatisfiedLinkError: cannot determine CPU -> [Help 1]
- See: https://github.com/spotify/docker-maven-plugin/issues/233
- Note: spotify dockerfile-maven-plugin is alternate to fabric8, but other ONAP teams are using fabric8 so there
appears to be no compelling reason to stick w spotify
- Note: docker pull/push repos must be managed
- edit messageservice/pom.xml
- add pull repo defintion
Note that onap ~/.m2/settings.xml defines
<docker.pull.registry>nexus3.onap.org:10001</docker.pull.registry>
which forces an amd64 specific image.
to support other platforms, add to docker-maven-plugin configuration:
<pullRegistry>${docker.pull.registry}</pullRegistry>
and invoke mvn with appropriate pull repo if building on non-Intel machine, e.g. on Arm
> mvn install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- add push repo definition
add <pushRegistry>${docker.push.registry}</pushRegistry>
push to local registry to test*
- create src/main/resources/docker/Dockerfile
> mvn install -DskipTests -Pdocker -Ddocker.push.registry=localhost:5000 docker:push
- Test, As per step 2
#### Step 4: build and run on Arm
- terminal on Arm machine
> git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice
- copy modified files from workstation to Arm server
$ scp pom.xml root@147.75.94.202:/root/onap/dmaap/messageservice
$ scp src/main/resources/docker/Dockerfile root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker
$ scp src/main/resources/docker-compose/docker-compose-local.yml root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker-compose
$ scp src/main/resources/docker-compose/MsgRtrApi.properties root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker-compose
> mvn clean install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
> cd src/main/resources/docker-compose
> docker-compose -f docker-compose-local.yml up -d
test as above
DMAAP KAFKA11aaf
## dmaap kafka
INT-933, INT-912
### Step 1: Build and run on local machine (Mac Air)
- in terminal
> git clone https://gerrit.onap.org/r/dmaap/kafka11aaf && (cd kafka11aaf && curl -Lo `git rev-parse --git-dir`/hooks/commit-msg https://gerrit.onap.org/r/tools/hooks/commit-msg; chmod +x `git rev-parse --git-dir`/hooks/commit-msg)
> cd kafka11aaf
> mvn clean install -P docker docker:build
> docker images
...
onap/dmaap/kafka01101 latest 6c10dd9e9095 About a minute ago 380MB
anapsix/alpine-java latest c45785c254c5 2 weeks ago 126MB
- Test
- no directions to test, ie no docker-compose.yml, README, etc.
> test via messageservice
### Step 2: migrate to alpine
- edit kafka11aaf/pom.xml
- change anapsix/alpine-java to openjdk:8-alpine
> mvn clean install -P docker docker:build
> docker images
...
onap/dmaap/kafka01101 latest 5bfd9b168e39 53 seconds ago 439MB
openjdk 8-alpine 792ff45a2a17 4 weeks ago 105MB
Note that images are LARGER! change to multi-platform normative base image openjdk:8-alpine required nevertheless
- test as in step 1
#### Step 3: migrate pom to fabric8 docker-maven-plugin
- Note: spotify docker-maven-plugin does not run on Arm
[ERROR] Failed to execute goal com.spotify:docker-maven-plugin:1.0.0:build ...jnr.ffi.provider.jffi.Provider: ExceptionInInitializerError: Can't overwrite cause with java.lang.UnsatisfiedLinkError: cannot determine CPU -> [Help 1]
- See: https://github.com/spotify/docker-maven-plugin/issues/233
- Note: spotify dockerfile-maven-plugin is alternate to fabric8, but other ONAP teams are using fabric8 so there
appears to be no compelling reason to stick w spotify
- Note: docker pull/push repos must be managed
- edit kafka11aaf/pom.xml
- add pull repo defintion
Note that onap ~/.m2/settings.xml defines
<docker.pull.registry>nexus3.onap.org:10001</docker.pull.registry>
which forces an amd64 specific image.
to support other platforms, add to docker-maven-plugin configuration:
<pullRegistry>${docker.pull.registry}</pullRegistry>
and invoke mvn with appropriate pull repo if building on non-Intel machine, e.g. on Arm
> mvn install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- add push repo definition
add <pushRegistry>${docker.push.registry}</pushRegistry>
push to local registry to test
> mvn install -DskipTests -Pdocker -Ddocker.push.registry=localhost:5000 docker:push
- create src/main/resources/docker/Dockerfile
> mvn clean install -DskipTests -Ddocker.pull.registry=docker.io -Pdocker
- Test, As per step 2
#### Step 4: build and run on Arm
- terminal on Arm machine
> git clone https://gerrit.onap.org/r/dmaap/kafka
- copy modified files from workstation to Arm server
$ scp pom.xml root@147.75.94.202:/root/onap/dmaap/buscontroller/dmaap-bc
$ scp src/main/resources/docker-compose.yml root@147.75.94.202:/root/onap/dmaap/buscontroller/dmaap-bc/src/main/resources
$ scp src/main/resources/Dockerfile root@147.75.94.202:/root/onap/dmaap/buscontroller/dmaap-bc/src/main/resources
> mvn clean install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
> cd src/main/resources
- edit docker-compose.yml
- crunchydata/crunchy-postgres:centos7-10.4-2.0.0 not multi-platform
- use postgres
> docker-compose up