...
- throws 'broken pipe' error which is amisdirection to registry naming issue
- See https://github.com/spotify/dockerfile-maven/issues/95
- maven install error
- See: https://github.com/spotify/dockerfile-maven/issues/252
- upgrade Arm server to maven 3.6.0
- see: https://maven.apache.org/users/index.html
- requires multiple execution blocks for multiple tags
- see https://github.com/spotify/dockerfile-maven/issues/10
DMAAP - messageservice
## dmaap-mr# CIA image refactoring for dmaap message router
Objective: Minimize size and support multi-platfrom execution of dmaap docker images
Jira: INT-910, INT-932
Gerrit Project: dmaap/messagerouter/messageservice
INT-910, INT-932
to test mr requires kafka11aaf and zookeeper
### Step 1: Build and run on local machine (Mac Air)
- in Note that dmaap-mr depends on kafka and zookeeper images specialized by the project
Sequence of steps:
1) build and test as-is on local machine
2) migrate base images to alpine
3) build and test alpine based images on Arm machine
4) test using ONAP OOM (K8s/Helm) on Intel
5) test using ONAP OOM (K8s/Helm) on Arm
6) patch repo with changes
## Step 1: build and test as-is on local machine
- Build local images
- Start containers
- Test
### build local images
- open terminal
$ git clone ssh://sandovalfrOAM@gerrit.onap.org:29418/dmaap/messagerouter/messageservice
$ cd messageservice
- See per docs/installation/installation.rst
- edit MsgRtrApi.propertiesbundleconfig-local/etc/appprops/MsgRtrApi.properties
config.zk.servers=zookeeper
kafka.metadata.broker.list=kafka:9092
- docker-compose network maps service name(zookeeper, kafka) to IP
> cp bundleconfig-local/etc/appprops/MsgRtrApi.properties /var/tmp
- configure 'docker->properties->file sharing' to include /var/tmp
$ mvn clean install -DskipTests -Pdocker
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190304T195732Z ec37060b0ef0 4 minutes ago 782MB
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest ec37060b0ef0 4 minutes ago 782MB
onap/dmaap/dmaap-mr latest ec37060b0ef0 4 minutes ago 782MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190304T195732Z ec37060b0ef0 4 minutes ago 782MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest ec37060b0ef0 4 minutes ago 782MB
openjdk 8-.
openjdk 8-jdk 5f8e49b0a018 3 weeks ago 624MB
- build kafka11aaf image (see separate guide)
- build zookeeper image (see separate guide)
### start containers
$ cd src/main/resources/docker-compose
4 $ cp docker-compose.yml docker-compose-local.yml
- edit docker-compose-local.yml
image: onap/dmaap/zookeeper:latest
image: onap/dmaap/kafka01101:latest
image: onap/dmaap/dmaap-mr:latest
> $ docker-compose -f docker-compose-local.yml up -d
### Test
See https://docs.onap.org/en/latest/submodules/dmaap/messagerouter/messageservice.git/docs/offeredapis/offeredapis.html
- create sample file (any arbitrary content,e g "sample data")
> e.g. $ echo "sample data" > sample.txt
$ curl -H "Content-Type:text/plain" -X POST -d @sample.txt http://localhost:3904/events/TestTopic1
{"serverTimeMs": 645, ...
> - give it some time....
$ curl -H "Content-Type:text/plain" -X GET http://localhost:3904/events/TestTopic1/CG1/C1?timeout=1000
["sample data"]
### Step 2: migrate to alpine
- edit messageservice/src/main/resources/Dockerfile
- from openjdk:8-jdk to openjdk:8-alpine
> mvn clean install -DskipTests -Pdocker
> docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T122553Z e8fbf3bebd9b 46 seconds ago 244MB
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest e8fbf3bebd9b 46 seconds ago 244MB
onap/dmaap/dmaap-mr latest e8fbf3bebd9b 46 seconds ago 244MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T122553Z e8fbf3bebd9b 46 seconds ago 244MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-latest e8fbf3bebd9b 46 seconds ago 244MB
onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T121402Z 62f0a2a98c62 12 minutes ago 244MB
nexus3.onap.org:10003/onap/dmaap/dmaap-mr 1.1.11-SNAPSHOT-20190306T121402Z 62f0a2a98c62 12 minutes ago 244MB
openjdk 8-jre-alpine 1b46cc2ba839 4 weeks ago 85MB
Test as step 1
#### Step 3: migrate pom to fabric8 docker-maven-plugin- view topics
$ curl -H "Content-Type:text/plain" -X GET http://10.233.38.121:3904/topics
- scratch
curl -v -H "Content-Type:text/plain" -X POST -d @sample-body.txt http://10.233.38.121:3904/events/TestTopic1
curl -H "Content-Type:text/plain" -X GET http://10.233.102.148:3904/events/TestTopic1/CG1/C1/timeout=1000
curl -H "Content-Type:text/plain" -X GET http://10.233.102.148:3904/topics
## Step 2: migrate base images to alpine
### edit messageservice/src/main/resources/Dockerfile
- from openjdk:8-jdk to openjdk:8-alpine, etc (atp-get to apk, ...)
$ mvn clean install -DskipTests -Pdocker
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
...
onap/dmaap/dmaap-mr latest e8fbf3bebd9b 46 seconds ago 244MB
...
openjdk 8-jre-alpine 1b46cc2ba839 4 weeks ago 85MB
### Test as per step 1
## Step 3: build and test alpine based images on Arm machine
### edit messageservice/pom.xml
- migrate from spotify to fabric8 docker-maven-plugin
- add variables to control pull/push docker repos from cmd line
- Note: spotify docker-maven-plugin does not run on Arm
[ERROR] Failed to execute goal com.spotify:- See: https://github.com/spotify/docker-maven-plugin:1.0.0:build (default-cli) on project datarouter-prov: Exception caught: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException:
java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider: ExceptionInInitializerError: Can't overwrite cause with java.lang.UnsatisfiedLinkError: cannot determine CPU -> [Help 1]
- See: https://github.com/spotify/docker-maven-plugin/issues/233
- Note: spotify dockerfile-maven-plugin is alternate to fabric8, but other ONAP teams are using fabric8 so there
appears to be no compelling reason to stick w spotify
- Note: docker pull/push repos must be managed
- edit messageservice/pom.xml
- add pull repo defintion
Note that onap ~/.m2/settings.xml defines
<docker.pull.registry>nexus3.onap.org:10001</docker.pull.registry>
which forces an amd64 specific image.
to support other platforms, add to docker-maven-plugin configuration:
<pullRegistry>${docker.pull.registry}</pullRegistry>
and invoke mvn with appropriate pull repo if building on non-Intel machine, e.g. on Arm
> mvn install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- add push repo definition
add <pushRegistry>${docker.push.registry}</pushRegistry>
push to local registry to test*
- create src/main/resources/docker/Dockerfile
$ mvn install -DskipTests -Pdocker
- Test, As per step 2
#### Step 4: build and run on Arm
- terminal on Arm machine
$ git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice
- copy modified files from workstation to Arm server
$ scp pom.xml root@147.75.94.202:/root/onap/dmaap/messageservice
$ scp src/main/resources/docker/Dockerfile root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker
$ scp src/main/resources/docker-compose/docker-compose-local.yml root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker-compose
$ scp src/main/resources/docker-compose/MsgRtrApi.properties root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker-compose
$ mvn clean install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- build kafka11aaf image (see separate guide)
- build zookeeper image (see separate guide)
$ cd src/main/resources/docker-compose
$ docker-compose -f docker-compose-local.yml up -d
- Test as above/issues/233
- spotify dockerfile-maven-plugin is alternate to fabric8, but other ONAP teams are using fabric8 so there
appears to be no compelling reason to stick w spotify
- Note: default docker registry nexus3.onap.org:10001 does not support manifest images, i.e. no multi-platform support
- invoke mvn with appropriate pull repo if building on non-Intel machine, e.g. on Arm
> mvn install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- push to local registry to test
$ mvn install -DskipTests -Pdocker
### Local test of plugin, as per step 1
### Test on Arm
- access Arm server, i.e Platform bare-metal
- terminal on Arm machine
$ git clone https://gerrit.onap.org/r/dmaap/messagerouter/messageservice
- copy modified files from workstation to Arm server
$ scp pom.xml root@147.75.94.202:/root/onap/dmaap/messageservice
$ scp src/main/resources/docker/Dockerfile root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker
$ scp src/main/resources/docker-compose/docker-compose-local.yml root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker-compose
$ scp src/main/resources/docker-compose/MsgRtrApi.properties root@147.75.94.202:/root/onap/dmaap/messageservice/src/main/resources/docker-compose
$ mvn clean install -DskipTests -Pdocker -Ddocker.pull.registry=docker.io
- build kafka11aaf image (see separate guide)
- build zookeeper image (see separate guide)
### start containers
$ cd src/main/resources/docker-compose
$ docker-compose -f docker-compose-local.yml up -d
### Test as above
## Step 4: test using ONAP OOM (K8s/Helm) on Intel
### Intel - test as-is
- access Laas server https://labs.opnfv.org
Note: k8s and Helm operations can take some time (+several minutes)
patience is required to step through instructions and allow services to become operational
$ ssh-keygen
$ cat /home/opnfv/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ git clone https://gerrit.opnfv.org/gerrit/auto
$ cd auto/
$ export ONAP_BRANCH=master
$ export ONAP_COMPONENT="dmaap robot"
$ ./ci/deploy-onap-kubespray.sh <ip of server> |& tee ~/deploy.log
if opnfv@10.200.123.137's password: exit and ssh in again????
...
Nr. Datetime of check Err/Total PODs
1 Thu Apr 11 01:52:09 EDT 2019 0/0
...
Ctrl-C
$ sudo -i
# kubectl get pods --namespace onap
NAME READY STATUS RESTARTS AGE
dev-dbc-pg-0 1/1 Running 0 17m
dev-dbc-pg-1 1/1 Running 0 15m
dev-dbc-pgpool-7ff5894667-jlz9n 1/1 Running 0 17m
dev-dbc-pgpool-7ff5894667-v95gg 1/1 Running 0 17m
dev-dmaap-bc-547dc7594c-znhg6 1/1 Running 0 17m
dev-dmaap-bc-post-install-j9vgk 0/1 Completed 0 17m
dev-dmaap-dr-db-0 1/1 Running 0 17m
dev-dmaap-dr-db-1 1/1 Running 0 15m
dev-dmaap-dr-node-0 2/2 Running 0 17m
dev-dmaap-dr-prov-5f58fd778d-9vmdk 2/2 Running 0 17m
dev-message-router-0 1/1 Running 0 17m
dev-message-router-kafka-0 1/1 Running 3 17m
dev-message-router-kafka-1 1/1 Running 3 17m
dev-message-router-kafka-2 1/1 Running 3 17m
dev-message-router-mirrormaker-5dd964cdfc-pl5sr 0/1 CrashLoopBackOff 6 17m
dev-message-router-post-install-zdr4j 1/1 Running 0 14m
dev-message-router-zookeeper-0 1/1 Running 0 17m
dev-message-router-zookeeper-1 1/1 Running 0 17m
dev-message-router-zookeeper-2 1/1 Running 0 17m
dev-robot-8495d96c8-hx9hv 1/1 Running 0 17m
$ sudo -i
$ cd /home/opnfv/oom/kubernetes/robot/
$ ./ete-k8s.sh onap health | tee ~/onap-health.log
Starting Xvfb on display :92 with res 1280x1024x24
Executing robot tests at log level TRACE
==============================================================================
Testsuites
==============================================================================
Testsuites.Health-Check :: Testing ecomp components are available via calls.
==============================================================================
...
------------------------------------------------------------------------------
Basic DMAAP Data Router Health Check | PASS |
------------------------------------------------------------------------------
Basic DMAAP Message Router Health Check | PASS |
------------------------------------------------------------------------------
...
60 critical tests, 2 passed, 58 failed
60 tests total, 2 passed, 58 failed
==============================================================================
Testsuites | FAIL |
60 critical tests, 2 passed, 58 failed
60 tests total, 2 passed, 58 failed
==============================================================================
Output: /share/logs/0002_ete_health/output.xml
Log: /share/logs/0002_ete_health/log.html
Report: /share/logs/0002_ete_health/report.html
### clean-up
helm del --purge dev
kubectl delete --namespace onap pods,services,jobs,configmaps,deployments,secrets,pv,pvc --all
kubectl delete clusterrolebindings --namespace onap --all
# just to be sure do cleanup once more
kubectl delete --namespace onap pods,services,jobs,configmaps,deployments,secrets,pv,pvc --all
kubectl delete namespace onap
### Intel - test alpine images
- on local machine build alpine images, as in Step 2
- tag and push images to docker hub, so they can be pulled on lab machine
$ docker tag onap/dmaap/dmaap-mr:latest oamtech/dmaap-mr:latest
$ docker push oamtech/dmaap-mr:latest
- same for onap/dmaap/kafka and zookeeper
- on server
- NOTE: cleaning up k8s/Helm can be problematic. Sometimes its easiest to simply request a fresh server
split deploy-onap-kubespray.sh into two parts in order to edit helm charts
$ cp deploy-onap-kubespray.sh deploy-kubespray.sh
$ cp deploy-onap-kubespray.sh deploy-onap-helm.sh
-edit scripts to separate phases
* deploy-kubespray.sh: remove lines below 209. add exit 0
* deploy-onap-helm.sh: remove lines 164-179, 231-233, 240-243
$ ./deploy-kubespray.sh <ip of server> |& tee ~/deploy.log
- if fresh server. If Helm charts have been installed and cleaned up as above, start below
git clone -b master http://gerrit.onap.org/r/oom
edit kubernetes/dmaap/message-router charts to point to alpine images on docker hub, e.g.
auto/oom/kubernetes/dmaap/components/message-router/charts/message-router-kafka$ vi values.yaml
- change image to oamtech/message-router:latest
$ ./deploy-onap-helm.sh <ip of server> |& tee ~/deploy.log
### Tests
run robot tests
$ ./ete-k8s.sh onap health | tee ~/onap-health.log
then test via curl (from server) as in Step 1.
To find IP of message-router pod:
$ kubectl get service message-router -n onap
- message router service defined in oom/kubernetes/dmaap/components/message-router/vaues.yaml
DMAAP KAFKA11aaf
## dmaap kafka
...