Docker DevOps

Jan 24, 2019

Procedures for pulling/creating/updating/pushing Docker images.

Docker Image Build Guidelines

Questions

What is the difference between the 10001 and 10003 port versions for nexus3?

Which version of the maven plugin? https://gerrit.onap.org/r/#/c/8509/

Docker Image Build Guidelines

Docker Run

Docker Pull

Check your login first

$docker login -u docker -p docker nexus3.onap.org:10001 Login Succeeded

$docker login -u docker -p docker nexus3.onap.org:10001 Login Succeeded

Docker Exec

root@vm1-aai-inst1:~# docker ps -a

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

a0b610740d68 nexus3.onap.org:10001/openecomp/data-router "/opt/app/data-rou..." 15 hours ago Up 15 hours 0.0.0.0:9502->9502/tcp testconfig_datarouter_1

90b36b769927 nexus3.onap.org:10001/openecomp/search-data-service "/opt/app/search-d..." 15 hours ago Up 15 hours 0.0.0.0:9509->9509/tcp testconfig_aai.searchservice.simpledemo.openecomp.org_1

b347cb91e6c2 nexus3.onap.org:10001/openecomp/model-loader "/opt/app/model-lo..." 15 hours ago Up 15 hours testconfig_model-loader_1

0dca1d420750 aaionap/haproxy "/docker-entrypoin..." 15 hours ago Up 15 hours 0.0.0.0:8443->8443/tcp testconfig_aai.api.simpledemo.openecomp.org_1

cf82f991a05b nexus3.onap.org:10001/openecomp/aai-traversal "/bin/sh -c ./dock..." 15 hours ago Up 15 hours 0.0.0.0:8446->8446/tcp testconfig_aai-traversal.api.simpledemo.openecomp.org_1

b3284622c0aa nexus3.onap.org:10001/openecomp/aai-resources "/bin/sh -c ./dock..." 15 hours ago Up 15 hours 0.0.0.0:8447->8447/tcp testconfig_aai-resources.api.simpledemo.openecomp.org_1

245da2cf6bb8 nexus3.onap.org:10001/openecomp/sparky-be "/bin/sh -c /opt/a..." 15 hours ago Up 15 hours 8000/tcp, 0.0.0.0:9517->9517/tcp testconfig_sparky-be_1

ca140c25bb91 attos/dmaap "/bin/sh -c ./star..." 15 hours ago Up 15 hours 0.0.0.0:3904-3905->3904-3905/tcp dockerfiles_dmaap_1

d3dcbe8b3dd6 dockerfiles_kafka "start-kafka.sh" 15 hours ago Up 15 hours 0.0.0.0:9092->9092/tcp dockerfiles_kafka_1

52f929f6aa06 wurstmeister/zookeeper "/bin/sh -c '/usr/..." 15 hours ago Up 15 hours 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp dockerfiles_zookeeper_1

root@vm1-aai-inst1:~# docker ps -a

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

a0b610740d68 nexus3.onap.org:10001/openecomp/data-router "/opt/app/data-rou..." 15 hours ago Up 15 hours 0.0.0.0:9502->9502/tcp testconfig_datarouter_1

90b36b769927 nexus3.onap.org:10001/openecomp/search-data-service "/opt/app/search-d..." 15 hours ago Up 15 hours 0.0.0.0:9509->9509/tcp testconfig_aai.searchservice.simpledemo.openecomp.org_1

b347cb91e6c2 nexus3.onap.org:10001/openecomp/model-loader "/opt/app/model-lo..." 15 hours ago Up 15 hours testconfig_model-loader_1

0dca1d420750 aaionap/haproxy "/docker-entrypoin..." 15 hours ago Up 15 hours 0.0.0.0:8443->8443/tcp testconfig_aai.api.simpledemo.openecomp.org_1

cf82f991a05b nexus3.onap.org:10001/openecomp/aai-traversal "/bin/sh -c ./dock..." 15 hours ago Up 15 hours 0.0.0.0:8446->8446/tcp testconfig_aai-traversal.api.simpledemo.openecomp.org_1

b3284622c0aa nexus3.onap.org:10001/openecomp/aai-resources "/bin/sh -c ./dock..." 15 hours ago Up 15 hours 0.0.0.0:8447->8447/tcp testconfig_aai-resources.api.simpledemo.openecomp.org_1

245da2cf6bb8 nexus3.onap.org:10001/openecomp/sparky-be "/bin/sh -c /opt/a..." 15 hours ago Up 15 hours 8000/tcp, 0.0.0.0:9517->9517/tcp testconfig_sparky-be_1

ca140c25bb91 attos/dmaap "/bin/sh -c ./star..." 15 hours ago Up 15 hours 0.0.0.0:3904-3905->3904-3905/tcp dockerfiles_dmaap_1

d3dcbe8b3dd6 dockerfiles_kafka "start-kafka.sh" 15 hours ago Up 15 hours 0.0.0.0:9092->9092/tcp dockerfiles_kafka_1

52f929f6aa06 wurstmeister/zookeeper "/bin/sh -c '/usr/..." 15 hours ago Up 15 hours 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp dockerfiles_zookeeper_1

Docker log

https://lf-onap.atlassian.net/browse/LOG-47

docker logs -f testconfig_aai-resources.api.simpledemo.openecomp.org_1

docker logs -f testconfig_aai-resources.api.simpledemo.openecomp.org_1

Docker Build

docker build -t nexus3.onap.org:10001/openecomp/dcae/pgaas -f Dockerfile .

Step 29/29 : ENTRYPOINT usr/local/bin/entrypoint.sh

---> Using cache

---> d3cf07a80918

Successfully built d3cf07a80918

Successfully tagged nexus3.onap.org:10001/openecomp/dcae/pgaas:latest

obrienbiometrics:pgaas michaelobrien$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

nexus3.onap.org:10001/openecomp/dcae/pgaas latest d3cf07a80918 9 minutes ago 1.03GB

docker build -t nexus3.onap.org:10001/openecomp/dcae/pgaas -f Dockerfile .

Step 29/29 : ENTRYPOINT usr/local/bin/entrypoint.sh

---> Using cache

---> d3cf07a80918

Successfully built d3cf07a80918

Successfully tagged nexus3.onap.org:10001/openecomp/dcae/pgaas:latest

obrienbiometrics:pgaas michaelobrien$ docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

nexus3.onap.org:10001/openecomp/dcae/pgaas latest d3cf07a80918 9 minutes ago 1.03GB

Docker Registry Image Search

where the image name is "dcae-controller" obrienbiometrics:pgaas michaelobrien$ curl -Ss -u docker:docker https://nexus3.onap.org:10001/v2/dcae-controller/tags/list \| jq { "name": "dcae-controller", "tags": [ "0.1.0-SNAPSHOT", "1.0.0" ]}

where the image name is "dcae-controller" obrienbiometrics:pgaas michaelobrien$ curl -Ss -u docker:docker https://nexus3.onap.org:10001/v2/dcae-controller/tags/list \| jq { "name": "dcae-controller", "tags": [ "0.1.0-SNAPSHOT", "1.0.0" ]}

Docker Push

Dockerhub

root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# export DOCKER_ID_USER="obrienlabs" root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: obrienlabs Password: Login Succeeded root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker build -t obrienlabs/pgaas -f Dockerfile . Sending build context to Docker daemon 23.55 kB Step 1 : FROM ubuntu:16.04 ---> d355ed3537e9 .... Successfully built 257430a38254 root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker tag obrienlabs/pgaas obrienlabs/pgaas:1 root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker push obrienlabs/pgaas:1 The push refers to a repository [docker.io/obrienlabs/pgaas] ee716c896001: Pushed ....5a4c2c9a24fc: Pushed cb11ba605400: Pushed 1: digest: sha256:fb09855b323fe83aaa161725910cd2f6d7ff6306035e6f5f2df40b99149e8fed size: 6157 pull on separate machine obrienbiometrics:oom michaelobrien$ docker pull obrienlabs/pgaas:1 1: Pulling from obrienlabs/pgaas 23a6960fe4a9: Extracting [===> ] 3.441MB/45.89MB e9e104b0e69d: Download complete cd33d2ea7970: Download complete 534ff7b7d120: Download complete

root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# export DOCKER_ID_USER="obrienlabs" root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username: obrienlabs Password: Login Succeeded root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker build -t obrienlabs/pgaas -f Dockerfile . Sending build context to Docker daemon 23.55 kB Step 1 : FROM ubuntu:16.04 ---> d355ed3537e9 .... Successfully built 257430a38254 root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker tag obrienlabs/pgaas obrienlabs/pgaas:1 root@obriensystemskub0:~/oom/kubernetes/dcae/pgaas# docker push obrienlabs/pgaas:1 The push refers to a repository [docker.io/obrienlabs/pgaas] ee716c896001: Pushed ....5a4c2c9a24fc: Pushed cb11ba605400: Pushed 1: digest: sha256:fb09855b323fe83aaa161725910cd2f6d7ff6306035e6f5f2df40b99149e8fed size: 6157 pull on separate machine obrienbiometrics:oom michaelobrien$ docker pull obrienlabs/pgaas:1 1: Pulling from obrienlabs/pgaas 23a6960fe4a9: Extracting [===> ] 3.441MB/45.89MB e9e104b0e69d: Download complete cd33d2ea7970: Download complete 534ff7b7d120: Download complete

Docker save/load

TODO: document - use case for a bulk preload pull of docker images to pass to machines that are behind a firewall where the 10001 port of nexus3.onap.org is inaccessible

Thanks @Mohsen Ansari

Docker cleanup

Stop all containers

docker stop $(docker ps -a -q)

Remove all containers

docker rm $(docker ps -a -f status=exited -q)

Remove all images

docker rmi $(docker images -a -q) --force

Remove all volumes

docker volume rm $(docker volume ls)

ONAP nexus3

We cannot push directly to nexus3 - we need to do a staged release - https://lf-onap.atlassian.net/browse/CIMAN-22

Not currently working

obrienbiometrics:pgaas michaelobrien$ docker push nexus3.onap.org:10001/openecomp/dcae/pgaas The push refers to a repository [nexus3.onap.org:10001/openecomp/dcae/pgaas] b4f440dc89cd: Preparing b1a91ac0378f: Preparing a6ab361d9284: Preparing 01d34e08a9e0: Preparing 67166bc93a3a: Preparing f0cc66c45ab4: Waiting d09c777e7ae4: Waiting e30c66e473f0: Waiting 01b1f5510a52: Waiting 94f82c263b44: Waiting a1df65c04630: Waiting 3c8596501eba: Waiting 17ba7123d342: Waiting 1fcfa71c715c: Waiting 7ed908687894: Waiting 764d7670ed3c: Waiting 28868d65df7c: Waiting a55b0c3f0a89: Waiting 372ceae4f0a6: Waiting 63a268fe22b1: Waiting 28312f9d3023: Waiting ef1d2d917acc: Waiting a09947e71dc0: Waiting 9c42c2077cde: Waiting 625c7a2a783b: Waiting 25e0901a71b8: Waiting 8aa4fcad5eeb: Waiting error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "\n<!DOCTYPE html>\n<html>\n<head>\n <title>404 - Nexus Repository Manager</title>\n <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\"/>\n\n\n \n <link rel=\"icon\" type=\"image/png\" href=\"https://nexus3.onap.org/favicon-32x32.png?3.2.0-01\" sizes=\"32x32\">\n <link rel=\"mask-icon\" href=\"https://nexus3.onap.org/safari-pinned-tab.svg?3.2.0-01\" color=\"#5bbad5\">\n <link rel=\"icon\" type=\"image/png\" href=\"https://nexus3.onap.org/favicon-16x16.png?3.2.0-01\" sizes=\"16x16\">\n <link rel=\"shortcut icon\" href=\"https://nexus3.onap.org/favicon.ico?3.2.0-01\">\n <meta name=\"msapplication-TileImage\" content=\"https://nexus3.onap.org/mstile-144x144.png?3.2.0-01\">\n <meta name=\"msapplication-TileColor\" content=\"#00a300\">\n\n <link rel=\"stylesheet\" type=\"text/css\" href=\"https://nexus3.onap.org/static/css/nexus-content.css?3.2.0-01\"/>\n</head>\n<body>\n<div class=\"nexus-header\">\n <a href=\"https://nexus3.onap.org\">\n <div class=\"product-logo\">\n <img src=\"https://nexus3.onap.org/static/images/nexus.png?3.2.0-01\"/>\n </div>\n <div class=\"product-id\">\n <div class=\"product-id__line-1\">\n <span class=\"product-name\">Nexus Repository Manager</span>\n </div>\n <div class=\"product-id__line-2\">\n <span class=\"product-spec\">3.2.0-01</span>\n </div>\n </div>\n </a>\n</div>\n\n<div class=\"nexus-body\">\n <div class=\"content-header\">\n <img src=\"https://nexus3.onap.org/static/rapture/resources/icons/x32/exclamation.png?3.2.0-01\"/>\n <span class=\"title\">Error 404</span>\n <span class=\"description\">Not Found</span>\n </div>\n <div class=\"content-body\">\n <div class=\"content-section\">\n Not Found\n </div>\n </div>\n</div>\n</body>\n</html>\n\n"

Not currently working

obrienbiometrics:pgaas michaelobrien$ docker push nexus3.onap.org:10001/openecomp/dcae/pgaas The push refers to a repository [nexus3.onap.org:10001/openecomp/dcae/pgaas] b4f440dc89cd: Preparing b1a91ac0378f: Preparing a6ab361d9284: Preparing 01d34e08a9e0: Preparing 67166bc93a3a: Preparing f0cc66c45ab4: Waiting d09c777e7ae4: Waiting e30c66e473f0: Waiting 01b1f5510a52: Waiting 94f82c263b44: Waiting a1df65c04630: Waiting 3c8596501eba: Waiting 17ba7123d342: Waiting 1fcfa71c715c: Waiting 7ed908687894: Waiting 764d7670ed3c: Waiting 28868d65df7c: Waiting a55b0c3f0a89: Waiting 372ceae4f0a6: Waiting 63a268fe22b1: Waiting 28312f9d3023: Waiting ef1d2d917acc: Waiting a09947e71dc0: Waiting 9c42c2077cde: Waiting 625c7a2a783b: Waiting 25e0901a71b8: Waiting 8aa4fcad5eeb: Waiting error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "\n<!DOCTYPE html>\n<html>\n<head>\n <title>404 - Nexus Repository Manager</title>\n <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\"/>\n\n\n \n <link rel=\"icon\" type=\"image/png\" href=\"https://nexus3.onap.org/favicon-32x32.png?3.2.0-01\" sizes=\"32x32\">\n <link rel=\"mask-icon\" href=\"https://nexus3.onap.org/safari-pinned-tab.svg?3.2.0-01\" color=\"#5bbad5\">\n <link rel=\"icon\" type=\"image/png\" href=\"https://nexus3.onap.org/favicon-16x16.png?3.2.0-01\" sizes=\"16x16\">\n <link rel=\"shortcut icon\" href=\"https://nexus3.onap.org/favicon.ico?3.2.0-01\">\n <meta name=\"msapplication-TileImage\" content=\"https://nexus3.onap.org/mstile-144x144.png?3.2.0-01\">\n <meta name=\"msapplication-TileColor\" content=\"#00a300\">\n\n <link rel=\"stylesheet\" type=\"text/css\" href=\"https://nexus3.onap.org/static/css/nexus-content.css?3.2.0-01\"/>\n</head>\n<body>\n<div class=\"nexus-header\">\n <a href=\"https://nexus3.onap.org\">\n <div class=\"product-logo\">\n <img src=\"https://nexus3.onap.org/static/images/nexus.png?3.2.0-01\"/>\n </div>\n <div class=\"product-id\">\n <div class=\"product-id__line-1\">\n <span class=\"product-name\">Nexus Repository Manager</span>\n </div>\n <div class=\"product-id__line-2\">\n <span class=\"product-spec\">3.2.0-01</span>\n </div>\n </div>\n </a>\n</div>\n\n<div class=\"nexus-body\">\n <div class=\"content-header\">\n <img src=\"https://nexus3.onap.org/static/rapture/resources/icons/x32/exclamation.png?3.2.0-01\"/>\n <span class=\"title\">Error 404</span>\n <span class=\"description\">Not Found</span>\n </div>\n <div class=\"content-body\">\n <div class=\"content-section\">\n Not Found\n </div>\n </div>\n</div>\n</body>\n</html>\n\n"

Updating WARs/JARs

Attaching to a Debug Port

https://lf-onap.atlassian.net/browse/UCA-25

-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5432

List Docker Mounts

Example retrieving docker mounts for the CDAP Collector VM

root@zldciad4vicoll00:~# df

Filesystem     1K-blocks    Used Available Use% Mounted on

none            41251136 5354876  34097432  14% /opt/app/docker/data/aufs/mnt/14a5fd911ae442f1041a51f05d760dcd75969f455e6e6471e5f62eb102e400ff

shm                65536       0     65536   0% /opt/app/docker/data/containers/98bdea201c182d68f95e814a0f9b8667e3cd38e6017fb632368b995920bda825/shm

none            41251136 5354876  34097432  14% /opt/app/docker/data/aufs/mnt/86e5d7ccec8b8cb80d934baa0259c1aa582526495fb4fa004afe6f49e0cde3ec

shm                65536       0     65536   0% /opt/app/docker/data/containers/c12cf4f98007316e0ed9f11bf70ebf2db8969e1a2426a65144f023d7db477131/shm

none            41251136 5354876  34097432  14% /opt/app/docker/data/aufs/mnt/03e8aaf09316cda6a17063b6ae6d0ecfc0e6c1420a804f627d895f1b3305b803

shm                65536       0     65536   0% /opt/app/docker/data/containers/5d5fb7726fb34150b5a188570bcb10359e254d68161f7b545936e5ed8980e047/shm

none            41251136 5354876  34097432  14% /opt/app/docker/data/aufs/mnt/4bd2a7e73e86618a5c547d33758a9d7c9e402b4db108b823614a0319b320e62b

shm                65536       0     65536   0% /opt/app/docker/data/containers/53bba60de57d924780913f75e4ca815c83a7c7e51e890b661df3f70b818d283d/shm

none            41251136 5354876  34097432  14% /opt/app/docker/data/aufs/mnt/e01ab748cc467704e874f7240b72bfc26e85e10897fae565028bb66189e31683

shm                65536       0     65536   0% /opt/app/docker/data/containers/72956fb48e80ad39c56d30e181e22e42865723f265eefd1975d31cb182bef416/shm




root@zldciad4vicoll00:~# docker ps

CONTAINER ID        IMAGE                                                                            COMMAND                  CREATED             STATUS              PORTS                                                                     NAMES

72956fb48e80        nexus3.onap.org:10001/openecomp/dcae-collector-common-event:1.0-STAGING-latest   "/opt/app/manager/..."   20 hours ago        Up 20 hours         0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp, 0.0.0.0:32768->9999/tcp   dcae_docker-common-event_iad4

53bba60de57d        nexus3.onap.org:10001/openecomp/dcae-dmaapbc:1.0-STAGING-latest                  "./bin/dmaapbc deploy"   20 hours ago        Up 20 hours         0.0.0.0:18080->8080/tcp, 0.0.0.0:18443->8443/tcp                          dcae_docker-databus-controller_iad4

5d5fb7726fb3        attos/dmaap                                                                      "/bin/sh -c ./star..."   21 hours ago        Up 21 hours         0.0.0.0:3904-3905->3904-3905/tcp                                          dockerfiles_dmaap_1

c12cf4f98007        wurstmeister/zookeeper                                                           "/bin/sh -c '/usr/..."   21 hours ago        Up 21 hours         22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp                        dockerfiles_zookeeper_1

98bdea201c18        dockerfiles_kafka                                                                "start-kafka.sh"         21 hours ago        Up 21 hours         0.0.0.0:9092->9092/tcp                                                    dockerfiles_kafka_1

root@zldciad4vicoll00:~# docker inspect -f '{{ .Mounts }}' 98bdea201c18

[{bind  /opt/app/message-router/docker_files/start-kafka.sh /start-kafka.sh  rw true } {bind  /var/run/docker.sock /var/run/docker.sock  rw true } {bind  /opt/app/message-router/docker_files/data-kafka /kafka  rw true }]

root@zldciad4vicoll00:~# docker inspect -f '{{ .Mounts }}' c12cf4f98007

[{bind  /opt/app/message-router/docker_files/data-zookeeper /opt/zookeeper-3.4.9/data  rw true } {volume 786f3819c8cfee7ff5a2d22fc644563d78b550fb05977573ea663550f2356f43 /opt/app/docker/data/volumes/786f3819c8cfee7ff5a2d22fc644563d78b550fb05977573ea663550f2356f43/_data /opt/zookeeper-3.4.9/conf local  true }]

root@zldciad4vicoll00:~# docker inspect -f '{{ .Mounts }}' 5d5fb7726fb3

[{bind  /opt/app/message-router/docker_files/MsgRtrApi.properties /appl/dmaapMR1/bundleconfig/etc/appprops/MsgRtrApi.properties  rw true } {bind  /opt/app/message-router/docker_files/mykey /appl/dmaapMR1/etc/keyfile  rw true } {bind  /opt/app/message-router/docker_files/cadi.properties /appl/dmaapMR1/etc/cadi.properties  rw true }]

root@zldciad4vicoll00:~# docker inspect -f '{{ .Mounts }}' 53bba60de57d

[{volume 6f8c7044b305bf02c8d0cdf7656e0d1bd446ef0799c825387ee2a7ec0b322c35 /opt/app/docker/data/volumes/6f8c7044b305bf02c8d0cdf7656e0d1bd446ef0799c825387ee2a7ec0b322c35/_data /opt/app/dmaapbc/log local  true } {bind  /var/log/DCAE/docker-databus-controller/iad4/manager /opt/app/logs   true } {bind  /var/config/DCAE/docker-databus-controller/iad4/manager /opt/app/config   true } {bind  /opt/app/dcae-certificate /opt/app/dcae-certificate   true } {bind  /opt/app/groovy /opt/app/groovy   true } {bind  /etc/hosts /opt/app/etc/hosts   true }]

root@zldciad4vicoll00:~# docker inspect -f '{{ .Mounts }}' 72956fb48e80

[{bind  /var/log/DCAE/docker-common-event/iad4/app /opt/app/SEC/logs   true } {bind  /var/log/DCAE/docker-common-event/iad4/manager /opt/app/manager/logs   true } {bind  /var/config/DCAE/docker-common-event/iad4/manager /opt/app/manager/config   true } {bind  /opt/app/dcae-certificate /opt/app/dcae-certificate   true } {bind  /opt/app/groovy /opt/app/groovy   true }]

Docker fails to start on OSX

https://github.com/docker/for-mac/issues/989

reset to defaults

Links

View docker images (don't sign in)

https://nexus3.onap.org/#browse/browse/components:docker.public

https://github.com/moby/moby/issues/22910

https://lf-onap.atlassian.net/browse/SDC-277