/
OOM RKE Kubernetes Deployment
OOM RKE Kubernetes Deployment
- Michael O'Brien
Owned by Michael O'Brien
Tracking OOM-1598 - Getting issue details... STATUS
NON-HA version in https://git.onap.org/oom/tree/kubernetes/contrib/tools/rke/rke_setup.sh
https://gerrit.onap.org/r/#/c/79067/
Move to https://onap.readthedocs.io/en/beijing/submodules/oom.git/docs/oom_cloud_setup_guide.html or similar when this documentation is released
Automated Installation Video of RKE install
20190227 - VMware
20190330 - AWS
Quickstart
Get your public and private keys on the Ubuntu 16.04 VM.
Adjust authorized_keys with your public key if not already - aws has it, openstack may not
get rke script from jira, gerrit or by cloning OOM when the review is done.
# on your laptop/where your cert is # chmod 777 your cert before you scp it over obrienbiometrics:full michaelobrien$ scp ~/wse_onap/onap_rsa ubuntu@rke0.onap.info:~/ # on the host sudo cp onap_rsa ~/.ssh sudo chmod 400 ~/.ssh/onap_rsa sudo chown ubuntu:ubuntu ~/.ssh/onap_rsa # just verify sudo vi ~/.ssh/authorized_keys git clone --recurse-submodules https://gerrit.onap.org/r/oom sudo cp oom/kubernetes/contrib/tools/rke/rke_setup.sh . sudo nohup ./rke_setup.sh -b master -s 104.209.161.210 -e onap -k onap_rsa -l ubuntu & ubuntu@a-rke0-master:~$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE ingress-nginx default-http-backend-797c5bc547-55fpn 1/1 Running 0 4m ingress-nginx nginx-ingress-controller-znhgz 1/1 Running 0 4m kube-system canal-dqt2m 3/3 Running 0 5m kube-system kube-dns-7588d5b5f5-pzdfh 3/3 Running 0 5m kube-system kube-dns-autoscaler-5db9bbb766-b7vvg 1/1 Running 0 5m kube-system metrics-server-97bc649d5-fmqjd 1/1 Running 0 4m kube-system rke-ingress-controller-deploy-job-dxmbd 0/1 Completed 0 4m kube-system rke-kubedns-addon-deploy-job-wqccp 0/1 Completed 0 5m kube-system rke-metrics-addon-deploy-job-ssrgp 0/1 Completed 0 4m kube-system rke-network-plugin-deploy-job-jkffq 0/1 Completed 0 5m kube-system tiller-deploy-759cb9df9-rlt7v 1/1 Running 0 2m ubuntu@a-rke0-master:~$ helm list
Versions
Currently Docker 18.06, RKE 0.1.16, Kubernetes 1.11.6, Kubectl 1.11.6, Helm 2.12.3
TODO: verify later versions of helm and a way to get RKE to install Kubernetes 1.13
Prerequisites
Ubuntu 16.04 VM
Determine RKE and Docker versions
Don't just use the latest docker version - check the RKE release page to get the version pair - 0.1.15/17.03 and 0.1.16/18.06 - see https://github.com/docker/docker-ce/releases - currently https://github.com/docker/docker-ce/releases/tag/v18.06.3-ce
ubuntu@a-rke:~$ sudo curl https://releases.rancher.com/install-docker/18.06.sh | sh ubuntu@a-rke:~$ sudo usermod -aG docker ubuntu ubuntu@a-rke:~$ sudo docker version Client: Version: 18.06.3-ce API version: 1.38 Go version: go1.10.3 Git commit: d7080c1 Built: Wed Feb 20 02:27:18 2019 # install RKE sudo wget https://github.com/rancher/rke/releases/download/v0.1.16/rke_linux-amd64 mv rke_linux-amd64 rke sudo mv ./rke /usr/local/bin/rke ubuntu@a-rke:~$ rke --version rke version v0.1.16
Private SSH key
scp your public key to the box - ideally to ~/.ssh and chmod 400 it - make sure you add your key to authorized_keys
Elastic Reserved IP
get a VIP or EIP and assign this to your VM
generate cluster.yml - optional
cluster.yml will generated by the script rke_setup.sh
azure config - no need to hand build the yml
Watch the path of your 2 keys
Also don't add an "addon" until you have one of the config job will fail
{noformat}
ubuntu@a-rke:~$ rke config --name cluster.yml
[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: ~/.ssh/onap_rsa
[+] Number of Hosts [1]:
[+] SSH Address of host (1) [none]: rke.onap.cloud
[+] SSH Port of host (1) [22]:
[+] SSH Private Key Path of host (rke.onap.cloud) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (rke.onap.cloud) [ubuntu]:
[+] Is host (rke.onap.cloud) a Control Plane host (y/n)? [y]: y
[+] Is host (rke.onap.cloud) a Worker host (y/n)? [n]: y
[+] Is host (rke.onap.cloud) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (rke.onap.cloud) [none]:
[+] Internal IP of host (rke.onap.cloud) [none]:
[+] Docker socket path on host (rke.onap.cloud) [/var/run/docker.sock]:
[+] Network Plugin Type (flannel, calico, weave, canal) [canal]:
[+] Authentication Strategy [x509]:
[+] Authorization Mode (rbac, none) [rbac]:
[+] Kubernetes Docker image [rancher/hyperkube:v1.11.6-rancher1]:
[+] Cluster domain [cluster.local]:
[+] Service Cluster IP Range [10.43.0.0/16]:
[+] Enable PodSecurityPolicy [n]:
[+] Cluster Network CIDR [10.42.0.0/16]:
[+] Cluster DNS Service IP [10.43.0.10]:
[+] Add addon manifest URLs or YAML files [no]: no
ubuntu@a-rke:~$ sudo cat cluster.yml
# If you intened to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
nodes:
- address: rke.onap.cloud
port: "22"
internal_address: ""
role:
- controlplane
- worker
- etcd
hostname_override: ""
user: ubuntu
docker_socket: /var/run/docker.sock
ssh_key: ""
ssh_key_path: ~/.ssh/onap_rsa
labels: {}
services:
etcd:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
external_urls: []
ca_cert: ""
cert: ""
key: ""
path: ""
snapshot: null
retention: ""
creation: ""
kube-api:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
service_cluster_ip_range: 10.43.0.0/16
service_node_port_range: ""
pod_security_policy: false
kube-controller:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
cluster_cidr: 10.42.0.0/16
service_cluster_ip_range: 10.43.0.0/16
scheduler:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
kubelet:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
cluster_domain: cluster.local
infra_container_image: ""
cluster_dns_server: 10.43.0.10
fail_swap_on: false
kubeproxy:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
network:
plugin: canal
options: {}
authentication:
strategy: x509
options: {}
sans: []
system_images:
etcd: rancher/coreos-etcd:v3.2.18
alpine: rancher/rke-tools:v0.1.15
nginx_proxy: rancher/rke-tools:v0.1.15
cert_downloader: rancher/rke-tools:v0.1.15
kubernetes_services_sidecar: rancher/rke-tools:v0.1.15
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.10
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.10
kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.10
kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
kubernetes: rancher/hyperkube:v1.11.6-rancher1
flannel: rancher/coreos-flannel:v0.10.0
flannel_cni: rancher/coreos-flannel-cni:v0.3.0
calico_node: rancher/calico-node:v3.1.3
calico_cni: rancher/calico-cni:v3.1.3
calico_controllers: ""
calico_ctl: rancher/calico-ctl:v2.0.0
canal_node: rancher/calico-node:v3.1.3
canal_cni: rancher/calico-cni:v3.1.3
canal_flannel: rancher/coreos-flannel:v0.10.0
wave_node: weaveworks/weave-kube:2.1.2
weave_cni: weaveworks/weave-npc:2.1.2
pod_infra_container: rancher/pause-amd64:3.1
ingress: rancher/nginx-ingress-controller:0.16.2-rancher1
ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.4
metrics_server: rancher/metrics-server-amd64:v0.2.1
ssh_key_path: ~/.ssh/onap_rsa
ssh_agent_auth: false
authorization:
mode: rbac
options: {}
ignore_docker_version: false
kubernetes_version: ""
private_registries: []
ingress:
provider: ""
options: {}
node_selector: {}
extra_args: {}
cluster_name: ""
cloud_provider:
name: ""
prefix_path: ""
addon_job_timeout: 0
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
monitoring:
provider: ""
options: {}
{noformat}
Kubernetes Single Node Developer Installation
sudo ./rke_install.sh -b master -s localhost -e onap -l ubuntu
Kubernetes HA Cluster Production Installation
Design Issues
DI 20190225-1: RKE/Docker version pair
As of 20190215 RKE 0.16 supports Docker 18.06-ce (and 18.09 non-ce) (up from 0.15 supporting 17.03)
https://github.com/docker/docker-ce/releases/tag/v18.06.3-ce
https://github.com/rancher/rke/releases/tag/v0.1.16
ubuntu@a-rke:~$ sudo rke up INFO[0000] Building Kubernetes cluster INFO[0000] [dialer] Setup tunnel for host [rke.onap.cloud] FATA[0000] Unsupported Docker version found [18.06.3-ce], supported versions are [1.11.x 1.12.x 1.13.x 17.03.x]
DI 20190225-2: RKE upgrade from 0.15 to 0.16 - not working
Does rke remove, regenerate the yaml (or hand upgrade the versions) then rke up
ubuntu@a-rke:~$ sudo rke remove Are you sure you want to remove Kubernetes cluster [y/n]: y INFO[0002] Tearing down Kubernetes cluster INFO[0002] [dialer] Setup tunnel for host [rke.onap.cloud] INFO[0002] [worker] Tearing down Worker Plane.. INFO[0002] [remove/kubelet] Successfully removed container on host [rke.onap.cloud] INFO[0003] [remove/kube-proxy] Successfully removed container on host [rke.onap.cloud] INFO[0003] [remove/service-sidekick] Successfully removed container on host [rke.onap.cloud] INFO[0003] [worker] Successfully tore down Worker Plane.. INFO[0003] [controlplane] Tearing down the Controller Plane.. INFO[0003] [remove/kube-apiserver] Successfully removed container on host [rke.onap.cloud] INFO[0003] [remove/kube-controller-manager] Successfully removed container on host [rke.onap.cloud] INFO[0004] [remove/kube-scheduler] Successfully removed container on host [rke.onap.cloud] INFO[0004] [controlplane] Host [rke.onap.cloud] is already a worker host, skipping delete kubelet and kubeproxy. INFO[0004] [controlplane] Successfully tore down Controller Plane.. INFO[0004] [etcd] Tearing down etcd plane.. INFO[0004] [remove/etcd] Successfully removed container on host [rke.onap.cloud] INFO[0004] [etcd] Successfully tore down etcd plane.. INFO[0004] [hosts] Cleaning up host [rke.onap.cloud] INFO[0004] [hosts] Cleaning up host [rke.onap.cloud] INFO[0004] [hosts] Running cleaner container on host [rke.onap.cloud] INFO[0005] [kube-cleaner] Successfully started [kube-cleaner] container on host [rke.onap.cloud] INFO[0005] [hosts] Removing cleaner container on host [rke.onap.cloud] INFO[0005] [hosts] Removing dead container logs on host [rke.onap.cloud] INFO[0006] [cleanup] Successfully started [rke-log-cleaner] container on host [rke.onap.cloud] INFO[0006] [remove/rke-log-cleaner] Successfully removed container on host [rke.onap.cloud] INFO[0006] [hosts] Successfully cleaned up host [rke.onap.cloud] INFO[0006] [hosts] Cleaning up host [rke.onap.cloud] INFO[0006] [hosts] Cleaning up host [rke.onap.cloud] INFO[0006] [hosts] Running cleaner container on host [rke.onap.cloud] INFO[0007] [kube-cleaner] Successfully started [kube-cleaner] container on host [rke.onap.cloud] INFO[0008] [hosts] Removing cleaner container on host [rke.onap.cloud] INFO[0008] [hosts] Removing dead container logs on host [rke.onap.cloud] INFO[0008] [cleanup] Successfully started [rke-log-cleaner] container on host [rke.onap.cloud] INFO[0009] [remove/rke-log-cleaner] Successfully removed container on host [rke.onap.cloud] INFO[0009] [hosts] Successfully cleaned up host [rke.onap.cloud] INFO[0009] [hosts] Cleaning up host [rke.onap.cloud] INFO[0009] [hosts] Cleaning up host [rke.onap.cloud] INFO[0009] [hosts] Running cleaner container on host [rke.onap.cloud] INFO[0010] [kube-cleaner] Successfully started [kube-cleaner] container on host [rke.onap.cloud] INFO[0010] [hosts] Removing cleaner container on host [rke.onap.cloud] INFO[0010] [hosts] Removing dead container logs on host [rke.onap.cloud] INFO[0011] [cleanup] Successfully started [rke-log-cleaner] container on host [rke.onap.cloud] INFO[0011] [remove/rke-log-cleaner] Successfully removed container on host [rke.onap.cloud] INFO[0011] [hosts] Successfully cleaned up host [rke.onap.cloud] INFO[0011] Removing local admin Kubeconfig: ./kube_config_cluster.yml INFO[0011] Local admin Kubeconfig removed successfully INFO[0011] Cluster removed successfully ubuntu@a-rke:~$ rke config --name cluster.ym ubuntu@a-rke:~$ sudo rke up INFO[0000] Building Kubernetes cluster INFO[0000] [dialer] Setup tunnel for host [rke.onap.cloud] INFO[0000] [network] Deploying port listener containers INFO[0001] [network] Successfully started [rke-etcd-port-listener] container on host [rke.onap.cloud] INFO[0001] [network] Successfully started [rke-cp-port-listener] container on host [rke.onap.cloud] INFO[0002] [network] Successfully started [rke-worker-port-listener] container on host [rke.onap.cloud] INFO[0002] [network] Port listener containers deployed successfully INFO[0002] [network] Running control plane -> etcd port checks INFO[0003] [network] Successfully started [rke-port-checker] container on host [rke.onap.cloud] INFO[0003] [network] Running control plane -> worker port checks INFO[0004] [network] Successfully started [rke-port-checker] container on host [rke.onap.cloud] INFO[0004] [network] Running workers -> control plane port checks INFO[0005] [network] Successfully started [rke-port-checker] container on host [rke.onap.cloud] INFO[0005] [network] Checking KubeAPI port Control Plane hosts INFO[0005] [network] Removing port listener containers INFO[0005] [remove/rke-etcd-port-listener] Successfully removed container on host [rke.onap.cloud] INFO[0006] [remove/rke-cp-port-listener] Successfully removed container on host [rke.onap.cloud] INFO[0006] [remove/rke-worker-port-listener] Successfully removed container on host [rke.onap.cloud] INFO[0006] [network] Port listener containers removed successfully INFO[0006] [certificates] Attempting to recover certificates from backup on [etcd,controlPlane] hosts INFO[0007] [certificates] No Certificate backup found on [etcd,controlPlane] hosts INFO[0007] [certificates] Generating CA kubernetes certificates INFO[0007] [certificates] Generating Kubernetes API server certficates INFO[0008] [certificates] Generating Kube Controller certificates INFO[0008] [certificates] Generating Kube Scheduler certificates INFO[0008] [certificates] Generating Kube Proxy certificates INFO[0009] [certificates] Generating Node certificate INFO[0009] [certificates] Generating admin certificates and kubeconfig INFO[0009] [certificates] Generating etcd-rke.onap.cloud certificate and key INFO[0009] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates INFO[0009] [certificates] Generating Kubernetes API server proxy client certificates INFO[0010] [certificates] Temporarily saving certs to [etcd,controlPlane] hosts INFO[0016] [certificates] Saved certs to [etcd,controlPlane] hosts INFO[0016] [reconcile] Reconciling cluster state INFO[0016] [reconcile] This is newly generated cluster INFO[0016] [certificates] Deploying kubernetes certificates to Cluster nodes INFO[0022] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml] INFO[0022] [certificates] Successfully deployed kubernetes certificates to Cluster nodes INFO[0022] Pre-pulling kubernetes images INFO[0022] Kubernetes images pulled successfully INFO[0022] [etcd] Building up etcd plane.. INFO[0023] [etcd] Successfully started [etcd] container on host [rke.onap.cloud] INFO[0023] [etcd] Saving snapshot [etcd-rolling-snapshots] on host [rke.onap.cloud] INFO[0028] [certificates] Successfully started [rke-bundle-cert] container on host [rke.onap.cloud] INFO[0029] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [rke.onap.cloud] INFO[0029] [etcd] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0030] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0030] [etcd] Successfully started etcd plane.. INFO[0030] [controlplane] Building up Controller Plane.. INFO[0031] [controlplane] Successfully started [kube-apiserver] container on host [rke.onap.cloud] INFO[0031] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [rke.onap.cloud] INFO[0045] [healthcheck] service [kube-apiserver] on host [rke.onap.cloud] is healthy INFO[0046] [controlplane] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0046] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0047] [controlplane] Successfully started [kube-controller-manager] container on host [rke.onap.cloud] INFO[0047] [healthcheck] Start Healthcheck on service [kube-controller-manager] on host [rke.onap.cloud] INFO[0052] [healthcheck] service [kube-controller-manager] on host [rke.onap.cloud] is healthy INFO[0053] [controlplane] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0053] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0054] [controlplane] Successfully started [kube-scheduler] container on host [rke.onap.cloud] INFO[0054] [healthcheck] Start Healthcheck on service [kube-scheduler] on host [rke.onap.cloud] INFO[0059] [healthcheck] service [kube-scheduler] on host [rke.onap.cloud] is healthy INFO[0060] [controlplane] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0060] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0060] [controlplane] Successfully started Controller Plane.. INFO[0060] [authz] Creating rke-job-deployer ServiceAccount INFO[0060] [authz] rke-job-deployer ServiceAccount created successfully INFO[0060] [authz] Creating system:node ClusterRoleBinding INFO[0060] [authz] system:node ClusterRoleBinding created successfully INFO[0060] [certificates] Save kubernetes certificates as secrets INFO[0060] [certificates] Successfully saved certificates as kubernetes secret [k8s-certs] INFO[0060] [state] Saving cluster state to Kubernetes INFO[0061] [state] Successfully Saved cluster state to Kubernetes ConfigMap: cluster-state INFO[0061] [state] Saving cluster state to cluster nodes INFO[0061] [state] Successfully started [cluster-state-deployer] container on host [rke.onap.cloud] INFO[0062] [remove/cluster-state-deployer] Successfully removed container on host [rke.onap.cloud] INFO[0062] [worker] Building up Worker Plane.. INFO[0062] [remove/service-sidekick] Successfully removed container on host [rke.onap.cloud] INFO[0063] [worker] Successfully started [kubelet] container on host [rke.onap.cloud] INFO[0063] [healthcheck] Start Healthcheck on service [kubelet] on host [rke.onap.cloud] INFO[0068] [healthcheck] service [kubelet] on host [rke.onap.cloud] is healthy INFO[0069] [worker] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0070] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0070] [worker] Successfully started [kube-proxy] container on host [rke.onap.cloud] INFO[0070] [healthcheck] Start Healthcheck on service [kube-proxy] on host [rke.onap.cloud] INFO[0076] [healthcheck] service [kube-proxy] on host [rke.onap.cloud] is healthy INFO[0076] [worker] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0077] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0077] [worker] Successfully started Worker Plane.. INFO[0077] [sync] Syncing nodes Labels and Taints INFO[0077] [sync] Successfully synced nodes Labels and Taints INFO[0077] [network] Setting up network plugin: canal INFO[0077] [addons] Saving addon ConfigMap to Kubernetes INFO[0077] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-network-plugin INFO[0077] [addons] Executing deploy job.. INFO[0082] [addons] Setting up KubeDNS INFO[0082] [addons] Saving addon ConfigMap to Kubernetes INFO[0082] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-kubedns-addon INFO[0082] [addons] Executing deploy job.. INFO[0087] [addons] KubeDNS deployed successfully.. INFO[0087] [addons] Setting up Metrics Server INFO[0087] [addons] Saving addon ConfigMap to Kubernetes INFO[0087] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-metrics-addon INFO[0087] [addons] Executing deploy job.. INFO[0092] [addons] KubeDNS deployed successfully.. INFO[0092] [ingress] Setting up nginx ingress controller INFO[0092] [addons] Saving addon ConfigMap to Kubernetes INFO[0092] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-ingress-controller INFO[0092] [addons] Executing deploy job.. INFO[0097] [ingress] ingress controller nginx is successfully deployed INFO[0097] [addons] Setting up user addons INFO[0097] [addons] Checking for included user addons WARN[0097] [addons] Unable to determine if is a file path or url, skipping INFO[0097] [addons] Deploying rke-user-includes-addons INFO[0097] [addons] Saving addon ConfigMap to Kubernetes INFO[0097] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-user-includes-addons INFO[0097] [addons] Executing deploy job.. WARN[0128] Failed to deploy addon execute job [rke-user-includes-addons]: Failed to get job complete status: <nil> INFO[0128] Finished building Kubernetes cluster successfully ubuntu@a-rke:~$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ec26c4bd24b5 846921f0fe0e "/server" 10 minutes ago Up 10 minutes k8s_default-http-backend_default-http-backend-797c5bc547-45msr_ingress-nginx_0eddfe19-394e-11e9-b708-000d3a0e23f3_0 f8d5db205e14 8a7739f672b4 "/sidecar --v=2 --lo…" 10 minutes ago Up 10 minutes k8s_sidecar_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 490461545ae4 rancher/metrics-server-amd64 "/metrics-server --s…" 10 minutes ago Up 10 minutes k8s_metrics-server_metrics-server-97bc649d5-q84tz_kube-system_0c566ec8-394e-11e9-b708-000d3a0e23f3_0 aaf03b62bd41 6816817d9dce "/dnsmasq-nanny -v=2…" 10 minutes ago Up 10 minutes k8s_dnsmasq_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 58ec007db72f 55ffe31ac578 "/kube-dns --domain=…" 10 minutes ago Up 10 minutes k8s_kubedns_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 0a95c06f6aa6 e183460c484d "/cluster-proportion…" 10 minutes ago Up 10 minutes k8s_autoscaler_kube-dns-autoscaler-5db9bbb766-6slz7_kube-system_08b5495c-394e-11e9-b708-000d3a0e23f3_0 968a7c99b210 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_default-http-backend-797c5bc547-45msr_ingress-nginx_0eddfe19-394e-11e9-b708-000d3a0e23f3_0 69969b331e49 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_metrics-server-97bc649d5-q84tz_kube-system_0c566ec8-394e-11e9-b708-000d3a0e23f3_0 baa5f03c16ff rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 82b2a9f640cb rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-dns-autoscaler-5db9bbb766-6slz7_kube-system_08b5495c-394e-11e9-b708-000d3a0e23f3_0 953a4d4be0c1 df4469c42185 "/usr/bin/dumb-init …" 10 minutes ago Up 10 minutes k8s_nginx-ingress-controller_nginx-ingress-controller-dfhp8_ingress-nginx_0ed3bdbf-394e-11e9-b708-000d3a0e23f3_0 cce552840749 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_nginx-ingress-controller-dfhp8_ingress-nginx_0ed3bdbf-394e-11e9-b708-000d3a0e23f3_0 baa65f9c6f97 f0fad859c909 "/opt/bin/flanneld -…" 10 minutes ago Up 10 minutes k8s_kube-flannel_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 1736ce68f41a 9f355e076ea7 "/install-cni.sh" 10 minutes ago Up 10 minutes k8s_install-cni_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 615d3f702ee7 7eca10056c8e "start_runit" 10 minutes ago Up 10 minutes k8s_calico-node_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 1c4a702f0f18 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 0da1cada08e1 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 10 minutes ago Up 10 minutes kube-proxy 57f44998f34a rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kubelet 50f424c4daec rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kube-scheduler 502d327912d9 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kube-controller-manager 9fc706bbf3a5 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kube-apiserver 2e7630c2047c rancher/coreos-etcd:v3.2.18 "/usr/local/bin/etcd…" 11 minutes ago Up 11 minutes etcd fef566337eb6 rancher/rke-tools:v0.1.15 "/opt/rke-tools/rke-…" 26 minutes ago Up 26 minutes etcd-rolling-snapshots amdocs@obriensystemsu0:~$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE ingress-nginx default-http-backend-797c5bc547-m8hbx 1/1 Running 0 1h ingress-nginx nginx-ingress-controller-2v7w7 1/1 Running 0 1h kube-system canal-thmfg 3/3 Running 0 1h kube-system kube-dns-7588d5b5f5-j66s8 3/3 Running 0 1h kube-system kube-dns-autoscaler-5db9bbb766-rg5n8 1/1 Running 0 1h kube-system metrics-server-97bc649d5-jd2rr 1/1 Running 0 1h kube-system rke-ingress-controller-deploy-job-znp9n 0/1 Completed 0 1h kube-system rke-kubedns-addon-deploy-job-dzxsj 0/1 Completed 0 1h kube-system rke-metrics-addon-deploy-job-gpm4j 0/1 Completed 0 1h kube-system rke-network-plugin-deploy-job-kqdds 0/1 Completed 0 1h kube-system tiller-deploy-69458576b-khgr5 1/1 Running 0 1h
DI 20190226-1: RKE up segmentation fault on 0.1.16 - use correct user
amdocs@obriensystemsu0:~$ sudo rke up Segmentation fault (core dumped) # issue was I was using ubuntu as the yml user not amdocs in this case for a particular VM
DI 20190227-1: Verify no 110 pod limit per VM
https://forums.rancher.com/t/solved-setting-max-pods/11866
kubelet:
image: ""
extra_args:
max-pods: 900
DI 20190228-1: deploy casablanca MR to RKE under K8S 1.11.6, Docker 18.06, Helm 2.12.3
sudo git clone https://gerrit.onap.org/r/logging-analytics sudo wget https://git.onap.org/oom/plain/kubernetes/onap/resources/environments/dev.yaml sudo cp dev.yaml dev0.yaml sudo vi dev0.yaml sudo cp dev0.yaml dev1.yaml sudo cp logging-analytics/deploy/cd.sh . sudo ./cd.sh -b casablanca -e onap -p false nexus3.onap.org:10001 -f true -s 300 -c true -d false -w false -r false no good for helm 2.12.3 deployment - just using 2.9.1 for now - Error: Chart incompatible with Tiller v2.12.3 in the casablanca branch only - flip https://git.onap.org/oom/tree/kubernetes/onap/Chart.yaml?h=casablanca#n24 tillerVersion: "~2.9.1"
DI 20190305-1: Azure 256G VM full ONAP Testing
obrienbiometrics:oom michaelobrien$ ssh ubuntu@onap-dmz.onap.cloud ./oom_deployment.sh -b master -s rke.onap.cloud -e onap -r a_rke0_master -t _arm_deploy_onap_cd.json -p _arm_deploy_onap_rke_z_parameters.json
DI 20190425: HA RKE install Testing
Manual first for RC0, later retrofit the script in https://git.onap.org/oom/tree/kubernetes/contrib/tools/rke/rke_setup.sh and move/adjust the heat template in https://git.onap.org/logging-analytics/tree/deploy/heat/logging_openstack_13_16g.yaml
Installing on 6 nodes on AWS (windriver is having an issue right now).
We are good on RKE 0.2.1, Ubuntu 18.04 / Kubernetes/kubectl 1.13.5 / Helm 2.13.1 / Docker 18.09.5
https://github.com/rancher/rke/releases RKE 0.2.2 has experimental k8s 1.14 support - running with 0.2.1 for now
Just need to test onap deployments
I'll do the NFS/EFS later before deployment
# on all VMs (control, etcd, worker)
# move the key to all vms
scp ~/wse_onap/onap_rsa ubuntu@rke0.onap.info:~/
sudo curl https://releases.rancher.com/install-docker/18.09.sh | sh
sudo usermod -aG docker ubuntu
# nfs server
# on control/etcd nodes only
# from script
sudo wget https://github.com/rancher/rke/releases/download/v0.2.1/rke_linux-amd64
mv rke_linux-amd64 rke
sudo chmod +x rke
sudo mv ./rke /usr/local/bin/rke
# one time setup of the yaml or use the generated one
ubuntu@ip-172-31-38-182:~$ sudo rke config
[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: ~/.ssh/onap_rsa
[+] Number of Hosts [1]: 6
[+] SSH Address of host (1) [none]: 3.14.102.175
[+] SSH Port of host (1) [22]:
[+] SSH Private Key Path of host (3.14.102.175) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (3.14.102.175) [ubuntu]: ubuntu
[+] Is host (3.14.102.175) a Control Plane host (y/n)? [y]: y
[+] Is host (3.14.102.175) a Worker host (y/n)? [n]: n
[+] Is host (3.14.102.175) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (3.14.102.175) [none]:
[+] Internal IP of host (3.14.102.175) [none]:
[+] Docker socket path on host (3.14.102.175) [/var/run/docker.sock]:
[+] SSH Address of host (2) [none]: 18.220.62.6
[+] SSH Port of host (2) [22]:
[+] SSH Private Key Path of host (18.220.62.6) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (18.220.62.6) [ubuntu]:
[+] Is host (18.220.62.6) a Control Plane host (y/n)? [y]: y
[+] Is host (18.220.62.6) a Worker host (y/n)? [n]: n
[+] Is host (18.220.62.6) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (18.220.62.6) [none]:
[+] Internal IP of host (18.220.62.6) [none]:
[+] Docker socket path on host (18.220.62.6) [/var/run/docker.sock]:
[+] SSH Address of host (3) [none]: 18.217.96.12
[+] SSH Port of host (3) [22]:
[+] SSH Private Key Path of host (18.217.96.12) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (18.217.96.12) [ubuntu]:
[+] Is host (18.217.96.12) a Control Plane host (y/n)? [y]: y
[+] Is host (18.217.96.12) a Worker host (y/n)? [n]: n
[+] Is host (18.217.96.12) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (18.217.96.12) [none]:
[+] Internal IP of host (18.217.96.12) [none]:
[+] Docker socket path on host (18.217.96.12) [/var/run/docker.sock]:
[+] SSH Address of host (4) [none]: 18.188.214.137
[+] SSH Port of host (4) [22]:
[+] SSH Private Key Path of host (18.188.214.137) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (18.188.214.137) [ubuntu]:
[+] Is host (18.188.214.137) a Control Plane host (y/n)? [y]: n
[+] Is host (18.188.214.137) a Worker host (y/n)? [n]: y
[+] Is host (18.188.214.137) an etcd host (y/n)? [n]: n
[+] Override Hostname of host (18.188.214.137) [none]:
[+] Internal IP of host (18.188.214.137) [none]:
[+] Docker socket path on host (18.188.214.137) [/var/run/docker.sock]:
[+] SSH Address of host (5) [none]: 18.220.70.253
[+] SSH Port of host (5) [22]:
[+] SSH Private Key Path of host (18.220.70.253) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (18.220.70.253) [ubuntu]:
[+] Is host (18.220.70.253) a Control Plane host (y/n)? [y]: n
[+] Is host (18.220.70.253) a Worker host (y/n)? [n]: y
[+] Is host (18.220.70.253) an etcd host (y/n)? [n]: n
[+] Override Hostname of host (18.220.70.253) [none]:
[+] Internal IP of host (18.220.70.253) [none]:
[+] Docker socket path on host (18.220.70.253) [/var/run/docker.sock]:
[+] SSH Address of host (6) [none]: 3.17.76.33
[+] SSH Port of host (6) [22]:
[+] SSH Private Key Path of host (3.17.76.33) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (3.17.76.33) [ubuntu]:
[+] Is host (3.17.76.33) a Control Plane host (y/n)? [y]: n
[+] Is host (3.17.76.33) a Worker host (y/n)? [n]: y
[+] Is host (3.17.76.33) an etcd host (y/n)? [n]: n
[+] Override Hostname of host (3.17.76.33) [none]:
[+] Internal IP of host (3.17.76.33) [none]:
[+] Docker socket path on host (3.17.76.33) [/var/run/docker.sock]:
[+] Network Plugin Type (flannel, calico, weave, canal) [canal]:
[+] Authentication Strategy [x509]:
[+] Authorization Mode (rbac, none) [rbac]:
[+] Kubernetes Docker image [rancher/hyperkube:v1.13.5-rancher1]:
[+] Cluster domain [cluster.local]:
[+] Service Cluster IP Range [10.43.0.0/16]:
[+] Enable PodSecurityPolicy [n]:
[+] Cluster Network CIDR [10.42.0.0/16]:
[+] Cluster DNS Service IP [10.43.0.10]:
[+] Add addon manifest URLs or YAML files [no]:
# new
[+] Cluster domain [cluster.local]:
ubuntu@ip-172-31-38-182:~$ sudo rke up
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [certificates] Generating CA kubernetes certificates
INFO[0000] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates
INFO[0000] [certificates] Generating Kubernetes API server certificates
INFO[0000] [certificates] Generating Service account token key
INFO[0000] [certificates] Generating etcd-3.14.102.175 certificate and key
INFO[0000] [certificates] Generating etcd-18.220.62.6 certificate and key
INFO[0001] [certificates] Generating etcd-18.217.96.12 certificate and key
INFO[0001] [certificates] Generating Kube Controller certificates
INFO[0001] [certificates] Generating Kube Scheduler certificates
INFO[0001] [certificates] Generating Kube Proxy certificates
INFO[0001] [certificates] Generating Node certificate
INFO[0001] [certificates] Generating admin certificates and kubeconfig
INFO[0001] [certificates] Generating Kubernetes API server proxy client certificates
INFO[0002] Successfully Deployed state file at [./cluster.rkestate]
INFO[0002] Building Kubernetes cluster
INFO[0002] [dialer] Setup tunnel for host [3.14.102.175]
INFO[0002] [dialer] Setup tunnel for host [18.188.214.137]
INFO[0002] [dialer] Setup tunnel for host [18.220.70.253]
INFO[0002] [dialer] Setup tunnel for host [18.220.62.6]
INFO[0002] [dialer] Setup tunnel for host [18.217.96.12]
INFO[0002] [dialer] Setup tunnel for host [3.17.76.33]
INFO[0002] [network] Deploying port listener containers
INFO[0002] [network] Pulling image [rancher/rke-tools:v0.1.27] on host [18.220.62.6]
INFO[0002] [network] Pulling image [rancher/rke-tools:v0.1.27] on host [3.14.102.175]
INFO[0002] [network] Pulling image [rancher/rke-tools:v0.1.27] on host [18.217.96.12]
INFO[0006] [network] Successfully pulled image [rancher/rke-tools:v0.1.27] on host [18.220.62.6]
INFO[0006] [network] Successfully pulled image [rancher/rke-tools:v0.1.27] on host [18.217.96.12]
INFO[0006] [network] Successfully pulled image [rancher/rke-tools:v0.1.27] on host [3.14.102.175]
INFO[0007] [network] Successfully started [rke-etcd-port-listener] container on host [18.220.62.6]
INFO[0007] [network] Successfully started [rke-etcd-port-listener] container on host [18.217.96.12]
INFO[0007] [network] Successfully started [rke-etcd-port-listener] container on host [3.14.102.175]
INFO[0008] [network] Successfully started [rke-cp-port-listener] container on host [18.217.96.12]
INFO[0008] [network] Successfully started [rke-cp-port-listener] container on host [18.220.62.6]
INFO[0008] [network] Successfully started [rke-cp-port-listener] container on host [3.14.102.175]
INFO[0008] [network] Pulling image [rancher/rke-tools:v0.1.27] on host [18.188.214.137]
INFO[0008] [network] Pulling image [rancher/rke-tools:v0.1.27] on host [18.220.70.253]
INFO[0008] [network] Pulling image [rancher/rke-tools:v0.1.27] on host [3.17.76.33]
INFO[0012] [network] Successfully pulled image [rancher/rke-tools:v0.1.27] on host [3.17.76.33]
INFO[0012] [network] Successfully pulled image [rancher/rke-tools:v0.1.27] on host [18.220.70.253]
INFO[0012] [network] Successfully pulled image [rancher/rke-tools:v0.1.27] on host [18.188.214.137]
INFO[0013] [network] Successfully started [rke-worker-port-listener] container on host [3.17.76.33]
INFO[0013] [network] Successfully started [rke-worker-port-listener] container on host [18.220.70.253]
INFO[0013] [network] Successfully started [rke-worker-port-listener] container on host [18.188.214.137]
INFO[0013] [network] Port listener containers deployed successfully
INFO[0013] [network] Running etcd <-> etcd port checks
INFO[0013] [network] Successfully started [rke-port-checker] container on host [18.220.62.6]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [3.14.102.175]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [18.217.96.12]
INFO[0014] [network] Running control plane -> etcd port checks
INFO[0014] [network] Successfully started [rke-port-checker] container on host [18.220.62.6]
INFO[0014] [network] Successfully started [rke-port-checker] container on host [3.14.102.175]
INFO[0014] [network] Successfully started [rke-port-checker] container on host [18.217.96.12]
INFO[0014] [network] Running control plane -> worker port checks
INFO[0015] [network] Successfully started [rke-port-checker] container on host [18.220.62.6]
INFO[0015] [network] Successfully started [rke-port-checker] container on host [18.217.96.12]
INFO[0015] [network] Successfully started [rke-port-checker] container on host [3.14.102.175]
INFO[0015] [network] Running workers -> control plane port checks
INFO[0015] [network] Successfully started [rke-port-checker] container on host [3.17.76.33]
INFO[0015] [network] Successfully started [rke-port-checker] container on host [18.220.70.253]
INFO[0015] [network] Successfully started [rke-port-checker] container on host [18.188.214.137]
INFO[0016] [network] Checking KubeAPI port Control Plane hosts
INFO[0016] [network] Removing port listener containers
INFO[0016] [remove/rke-etcd-port-listener] Successfully removed container on host [18.220.62.6]
INFO[0016] [remove/rke-etcd-port-listener] Successfully removed container on host [3.14.102.175]
INFO[0016] [remove/rke-etcd-port-listener] Successfully removed container on host [18.217.96.12]
INFO[0016] [remove/rke-cp-port-listener] Successfully removed container on host [18.217.96.12]
INFO[0016] [remove/rke-cp-port-listener] Successfully removed container on host [3.14.102.175]
INFO[0016] [remove/rke-cp-port-listener] Successfully removed container on host [18.220.62.6]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [18.220.70.253]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [3.17.76.33]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [18.188.214.137]
INFO[0017] [network] Port listener containers removed successfully
INFO[0017] [certificates] Deploying kubernetes certificates to Cluster nodes
INFO[0022] [reconcile] Rebuilding and updating local kube config
INFO[0022] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
INFO[0022] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
INFO[0022] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
INFO[0022] [certificates] Successfully deployed kubernetes certificates to Cluster nodes
INFO[0022] [reconcile] Reconciling cluster state
INFO[0022] [reconcile] This is newly generated cluster
INFO[0022] Pre-pulling kubernetes images
INFO[0022] [pre-deploy] Pulling image [rancher/hyperkube:v1.13.5-rancher1] on host [3.14.102.175]
INFO[0022] [pre-deploy] Pulling image [rancher/hyperkube:v1.13.5-rancher1] on host [18.188.214.137]
INFO[0022] [pre-deploy] Pulling image [rancher/hyperkube:v1.13.5-rancher1] on host [18.220.70.253]
INFO[0022] [pre-deploy] Pulling image [rancher/hyperkube:v1.13.5-rancher1] on host [18.217.96.12]
INFO[0022] [pre-deploy] Pulling image [rancher/hyperkube:v1.13.5-rancher1] on host [3.17.76.33]
INFO[0022] [pre-deploy] Pulling image [rancher/hyperkube:v1.13.5-rancher1] on host [18.220.62.6]
INFO[0038] [pre-deploy] Successfully pulled image [rancher/hyperkube:v1.13.5-rancher1] on host [18.220.62.6]
INFO[0038] [pre-deploy] Successfully pulled image [rancher/hyperkube:v1.13.5-rancher1] on host [18.220.70.253]
INFO[0039] [pre-deploy] Successfully pulled image [rancher/hyperkube:v1.13.5-rancher1] on host [3.17.76.33]
INFO[0039] [pre-deploy] Successfully pulled image [rancher/hyperkube:v1.13.5-rancher1] on host [18.217.96.12]
INFO[0039] [pre-deploy] Successfully pulled image [rancher/hyperkube:v1.13.5-rancher1] on host [18.188.214.137]
INFO[0039] [pre-deploy] Successfully pulled image [rancher/hyperkube:v1.13.5-rancher1] on host [3.14.102.175]
INFO[0039] Kubernetes images pulled successfully
INFO[0039] [etcd] Building up etcd plane..
INFO[0039] [etcd] Pulling image [rancher/coreos-etcd:v3.2.24-rancher1] on host [3.14.102.175]
INFO[0041] [etcd] Successfully pulled image [rancher/coreos-etcd:v3.2.24-rancher1] on host [3.14.102.175]
INFO[0051] [etcd] Successfully started [etcd] container on host [3.14.102.175]
INFO[0051] [etcd] Saving snapshot [etcd-rolling-snapshots] on host [3.14.102.175]
INFO[0052] [etcd] Successfully started [etcd-rolling-snapshots] container on host [3.14.102.175]
INFO[0057] [certificates] Successfully started [rke-bundle-cert] container on host [3.14.102.175]
INFO[0058] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [3.14.102.175]
INFO[0058] [etcd] Successfully started [rke-log-linker] container on host [3.14.102.175]
INFO[0058] [remove/rke-log-linker] Successfully removed container on host [3.14.102.175]
INFO[0058] [etcd] Pulling image [rancher/coreos-etcd:v3.2.24-rancher1] on host [18.220.62.6]
INFO[0063] [etcd] Successfully pulled image [rancher/coreos-etcd:v3.2.24-rancher1] on host [18.220.62.6]
INFO[0069] [etcd] Successfully started [etcd] container on host [18.220.62.6]
INFO[0069] [etcd] Saving snapshot [etcd-rolling-snapshots] on host [18.220.62.6]
INFO[0069] [etcd] Successfully started [etcd-rolling-snapshots] container on host [18.220.62.6]
INFO[0075] [certificates] Successfully started [rke-bundle-cert] container on host [18.220.62.6]
INFO[0075] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [18.220.62.6]
INFO[0076] [etcd] Successfully started [rke-log-linker] container on host [18.220.62.6]
INFO[0076] [remove/rke-log-linker] Successfully removed container on host [18.220.62.6]
INFO[0076] [etcd] Pulling image [rancher/coreos-etcd:v3.2.24-rancher1] on host [18.217.96.12]
INFO[0078] [etcd] Successfully pulled image [rancher/coreos-etcd:v3.2.24-rancher1] on host [18.217.96.12]
INFO[0078] [etcd] Successfully started [etcd] container on host [18.217.96.12]
INFO[0078] [etcd] Saving snapshot [etcd-rolling-snapshots] on host [18.217.96.12]
INFO[0078] [etcd] Successfully started [etcd-rolling-snapshots] container on host [18.217.96.12]
INFO[0084] [certificates] Successfully started [rke-bundle-cert] container on host [18.217.96.12]
INFO[0084] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [18.217.96.12]
INFO[0085] [etcd] Successfully started [rke-log-linker] container on host [18.217.96.12]
INFO[0085] [remove/rke-log-linker] Successfully removed container on host [18.217.96.12]
INFO[0085] [etcd] Successfully started etcd plane.. Checking etcd cluster health
INFO[0086] [controlplane] Building up Controller Plane..
INFO[0086] [controlplane] Successfully started [kube-apiserver] container on host [18.220.62.6]
INFO[0086] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [18.220.62.6]
INFO[0086] [controlplane] Successfully started [kube-apiserver] container on host [3.14.102.175]
INFO[0086] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [3.14.102.175]
INFO[0086] [controlplane] Successfully started [kube-apiserver] container on host [18.217.96.12]
INFO[0086] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [18.217.96.12]
INFO[0098] [healthcheck] service [kube-apiserver] on host [18.220.62.6] is healthy
INFO[0099] [healthcheck] service [kube-apiserver] on host [18.217.96.12] is healthy
INFO[0099] [controlplane] Successfully started [rke-log-linker] container on host [18.220.62.6]
INFO[0099] [controlplane] Successfully started [rke-log-linker] container on host [18.217.96.12]
INFO[0099] [remove/rke-log-linker] Successfully removed container on host [18.220.62.6]
INFO[0099] [healthcheck] service [kube-apiserver] on host [3.14.102.175] is healthy
INFO[0099] [remove/rke-log-linker] Successfully removed container on host [18.217.96.12]
INFO[0099] [controlplane] Successfully started [kube-controller-manager] container on host [18.220.62.6]
INFO[0099] [healthcheck] Start Healthcheck on service [kube-controller-manager] on host [18.220.62.6]
INFO[0100] [controlplane] Successfully started [kube-controller-manager] container on host [18.217.96.12]
INFO[0100] [healthcheck] Start Healthcheck on service [kube-controller-manager] on host [18.217.96.12]
INFO[0100] [controlplane] Successfully started [rke-log-linker] container on host [3.14.102.175]
INFO[0100] [remove/rke-log-linker] Successfully removed container on host [3.14.102.175]
INFO[0100] [healthcheck] service [kube-controller-manager] on host [18.220.62.6] is healthy
INFO[0100] [healthcheck] service [kube-controller-manager] on host [18.217.96.12] is healthy
INFO[0100] [controlplane] Successfully started [kube-controller-manager] container on host [3.14.102.175]
INFO[0100] [healthcheck] Start Healthcheck on service [kube-controller-manager] on host [3.14.102.175]
INFO[0101] [controlplane] Successfully started [rke-log-linker] container on host [18.220.62.6]
INFO[0101] [controlplane] Successfully started [rke-log-linker] container on host [18.217.96.12]
INFO[0101] [remove/rke-log-linker] Successfully removed container on host [18.220.62.6]
INFO[0101] [remove/rke-log-linker] Successfully removed container on host [18.217.96.12]
INFO[0101] [controlplane] Successfully started [kube-scheduler] container on host [18.220.62.6]
INFO[0101] [healthcheck] Start Healthcheck on service [kube-scheduler] on host [18.220.62.6]
INFO[0101] [healthcheck] service [kube-controller-manager] on host [3.14.102.175] is healthy
INFO[0101] [controlplane] Successfully started [kube-scheduler] container on host [18.217.96.12]
INFO[0101] [healthcheck] Start Healthcheck on service [kube-scheduler] on host [18.217.96.12]
INFO[0102] [controlplane] Successfully started [rke-log-linker] container on host [3.14.102.175]
INFO[0102] [healthcheck] service [kube-scheduler] on host [18.220.62.6] is healthy
INFO[0102] [remove/rke-log-linker] Successfully removed container on host [3.14.102.175]
INFO[0102] [healthcheck] service [kube-scheduler] on host [18.217.96.12] is healthy
INFO[0103] [controlplane] Successfully started [kube-scheduler] container on host [3.14.102.175]
INFO[0103] [healthcheck] Start Healthcheck on service [kube-scheduler] on host [3.14.102.175]
INFO[0103] [controlplane] Successfully started [rke-log-linker] container on host [18.220.62.6]
INFO[0103] [remove/rke-log-linker] Successfully removed container on host [18.220.62.6]
INFO[0103] [controlplane] Successfully started [rke-log-linker] container on host [18.217.96.12]
INFO[0103] [remove/rke-log-linker] Successfully removed container on host [18.217.96.12]
INFO[0103] [healthcheck] service [kube-scheduler] on host [3.14.102.175] is healthy
INFO[0104] [controlplane] Successfully started [rke-log-linker] container on host [3.14.102.175]
INFO[0104] [remove/rke-log-linker] Successfully removed container on host [3.14.102.175]
INFO[0104] [controlplane] Successfully started Controller Plane..
INFO[0104] [authz] Creating rke-job-deployer ServiceAccount
INFO[0104] [authz] rke-job-deployer ServiceAccount created successfully
INFO[0104] [authz] Creating system:node ClusterRoleBinding
INFO[0104] [authz] system:node ClusterRoleBinding created successfully
INFO[0104] Successfully Deployed state file at [./cluster.rkestate]
INFO[0104] [state] Saving full cluster state to Kubernetes
INFO[0104] [state] Successfully Saved full cluster state to Kubernetes ConfigMap: cluster-state
INFO[0104] [worker] Building up Worker Plane..
INFO[0104] [sidekick] Sidekick container already created on host [18.220.62.6]
INFO[0104] [sidekick] Sidekick container already created on host [3.14.102.175]
INFO[0104] [sidekick] Sidekick container already created on host [18.217.96.12]
INFO[0105] [worker] Successfully started [kubelet] container on host [3.14.102.175]
INFO[0105] [healthcheck] Start Healthcheck on service [kubelet] on host [3.14.102.175]
INFO[0105] [worker] Successfully started [kubelet] container on host [18.220.62.6]
INFO[0105] [healthcheck] Start Healthcheck on service [kubelet] on host [18.220.62.6]
INFO[0105] [worker] Successfully started [kubelet] container on host [18.217.96.12]
INFO[0105] [healthcheck] Start Healthcheck on service [kubelet] on host [18.217.96.12]
INFO[0105] [worker] Successfully started [nginx-proxy] container on host [18.220.70.253]
INFO[0105] [worker] Successfully started [nginx-proxy] container on host [3.17.76.33]
INFO[0105] [worker] Successfully started [nginx-proxy] container on host [18.188.214.137]
INFO[0106] [worker] Successfully started [rke-log-linker] container on host [18.220.70.253]
INFO[0106] [worker] Successfully started [rke-log-linker] container on host [3.17.76.33]
INFO[0106] [worker] Successfully started [rke-log-linker] container on host [18.188.214.137]
INFO[0106] [remove/rke-log-linker] Successfully removed container on host [3.17.76.33]
INFO[0106] [remove/rke-log-linker] Successfully removed container on host [18.220.70.253]
INFO[0106] [remove/rke-log-linker] Successfully removed container on host [18.188.214.137]
INFO[0106] [worker] Successfully started [kubelet] container on host [3.17.76.33]
INFO[0106] [healthcheck] Start Healthcheck on service [kubelet] on host [3.17.76.33]
INFO[0107] [worker] Successfully started [kubelet] container on host [18.220.70.253]
INFO[0107] [healthcheck] Start Healthcheck on service [kubelet] on host [18.220.70.253]
INFO[0107] [worker] Successfully started [kubelet] container on host [18.188.214.137]
INFO[0107] [healthcheck] Start Healthcheck on service [kubelet] on host [18.188.214.137]
INFO[0111] [healthcheck] service [kubelet] on host [18.220.62.6] is healthy
INFO[0111] [healthcheck] service [kubelet] on host [18.217.96.12] is healthy
INFO[0111] [healthcheck] service [kubelet] on host [3.14.102.175] is healthy
INFO[0111] [worker] Successfully started [rke-log-linker] container on host [18.220.62.6]
INFO[0111] [worker] Successfully started [rke-log-linker] container on host [3.14.102.175]
INFO[0111] [worker] Successfully started [rke-log-linker] container on host [18.217.96.12]
INFO[0112] [remove/rke-log-linker] Successfully removed container on host [18.220.62.6]
INFO[0112] [remove/rke-log-linker] Successfully removed container on host [18.217.96.12]
INFO[0112] [remove/rke-log-linker] Successfully removed container on host [3.14.102.175]
INFO[0112] [worker] Successfully started [kube-proxy] container on host [18.220.62.6]
INFO[0112] [healthcheck] Start Healthcheck on service [kube-proxy] on host [18.220.62.6]
INFO[0112] [worker] Successfully started [kube-proxy] container on host [18.217.96.12]
INFO[0112] [healthcheck] Start Healthcheck on service [kube-proxy] on host [18.217.96.12]
INFO[0112] [worker] Successfully started [kube-proxy] container on host [3.14.102.175]
INFO[0112] [healthcheck] Start Healthcheck on service [kube-proxy] on host [3.14.102.175]
INFO[0113] [healthcheck] service [kube-proxy] on host [18.220.62.6] is healthy
INFO[0113] [healthcheck] service [kube-proxy] on host [18.217.96.12] is healthy
INFO[0113] [healthcheck] service [kube-proxy] on host [3.14.102.175] is healthy
INFO[0113] [healthcheck] service [kubelet] on host [18.220.70.253] is healthy
INFO[0113] [healthcheck] service [kubelet] on host [3.17.76.33] is healthy
INFO[0113] [healthcheck] service [kubelet] on host [18.188.214.137] is healthy
INFO[0113] [worker] Successfully started [rke-log-linker] container on host [18.220.62.6]
INFO[0113] [worker] Successfully started [rke-log-linker] container on host [18.217.96.12]
INFO[0113] [worker] Successfully started [rke-log-linker] container on host [3.14.102.175]
INFO[0113] [worker] Successfully started [rke-log-linker] container on host [3.17.76.33]
INFO[0113] [worker] Successfully started [rke-log-linker] container on host [18.220.70.253]
INFO[0113] [remove/rke-log-linker] Successfully removed container on host [18.220.62.6]
INFO[0113] [worker] Successfully started [rke-log-linker] container on host [18.188.214.137]
INFO[0113] [remove/rke-log-linker] Successfully removed container on host [18.217.96.12]
INFO[0113] [remove/rke-log-linker] Successfully removed container on host [3.14.102.175]
INFO[0114] [remove/rke-log-linker] Successfully removed container on host [3.17.76.33]
INFO[0114] [remove/rke-log-linker] Successfully removed container on host [18.220.70.253]
INFO[0114] [remove/rke-log-linker] Successfully removed container on host [18.188.214.137]
INFO[0114] [worker] Successfully started [kube-proxy] container on host [3.17.76.33]
INFO[0114] [healthcheck] Start Healthcheck on service [kube-proxy] on host [3.17.76.33]
INFO[0114] [worker] Successfully started [kube-proxy] container on host [18.220.70.253]
INFO[0114] [healthcheck] Start Healthcheck on service [kube-proxy] on host [18.220.70.253]
INFO[0114] [worker] Successfully started [kube-proxy] container on host [18.188.214.137]
INFO[0114] [healthcheck] Start Healthcheck on service [kube-proxy] on host [18.188.214.137]
INFO[0114] [healthcheck] service [kube-proxy] on host [18.220.70.253] is healthy
INFO[0114] [healthcheck] service [kube-proxy] on host [3.17.76.33] is healthy
INFO[0115] [healthcheck] service [kube-proxy] on host [18.188.214.137] is healthy
INFO[0115] [worker] Successfully started [rke-log-linker] container on host [18.220.70.253]
INFO[0115] [worker] Successfully started [rke-log-linker] container on host [3.17.76.33]
INFO[0115] [worker] Successfully started [rke-log-linker] container on host [18.188.214.137]
INFO[0115] [remove/rke-log-linker] Successfully removed container on host [18.220.70.253]
INFO[0115] [remove/rke-log-linker] Successfully removed container on host [3.17.76.33]
INFO[0115] [remove/rke-log-linker] Successfully removed container on host [18.188.214.137]
INFO[0115] [worker] Successfully started Worker Plane..
INFO[0116] [cleanup] Successfully started [rke-log-cleaner] container on host [18.188.214.137]
INFO[0116] [cleanup] Successfully started [rke-log-cleaner] container on host [3.17.76.33]
INFO[0116] [cleanup] Successfully started [rke-log-cleaner] container on host [18.220.70.253]
INFO[0116] [cleanup] Successfully started [rke-log-cleaner] container on host [18.220.62.6]
INFO[0116] [cleanup] Successfully started [rke-log-cleaner] container on host [18.217.96.12]
INFO[0116] [cleanup] Successfully started [rke-log-cleaner] container on host [3.14.102.175]
INFO[0116] [remove/rke-log-cleaner] Successfully removed container on host [3.17.76.33]
INFO[0116] [remove/rke-log-cleaner] Successfully removed container on host [18.188.214.137]
INFO[0116] [remove/rke-log-cleaner] Successfully removed container on host [18.220.62.6]
INFO[0116] [remove/rke-log-cleaner] Successfully removed container on host [18.220.70.253]
INFO[0116] [remove/rke-log-cleaner] Successfully removed container on host [18.217.96.12]
INFO[0116] [remove/rke-log-cleaner] Successfully removed container on host [3.14.102.175]
INFO[0116] [sync] Syncing nodes Labels and Taints
INFO[0117] [sync] Successfully synced nodes Labels and Taints
INFO[0117] [network] Setting up network plugin: canal
INFO[0117] [addons] Saving ConfigMap for addon rke-network-plugin to Kubernetes
INFO[0117] [addons] Successfully saved ConfigMap for addon rke-network-plugin to Kubernetes
INFO[0117] [addons] Executing deploy job rke-network-plugin
INFO[0122] [addons] Setting up kube-dns
INFO[0122] [addons] Saving ConfigMap for addon rke-kube-dns-addon to Kubernetes
INFO[0122] [addons] Successfully saved ConfigMap for addon rke-kube-dns-addon to Kubernetes
INFO[0122] [addons] Executing deploy job rke-kube-dns-addon
INFO[0127] [addons] kube-dns deployed successfully
INFO[0127] [dns] DNS provider kube-dns deployed successfully
INFO[0127] [addons] Setting up Metrics Server
INFO[0127] [addons] Saving ConfigMap for addon rke-metrics-addon to Kubernetes
INFO[0127] [addons] Successfully saved ConfigMap for addon rke-metrics-addon to Kubernetes
INFO[0127] [addons] Executing deploy job rke-metrics-addon
INFO[0132] [addons] Metrics Server deployed successfully
INFO[0132] [ingress] Setting up nginx ingress controller
INFO[0132] [addons] Saving ConfigMap for addon rke-ingress-controller to Kubernetes
INFO[0132] [addons] Successfully saved ConfigMap for addon rke-ingress-controller to Kubernetes
INFO[0132] [addons] Executing deploy job rke-ingress-controller
INFO[0137] [ingress] ingress controller nginx deployed successfully
INFO[0137] [addons] Setting up user addons
INFO[0137] [addons] no user addons defined
INFO[0137] Finished building Kubernetes cluster successfully
# finish kubectl install
sudo curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.13.5/bin/linux/amd64/kubectl
sudo chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
sudo mkdir ~/.kube
# finish helm
#https://github.com/helm/helm/releases
# there is no helm 2.12.5 - last is 2.12.3 - trying 2.13.1
wget http://storage.googleapis.com/kubernetes-helm/helm-v2.13.1-linux-amd64.tar.gz
sudo tar -zxvf helm-v2.13.1-linux-amd64.tar.gz
sudo cp kube_config_cluster.yml ~/.kube/config
sudo chmod 777 ~/.kube/config
# test
ubuntu@ip-172-31-38-182:~$ kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx default-http-backend-78fccfc5d9-f6z25 1/1 Running 0 17m 10.42.5.2 18.220.70.253 <none> <none>
ingress-nginx nginx-ingress-controller-2zxs7 1/1 Running 0 17m 18.188.214.137 18.188.214.137 <none> <none>
ingress-nginx nginx-ingress-controller-6b7gs 1/1 Running 0 17m 3.17.76.33 3.17.76.33 <none> <none>
ingress-nginx nginx-ingress-controller-nv4qg 1/1 Running 0 17m 18.220.70.253 18.220.70.253 <none> <none>
kube-system canal-48579 2/2 Running 0 17m 18.220.62.6 18.220.62.6 <none> <none>
kube-system canal-6skkm 2/2 Running 0 17m 18.188.214.137 18.188.214.137 <none> <none>
kube-system canal-9xmxv 2/2 Running 0 17m 18.217.96.12 18.217.96.12 <none> <none>
kube-system canal-c582x 2/2 Running 0 17m 18.220.70.253 18.220.70.253 <none> <none>
kube-system canal-whbck 2/2 Running 0 17m 3.14.102.175 3.14.102.175 <none> <none>
kube-system canal-xbbnh 2/2 Running 0 17m 3.17.76.33 3.17.76.33 <none> <none>
kube-system kube-dns-58bd5b8dd7-6mcm7 3/3 Running 0 17m 10.42.3.3 3.17.76.33 <none> <none>
kube-system kube-dns-58bd5b8dd7-cd5dg 3/3 Running 0 17m 10.42.4.2 18.188.214.137 <none> <none>
kube-system kube-dns-autoscaler-77bc5fd84-p4zfd 1/1 Running 0 17m 10.42.3.2 3.17.76.33 <none> <none>
kube-system metrics-server-58bd5dd8d7-kftjn 1/1 Running 0 17m 10.42.3.4 3.17.76.33 <none> <none>
# install tiller
ubuntu@ip-172-31-38-182:~$ kubectl -n kube-system create serviceaccount tiller
serviceaccount/tiller created
ubuntu@ip-172-31-38-182:~$ kubectl create clusterrolebinding tiller --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
clusterrolebinding.rbac.authorization.k8s.io/tiller created
ubuntu@ip-172-31-38-182:~$ helm init --service-account tiller
Creating /home/ubuntu/.helm
Creating /home/ubuntu/.helm/repository
Creating /home/ubuntu/.helm/repository/cache
Creating /home/ubuntu/.helm/repository/local
Creating /home/ubuntu/.helm/plugins
Creating /home/ubuntu/.helm/starters
Creating /home/ubuntu/.helm/cache/archive
Creating /home/ubuntu/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /home/ubuntu/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!
ubuntu@ip-172-31-38-182:~$ kubectl -n kube-system rollout status deploy/tiller-deploy
deployment "tiller-deploy" successfully rolled out
ubuntu@ip-172-31-38-182:~$ sudo helm init --upgrade
$HELM_HOME has been configured at /home/ubuntu/.helm.
Tiller (the Helm server-side component) has been upgraded to the current version.
Happy Helming!
ubuntu@ip-172-31-38-182:~$ sudo helm version
Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
ubuntu@ip-172-31-38-182:~$ sudo helm serve &
[1] 706
ubuntu@ip-172-31-38-182:~$ Regenerating index. This may take a moment.
Now serving you on 127.0.0.1:8879
ubuntu@ip-172-31-38-182:~$ sudo helm list
ubuntu@ip-172-31-38-182:~$ sudo helm repo add local http://127.0.0.1:8879
"local" has been added to your repositories
ubuntu@ip-172-31-38-182:~$ sudo helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879
ubuntu@ip-172-31-38-182:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
18.188.214.137 Ready worker 22m v1.13.5 18.188.214.137 <none> Ubuntu 18.04.1 LTS 4.15.0-1021-aws docker://18.9.5
18.217.96.12 Ready controlplane,etcd 22m v1.13.5 18.217.96.12 <none> Ubuntu 18.04.1 LTS 4.15.0-1021-aws docker://18.9.5
18.220.62.6 Ready controlplane,etcd 22m v1.13.5 18.220.62.6 <none> Ubuntu 18.04.1 LTS 4.15.0-1021-aws docker://18.9.5
18.220.70.253 Ready worker 22m v1.13.5 18.220.70.253 <none> Ubuntu 18.04.1 LTS 4.15.0-1021-aws docker://18.9.5
3.14.102.175 Ready controlplane,etcd 22m v1.13.5 3.14.102.175 <none> Ubuntu 18.04.1 LTS 4.15.0-1021-aws docker://18.9.5
3.17.76.33 Ready worker 22m v1.13.5 3.17.76.33 <none> Ubuntu 18.04.1 LTS 4.15.0-1021-aws docker://18.9.5
# install make
sudo apt-get install make -y
# install nfs/efs
ubuntu@ip-172-31-38-182:~$ sudo apt-get install nfs-common -y
ubuntu@ip-172-31-38-182:~$ sudo mkdir /dockerdata-nfs
ubuntu@ip-172-31-38-182:~$ sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-5fd6ab26.efs.us-east-2.amazonaws.com:/ /dockerdata-nfs
# check
sudo nohup ./cd.sh -b master -e onap -p false -n nexus3.onap.org:10001 -f false -s 600 -c false -d false -w false -r false &
ubuntu@ip-172-31-38-182:~$ kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx default-http-backend-78fccfc5d9-f6z25 1/1 Running 0 103m 10.42.5.2 18.220.70.253 <none> <none>
ingress-nginx nginx-ingress-controller-2zxs7 1/1 Running 0 103m 18.188.214.137 18.188.214.137 <none> <none>
ingress-nginx nginx-ingress-controller-6b7gs 1/1 Running 0 103m 3.17.76.33 3.17.76.33 <none> <none>
ingress-nginx nginx-ingress-controller-nv4qg 1/1 Running 0 103m 18.220.70.253 18.220.70.253 <none> <none>
kube-system canal-48579 2/2 Running 0 103m 18.220.62.6 18.220.62.6 <none> <none>
kube-system canal-6skkm 2/2 Running 0 103m 18.188.214.137 18.188.214.137 <none> <none>
kube-system canal-9xmxv 2/2 Running 0 103m 18.217.96.12 18.217.96.12 <none> <none>
kube-system canal-c582x 2/2 Running 0 103m 18.220.70.253 18.220.70.253 <none> <none>
kube-system canal-whbck 2/2 Running 0 103m 3.14.102.175 3.14.102.175 <none> <none>
kube-system canal-xbbnh 2/2 Running 0 103m 3.17.76.33 3.17.76.33 <none> <none>
kube-system kube-dns-58bd5b8dd7-6mcm7 3/3 Running 0 103m 10.42.3.3 3.17.76.33 <none> <none>
kube-system kube-dns-58bd5b8dd7-cd5dg 3/3 Running 0 103m 10.42.4.2 18.188.214.137 <none> <none>
kube-system kube-dns-autoscaler-77bc5fd84-p4zfd 1/1 Running 0 103m 10.42.3.2 3.17.76.33 <none> <none>
kube-system metrics-server-58bd5dd8d7-kftjn 1/1 Running 0 103m 10.42.3.4 3.17.76.33 <none> <none>
kube-system tiller-deploy-5f4fc5bcc6-gc4tc 1/1 Running 0 84m 10.42.5.3 18.220.70.253 <none> <none>
onap onap-aai-aai-587cb79c6d-mzpbs 0/1 Init:0/1 1 13m 10.42.5.17 18.220.70.253 <none> <none>
onap onap-aai-aai-babel-8c755bcfc-kmzdm 2/2 Running 0 29m 10.42.5.8 18.220.70.253 <none> <none>
onap onap-aai-aai-champ-78b9d7f68b-98tm9 0/2 Init:0/1 2 29m 10.42.3.6 3.17.76.33 <none> <none>
onap onap-aai-aai-data-router-64fcfbc5bb-wkkvz 1/2 CrashLoopBackOff 9 29m 10.42.5.7 18.220.70.253 <none> <none>
onap onap-aai-aai-elasticsearch-6dcf5d9966-j7z67 1/1 Running 0 29m 10.42.4.4 18.188.214.137 <none> <none>
onap onap-aai-aai-gizmo-5bddb87589-zn8pl 2/2 Running 0 29m 10.42.4.3 18.188.214.137 <none> <none>
onap onap-aai-aai-graphadmin-774f9d698f-f8lwv 0/2 Init:0/1 2 29m 10.42.5.4 18.220.70.253 <none> <none>
onap onap-aai-aai-graphadmin-create-db-schema-94q4l 0/1 Init:Error 0 18m 10.42.4.16 18.188.214.137 <none> <none>
onap onap-aai-aai-graphadmin-create-db-schema-s42pq 0/1 Init:0/1 0 7m54s 10.42.5.20 18.220.70.253 <none> <none>
onap onap-aai-aai-graphadmin-create-db-schema-tsvcw 0/1 Init:Error 0 29m 10.42.4.5 18.188.214.137 <none> <none>
onap onap-aai-aai-modelloader-845fc684bd-r7mdw 2/2 Running 0 29m 10.42.4.6 18.188.214.137 <none> <none>
onap onap-aai-aai-resources-67f8dfcbdb-kz6cp 0/2 Init:0/1 2 29m 10.42.5.11 18.220.70.253 <none> <none>
onap onap-aai-aai-schema-service-6c56b45b7c-7zlfz 2/2 Running 0 29m 10.42.3.7 3.17.76.33 <none> <none>
onap onap-aai-aai-search-data-5d8d7759b8-flxwj 2/2 Running 0 29m 10.42.3.9 3.17.76.33 <none> <none>
onap onap-aai-aai-sparky-be-8444df749c-mzc2n 0/2 Init:0/1 0 29m 10.42.4.11 18.188.214.137 <none> <none>
onap onap-aai-aai-spike-54ff77787f-d678x 2/2 Running 0 29m 10.42.5.6 18.220.70.253 <none> <none>
onap onap-aai-aai-traversal-6ff868f477-lzv2f 0/2 Init:0/1 2 29m 10.42.3.8 3.17.76.33 <none> <none>
onap onap-aai-aai-traversal-update-query-data-9g2b8 0/1 Init:0/1 2 29m 10.42.5.12 18.220.70.253 <none> <none>
onap onap-dmaap-dbc-pg-0 1/1 Running 0 29m 10.42.5.9 18.220.70.253 <none> <none>
onap onap-dmaap-dbc-pg-1 1/1 Running 0 26m 10.42.3.14 3.17.76.33 <none> <none>
onap onap-dmaap-dbc-pgpool-8666b57857-97zjc 1/1 Running 0 29m 10.42.5.5 18.220.70.253 <none> <none>
onap onap-dmaap-dbc-pgpool-8666b57857-vr8gk 1/1 Running 0 29m 10.42.4.8 18.188.214.137 <none> <none>
onap onap-dmaap-dmaap-bc-745995bf74-m6hhq 0/1 Init:0/2 2 29m 10.42.4.12 18.188.214.137 <none> <none>
onap onap-dmaap-dmaap-bc-post-install-6ff4j 1/1 Running 0 29m 10.42.4.9 18.188.214.137 <none> <none>
onap onap-dmaap-dmaap-dr-db-0 1/1 Running 0 29m 10.42.4.10 18.188.214.137 <none> <none>
onap onap-dmaap-dmaap-dr-db-1 1/1 Running 1 24m 10.42.5.15 18.220.70.253 <none> <none>
onap onap-dmaap-dmaap-dr-node-0 2/2 Running 0 29m 10.42.3.11 3.17.76.33 <none> <none>
onap onap-dmaap-dmaap-dr-prov-fbf6c94f5-v9bmq 2/2 Running 0 29m 10.42.5.10 18.220.70.253 <none> <none>
onap onap-dmaap-message-router-0 1/1 Running 0 29m 10.42.4.14 18.188.214.137 <none> <none>
onap onap-dmaap-message-router-kafka-0 1/1 Running 1 29m 10.42.5.13 18.220.70.253 <none> <none>
onap onap-dmaap-message-router-kafka-1 1/1 Running 1 29m 10.42.3.13 3.17.76.33 <none> <none>
onap onap-dmaap-message-router-kafka-2 1/1 Running 0 29m 10.42.4.15 18.188.214.137 <none> <none>
onap onap-dmaap-message-router-mirrormaker-8587c4c9cf-lfnd8 0/1 CrashLoopBackOff 9 29m 10.42.4.7 18.188.214.137 <none> <none>
onap onap-dmaap-message-router-zookeeper-0 1/1 Running 0 29m 10.42.5.14 18.220.70.253 <none> <none>
onap onap-dmaap-message-router-zookeeper-1 1/1 Running 0 29m 10.42.4.13 18.188.214.137 <none> <none>
onap onap-dmaap-message-router-zookeeper-2 1/1 Running 0 29m 10.42.3.12 3.17.76.33 <none> <none>
onap onap-nfs-provisioner-nfs-provisioner-57c999dc57-mdcw5 1/1 Running 0 24m 10.42.3.15 3.17.76.33 <none> <none>
onap onap-robot-robot-677bdbb454-zj9jk 1/1 Running 0 24m 10.42.5.16 18.220.70.253 <none> <none>
onap onap-so-so-8569947cbd-jn5x4 0/1 Init:0/1 1 13m 10.42.4.19 18.188.214.137 <none> <none>
onap onap-so-so-bpmn-infra-78c8fd665d-b47qn 0/1 Init:0/1 1 13m 10.42.3.16 3.17.76.33 <none> <none>
onap onap-so-so-catalog-db-adapter-565f9767ff-lvbgx 0/1 Init:0/1 1 13m 10.42.3.17 3.17.76.33 <none> <none>
onap onap-so-so-mariadb-config-job-d9sdb 0/1 Init:0/2 0 3m37s 10.42.3.20 3.17.76.33 <none> <none>
onap onap-so-so-mariadb-config-job-rkqdl 0/1 Init:Error 0 13m 10.42.5.19 18.220.70.253 <none> <none>
onap onap-so-so-monitoring-69b9fdd94c-dks4v 1/1 Running 0 13m 10.42.4.18 18.188.214.137 <none> <none>
onap onap-so-so-openstack-adapter-5f9cf896d7-mgbdd 0/1 Init:0/1 1 13m 10.42.4.17 18.188.214.137 <none> <none>
onap onap-so-so-request-db-adapter-5c9bfd7b57-2krnp 0/1 Init:0/1 1 13m 10.42.3.18 3.17.76.33 <none> <none>
onap onap-so-so-sdc-controller-6fb5cf5775-bsxhm 0/1 Init:0/1 1 13m 10.42.4.20 18.188.214.137 <none> <none>
onap onap-so-so-sdnc-adapter-8555689c75-r6vkb 1/1 Running 0 13m 10.42.5.18 18.220.70.253 <none> <none>
onap onap-so-so-vfc-adapter-68fccc8bb8-c56t2 0/1 Init:0/1 1 13m 10.42.4.21 18.188.214.137 <none> <none>
onap onap-so-so-vnfm-adapter-65c4c5944b-72nlf 1/1 Running 0 13m 10.42.3.19 3.17.76.33 <none> <none>
# on worker nodes only
# nfs client
DI 20190507: ARM support using RKE 0.2.1 ARM friendly install
LOG-331 - Getting issue details... STATUS
a1.4xlarge $0.408
ami-0b9bd0b532ebcf4c9
Notes
Pre-RKE installation details in Cloud Native Deployment
, multiple selections available,
Related content
ONAP on VMware Integrated OpenStack 4.1 with Kubernetes
ONAP on VMware Integrated OpenStack 4.1 with Kubernetes
More like this
Cloud Native Deployment
Cloud Native Deployment
More like this
ONAP on Kubernetes
ONAP on Kubernetes
More like this
ONAP Deployment Installation
ONAP Deployment Installation
More like this
Integration with OOM and AAF
Integration with OOM and AAF
More like this
ONAP on Kubernetes on OpenStack
ONAP on Kubernetes on OpenStack
More like this