You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
Version 1
Next »
Note, this page is based on the SDNC-C Clustering on Kubernetes page as created by Beili Zhou.
About the APPC-C Clustering
The following diagram illustrates the desired APPC cluster deployment (as described in
APPC-476
-
Getting issue details...
STATUS
) from POD and SERVICE view, which will have:
- 2 DB pods
- 3 appc (ODL) pods
- 1 admin (portal) pod
- 1 dgbuilder pod
APPC DB Clustering details
See details from APPC DB (MySQL) Clustered Deployment.
Deployment Setup Steps
You can use Rancher or kubeadm to deploy your Kubernetes cluster:
APPC Startup Order
Troubleshoot situation where APPC pods do not start up in order
Usually, this is caused by init-container malfuntion. You can troubleshoot it by the following steps:
Restart Dead Instance (start a new one in its place)
When a pod is dead, Kubernetes automatically starts a new one to replace the dead pod.
Examples:
Example of restarting dead instance by deleting pod
sdnc-0 pod runs fine at the beginning
manually delete the sdnc-0 pod
once the sdnc-0 pod is fully terminated, a new sdnc-0 pod will be initialized
The new sdnc-0 pod will be brought to running status to replace the initial sdnc-0
The new and initial sdnc-0 pod has the same name (although with different internal IP)
Description of Initial appc-0 | Description of restarted appc-0 |
---|
ubuntu@sdnc-k8s:~/oom/kubernetes/config$ kubectl describe pod appc-0 -n onap-appc Name: appc-0 Namespace: onap-appc Node: sdnc-k8s-2/10.147.90.6 Start Time: Thu, 23 Nov 2017 22:37:00 +0000 Labels: app=sdnc controller-revision-hash=sdnc-2443484326 Annotations: kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"StatefulSet","namespace":"onap-sdnc","name":"sdnc","uid":"a576a1a3-d09f-11e7-957f-0269cb13eff1","apiVersio... pod.alpha.kubernetes.io/init-container-statuses=[{"name":"sdnc-readiness","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2017-11-23T22:37:08Z","finishedAt":"2017-11-23T22:37:53Z... pod.alpha.kubernetes.io/init-containers=[{"name":"sdnc-readiness","image":"oomk8s/readiness-check:1.0.0","command":["/root/ready.py"],"args":["--container-name","sdnc-db-container"],"env":[{"name":"NA... pod.beta.kubernetes.io/init-container-statuses=[{"name":"sdnc-readiness","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2017-11-23T22:37:08Z","finishedAt":"2017-11-23T22:37:53Z"... pod.beta.kubernetes.io/init-containers=[{"name":"sdnc-readiness","image":"oomk8s/readiness-check:1.0.0","command":["/root/ready.py"],"args":["--container-name","sdnc-db-container"],"env":[{"name":"NAM... Status: Running IP: 10.42.107.61 Created By: StatefulSet/appc Controlled By: StatefulSet/appc ... | ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$ kubectl describe pod sdnc-0 -n onap-sdnc Name: sdnc-0 Namespace: onap-appc Node: sdnc-k8s-2/10.147.90.6 Start Time: Thu, 23 Nov 2017 22:54:10 +0000 Labels: app=sdnc controller-revision-hash=appc-2443484326 Annotations: kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"StatefulSet","namespace":"onap-sdnc","name":"sdnc","uid":"a576a1a3-d09f-11e7-957f-0269cb13eff1","apiVersio... pod.alpha.kubernetes.io/init-container-statuses=[{"name":"sdnc-readiness","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2017-11-23T22:54:15Z","finishedAt":"2017-11-23T22:54:15Z... pod.alpha.kubernetes.io/init-containers=[{"name":"sdnc-readiness","image":"oomk8s/readiness-check:1.0.0","command":["/root/ready.py"],"args":["--container-name","sdnc-db-container"],"env":[{"name":"NA... pod.beta.kubernetes.io/init-container-statuses=[{"name":"sdnc-readiness","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2017-11-23T22:54:15Z","finishedAt":"2017-11-23T22:54:15Z"... pod.beta.kubernetes.io/init-containers=[{"name":"sdnc-readiness","image":"oomk8s/readiness-check:1.0.0","command":["/root/ready.py"],"args":["--container-name","sdnc-db-container"],"env":[{"name":"NAM... Status: Running IP: 10.42.185.227 Created By: StatefulSet/appc Controlled By: StatefulSet/appc ... |
Example of restarting dead instance by stopping karaf inside SDNC pod
Pod sdnc-0 is in Running STATUS with 0 RESTARTS
Enter pod container and stop Karaf
Kubernetes discovers pod process stopped and restarts the container
sdnc-0 pod will be brought up with the restart container and running with 2 out of 2 container READY