Setup Steps
We are using MySQL Master-Master replication strategy to set up replication across two kubernetes clusters (aka sites). The setup is done through below manual steps.
1. Verify MySQL server-id is different in both sites.
| Code Block |
|---|
#Login to master db pod on site 1 and site 2
ubuntu@k8s-s1-master:~$ kubectl exec -it sdnc-dbhost-0 -n onap bash
Defaulting container name to sdnc-db-container.
Use 'kubectl describe pod/sdnc-dbhost-0' to see all of the containers in this pod.
root@sdnc-dbhost-0:/#
#Verify server-id on site 1
root@sdnc-dbhost-0:/# cd /etc/mysql/conf.d
root@sdnc-dbhost-0:/etc/mysql/conf.d# cat server-id.cnf
[mysqld]
server-id=100
#Verify server-id on site 2
root@sdnc-dbhost-0:/# cd /etc/mysql/conf.d
root@sdnc-dbhost-0:/etc/mysql/conf.d# cat server-id.cnf
[mysqld]
server-id=200 |
2. If MySQL site-id for site 1 and site 2, resulting
...
from step 1, are not different, proceed with the below step to make them unique.
| Info |
|---|
You can skip this step if the server-ids are verified to be unique in step #1. This step is only to be followed on site 2 (i.e. secondary site). |
| Code Block |
|---|
#Set server-id to a different number (e.g. 200) in the secondary site.
#Assuming the current value for server-id on site 2 is 100, we aim to change it to 200. Login to master DB pod on the Secondary site and change it.
ubuntu@k8s-s2-master:~$ kubectl exec -it sdnc-dbhost-0 -n onap bash
root@sdnc-dbhost-0:/# sed -i 's/100/200/g' /etc/mysql/conf.d/server-id.cnf
root@sdnc-dbhost-0:/etc/mysql/conf.d# service mysql restart
[info] Stopping MySQL Community Server 5.7.21.
...........command terminated with exit code 137
ubuntu@k8s-s1-master:~$
#verify
root@sdnc-dbhost-0:/# cd /etc/mysql/conf.d
root@sdnc-dbhost-0:/etc/mysql/conf.d# cat server-id.cnf
[mysqld]
server-id=200 |
3.
...
Verify NodePort Service Port Number on each site.
| Code Block |
|---|
...
#verify |
...
nodeport |
...
service |
...
exists |
...
ubuntu@k8s-s1-master:~$ kubectl |
...
get svc -n onap | grep mysql
mysql-nodeport NodePort 10.107.197.242 <none> 3306:30231/TCP,3307:30232/TCP 29s
ubuntu@k8s-s1-master:~$
#verify the endpoints are assigned for the service
#make a note of Node Port Number to use in step#5
ubuntu@k8s-s1-master:~$ kubectl describe svc mysql-nodeport -n onap
Name: mysql-nodeport
Namespace: onap
Labels: statefulset.kubernetes.io/pod-name=sdnc-dbhost-0
Annotations: <none>
Selector: statefulset.kubernetes.io/pod-name=sdnc-dbhost-0
Type: NodePort
IP: 10.107.197.242
Port: sdnc-dbhost-0-port-3306 3306/TCP
TargetPort: 3306/TCP
NodePort: sdnc-dbhost-0-port-3306 30231/TCP
Endpoints: 10.44.0.3:3306
Port: sdnc-dbhost-0-port-3307 3307/TCP
TargetPort: 3307/TCP
NodePort: sdnc-dbhost-0-port-3307 30232/TCP
Endpoints: 10.44.0.3:3307
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
ubuntu@k8s-s1-master:~$
|
4. Setup Master-Master Replication.
On site 2 (Secondary site):
| Code Block |
|---|
#Login to the master db pod ubuntu@k8s-s1-master:~$ kubectl exec -it sdnc-dbhost-0 -n onap bash Defaulting container name to sdnc-db-container. Use 'kubectl describe pod/sdnc-dbhost-0' to see all of the containers in this pod. #Login to mysql root@sdnc-dbhost-0:/etc/mysql/conf.d# mysql -u root -popenECOMP1.0 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 46083 Server version: 5.7.21-log MySQL Community Server (GPL) Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> #find out log bin position mysql> show master status; +--------------------------+----------+--------------+------------------+-------------------+ |
...
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +--------------------------+----------+--------------+------------------+------------------- |
...
+ | sdnc-dbhost-0-bin.000004 | 154 | | | | +--------------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec) mysql> stop slave; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> #Set: # - "master_host" to peer site's 'K8S master node' IP address # - "master_port" to port number of the NodePort service (from step#3) # - "master_user" to user name (here 'root') # - "master_password" to user's password (here 'root' user's default password) # - "master_log_file" to log_file name from "show master status" commands on the peer site (Primary) # - "master_log_pos" to log_position from "show master status" commands on the peer site (Primary) mysql> CHANGE MASTER TO master_host='10.147.112.136', master_port= |
...
30231, master_user='root', master_password='openECOMP1.0', master_log_file='sdnc-dbhost-0-bin.000002', master_log_pos=37433012; mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> |
On site 1 (Primary site):
| Code Block |
|---|
#Login to the master db pod
ubuntu@k8s-s1-master:~$ kubectl exec -it sdnc-dbhost-0 -n onap bash
Defaulting container name to sdnc-db-container.
Use 'kubectl describe pod/sdnc-dbhost-0' to see all of the containers in this pod.
#Login to mysql
root@sdnc-dbhost-0:/etc/mysql/conf.d# mysql -u root -popenECOMP1.0
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 46083 Server version: 5.7.21-log MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates.
Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
#find out log bin position
mysql> show master status;
+--------------------------+----------+--------------+------------------+-------------------+ |
...
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +--------------------------+----------+--------------+------------------+-------------------+ | sdnc-dbhost-0-bin.000003 | 37433012 | | | | +--------------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec) mysql> stop slave; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> #Set: # - "master_host" to peer site's 'K8S master node' IP address # - "master_port" to port number of the NodePort service (from step#3) # - "master_user" to user name (here 'root') # - "master_password" to user's password (here 'root' user's default password) # - "master_log_file" to log_file name from "show master status" commands on peer site (Seconday) # - "master_log_pos" to log_position from "show master status" commands on the peer site (Seconday) mysql> CHANGE MASTER TO master_host='10.147.113.36', master_port= |
...
30231, master_user='root', master_password='openECOMP1.0', master_log_file='sdnc-dbhost-0-bin.000004',master_log_pos=154; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> |
Verify replication status on both sites.
| Code Block |
|---|
mysql> show slave status\G; |
It should look like below:
