Setup ODL Clustering

Get akka.yaml from Gerrit Topic SDNC-214 (download akka.yaml and place it in <directory to oom repo>/kubernetes/sdnc templates on both Kubernetes master nodes.
TODO: configure_cluster.sh
Update the values.yaml file

The values.yaml file is what defines the GEO partitioning for the clusters, so for the primary cluster myODLCluster should have the IP address of the master node of the primary cluster and the peerODLCluster should have the IP address of the master node of the secondary (standby) cluster.

And the secondary cluster, it would be the reverse of the primary cluster.
Example of <oom repo>/kubernetes/sdnc/values.yaml

enableODLCluster: true enableGR: true
peerODLCluster: 10.147.114.5
myODLCluster: 10.147.114.140
Restart the SDN-C pods and log into the one of the SDN-C pods.
Once SDNC-C pod is up and running again, it might take up to 5 - 7 minutes, the akka.conf should be update with IP addresses describe in Step 3.

- Log in to one of the SDN-C pods and check the akka.conf from /opt/opendaylight/current/configuration/initial/:
Example akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = off
        canonical.hostname= "10.147.114.5"
        canonical.port = 30251
      }
      netty.tcp {
        bind-hostname= "10.36.0.3"
        bind-port = 2550
        hostname= "10.147.114.5"
        port = 30251
      }
      # when under load we might trip a false positive on the failure detector
      # transport-failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }
    cluster {
      # Remove ".tcp" when using artery.
      seed-nodes = ["akka.tcp://opendaylight-cluster-data@10.147.114.5:30251",
                                "akka.tcp://opendaylight-cluster-data@10.147.114.5:30252",
                                "akka.tcp://opendaylight-cluster-data@10.147.114.5:30253",
                                "akka.tcp://opendaylight-cluster-data@10.147.114.140:30251",
                                "akka.tcp://opendaylight-cluster-data@10.147.114.140:30252",
                                "akka.tcp://opendaylight-cluster-data@10.147.114.140:30253"]
      roles = ["member-1"]
    }
    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path.
      # The relative path is always relative to KARAF_HOME.
      # snapshot-store.local.dir = "target/snapshots"
      # journal.leveldb.dir = "target/journal"
      journal {
        leveldb {
          # Set native = off to use a Java-only implementation of leveldb.
          # Note that the Java-only version is not currently considered by Akka to be production quality.
          # native = off
        }
      }
    }
  }
}

To test ODL clustering:

Install Cluster Monitoring tool

Download the cluster monitor from github

github cluster monitory

git clone https://github.com/opendaylight/integration-test.git

Update ./integration-test/tools/clustering/cluster-monitor/cluster.json with the ip (from above) of your ODL cluster nodes

Example of cluster.json

{
    "cluster": {
        "controllers": [
            {"ip": "10.44.0.3", "port": "8181"},
            {"ip": "10.44.0.2", "port": "8181"},
            {"ip": "10.36.0.3", "port": "8181"}
        ],
        "user": "admin",
        "pass": "admin",
        "shards_to_exclude": []
    }
}

Install the attached monitor.py script in <integration-test repo>/tools/clustering/cluster-monitor and run it.

Run monitor.py

#Install python-pycurl at this point

sudo apt-get update
sudo apt-get install python-pycurl
cd ./integration-test/tools/clustering/cluster-monitor
python monitor.py

Something like this should appear on each master