...
Once you have your Rancher and Kubernetes environment running on your Openstack environment, you will need to add raw volumes to each Kubernetes node from Openstack console. In this document, we assume that the volumes were mounted on /dev/vdb.
(Openstack will tell you which device path your volume is mounted when you attach your volume to your instance).
<<TODO: screenshots or video on adding storage>>
...
GlusterFS setup
Once you have your infrastructure running and your RAW volumes mounted to your kubernetes nodes, deploy your Heketi / GlusterFS infrastructure. You can use the scripts included in OOM to automate this in your lab. (Not recommended for production install).
Grab the OOM artifacts from Gerrit (we did this on the Rancher master node in our deployment):
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
git clone http://gerrit.onap.org/r/oom
cd oom/kubernetes
cd onap/resources/scripts
ls -l |
total 16
-rw-r--r-- 1 root root 1612 Sep 11 16:18 cleanup_gluster.bash
-rw-r--r-- 1 root root 9956 Sep 11 18:51 deploy_glusterfs.bash
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
bash deploy_glusterfs.bash |
deploy_glusterfs.bash: usage: deploy_gluster.bash <dev path> namespace
e.g. deploy_glusterfs.bash /dev/vdb onap
This script deploys a GlusterFS kubernetes on OpenStack to be used as Persistent Volumes| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
bash deploy_glusterfs.bash /dev/vdb onap |
The script will prompt you to hit "Enter" every once in a while to let you confirm there are no errors. There is minimal error checking in this script, so pay attention, especially when you are re-running the script after previously deploying GlusterFS.
The script will start off to direct you to run some commands manually on the other Kubernetes worker nodes (Openstack VMs):
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
iptables -N HEKETI
iptables -A HEKETI -p tcp -m state --state NEW -m tcp --dport 24007 -j ACCEPT
iptables -A HEKETI -p tcp -m state --state NEW -m tcp --dport 24008 -j ACCEPT
iptables -A HEKETI -p tcp -m state --state NEW -m tcp --dport 2222 -j ACCEPT
iptables -A HEKETI -p tcp -m state --state NEW -m multiport --dports 49152:49251 -j ACCEPT
service iptables save
modprobe dm_snapshot
modprobe dm_mirror
modprobe dm_thin_pool
lsmod |egrep 'dm_snapshot|dm_mirror|dm_thin_pool'
|
<<TODO: video on running script>>
Once the script is finished, check to make sure you have a valid StorageClass defined, and GlusterFS/Heketi Pods running on each Kubernetes node:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
kubectl get pods --namespace onap
kubectl describe sc --namespace onap
kubectl get service --namespace onap |
e.g
NAME READY STATUS RESTARTS AGE
glusterfs-cxqc2 1/1 Running 0 4d
glusterfs-djq4x 1/1 Running 0 4d
glusterfs-t7cj5 1/1 Running 0 4d
glusterfs-z4vk6 1/1 Running 0 4d
heketi-5876bd4875-hzw2d 1/1 Running 0 4d
Name: glusterfs-sc
IsDefaultClass: No
Annotations: <none>
Provisioner: kubernetes.io/glusterfs
Parameters: resturl=http://10.43.185.167:8080,restuser=,restuserkey=
Events: <none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
heketi ClusterIP 10.43.185.167 <none> 8080/TCP 4d
heketi-storage-endpoints ClusterIP 10.43.227.203 <none> 1/TCP 4d