| Tip |
|---|
You can skip this step if your kubernetes deployment in is on a single VM or physical system. |
When setting up a kubernetes k8s cluster, the folder /dockerdata-nfs must be shared between all the kubernetes k8s nodes. This folder is used as a volume by the onap pods to share data and so there can only be one copy.
...
On this page we will attempt to do this by setting up a nfs server on the kubernetes k8s master node vm and then mount the exported directory on each of kubernetes’ the k8s’ nodes VMs.
These instruction where written using VMs create from a ubuntu-16.04-server-cloudimg-amd64-disk1 image.
| Table of Contents |
|---|
Create NFS Server and export /dockerdata-nfs on Kubernetes master node VM
The actual /dockerdata-nfs folder will live on the kubernetes k8s master node VM.
Create the directory as root on the k8s master node VM.
| Code Block | ||
|---|---|---|
| ||
sudo mkdir -p /export/dockerdata-nfs |
In our environment we provision an Openstack volume to allocate more space instead of of using the dockerdata-nfs folder as a Openstack volume and used that as the VM default backing store. This is optional and can be skipped. See Create an Openstack OpenStack VolumeThe following script can be used to setup for details
Setup the NFS server on the kubernetes k8s master node VM and export the dockerdata-nfs folder
| Code Block | ||||
|---|---|---|---|---|
| ||||
sudo apt update sudo apt install nfs-kernel-server mkdir ~/dockerdata-nfs sudo mkdir -p /export/dockerdata-nfs sudo chmod 777 /export sudo chmod 777 /export/dockerdata-nfs sudo vi /etc/exports # append "/export/dockerdata-nfs *(rw,no_root_squash,no_subtree_check)" sudo mount --bind /home/ubuntu/dockerdata-nfs /export/dockerdata-nfs sudo vi /etc/fstab # append "/home/ubuntu/dockerdata-nfs /export/dockerdata-nfs none bind 0 0" sudo service nfs-kernel-server restart |
Mount the Volume to Other VM Instancesthe dockerdata-nfs folder on each of the kubernetes node VMs.
| Warning | ||
|---|---|---|
| ||
More investigatio needed (as part of multi-nodes kubernetes cluser) as I'm having "Operation not permitted" error in sdnc-dbhost pod when deploying SDN-C cluster with the mounted /dockerdata-nfs from this instruction:
|
...
On
...
Install exportfs
if exportfs is not installed, install it with the following command:
sudo apt install nfs-kernel-server
...
| title | Example of installing exportfs |
|---|
ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$ sudo apt install nfs-kernel-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
keyutils libnfsidmap2 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libtirpc1 nfs-common python python-minimal python2.7 python2.7-minimal rpcbind
Suggested packages:
watchdog python-doc python-tk python2.7-doc binutils binfmt-support
The following NEW packages will be installed:
keyutils libnfsidmap2 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libtirpc1 nfs-common nfs-kernel-server python python-minimal python2.7 python2.7-minimal rpcbind
0 upgraded, 13 newly installed, 0 to remove and 27 not upgraded.
Need to get 4,383 kB of archives.
After this operation, 18.5 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
…
Replacing config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for systemd (229-4ubuntu19) ...
Processing triggers for ureadahead (0.100.0-19) ...
ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$
...
Modify /etc/exports file to export the /dockerdata-nfs mount point
...
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s:~$ more /etc/exports |
...
Export the /dockerdata-nfs mount point
sudo exportfs -rav
| Expand | ||
|---|---|---|
| ||
exporting sdnc-k8s:/dockerdata-nfs |
...
On the server of the other VM instance
...
Install nfs-common
...
sudo apt install nfs-common
...
Mount the volume as the /dockerdata-nfs directory
...
sudo mkdir -p /dockerdata-nfs
sudo mount <mount point server IP>:/dockerdata-nfs /dockerdata-nfs
...
Validate the mount
...
| title | Example of using df command to validate |
|---|
ubuntu@sdnc-k8s-2:~$ df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 8209144 0 8209144 0% /dev
tmpfs 1643244 9804 1633440 1% /run
/dev/vda1 20263528 9778736 10468408 49% /
tmpfs 8216216 2460 8213756 1% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 8216216 0 8216216 0% /sys/fs/cgroup
tmpfs 1643244 0 1643244 0% /run/user/1000
...
each of the k8s node vm' mount the /dockerdata-nfs folder that is being served from the k8s master VM.
| Code Block | ||||
|---|---|---|---|---|
| ||||
sudo apt install nfs-common
sudo mkdir /dockerdata-nfs
sudo chmod 777 /dockerdata-nfs
sudo mount -t nfs -o proto=tcp,port=2049 k8s-master:/export/dockerdata-nfs /dockerdata-nfs
sudo vi /etc/fstab
# append "<host|IP of k8s master node vm>:/export/dockerdata-nfs /dockerdata-nfs nfs auto 0 0" |
Tips
unmount
Use the lazy (-l) option to force unmount the mount point.
...