...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
| Tip |
|---|
You can skip this step if your kubernetes Kubernetes cluster deployment in is on a single VM or physical system. |
When setting up a kubernetes Kubernetes cluster, the folder /dockerdata-nfs must be shared between all of the kubernetes Kubernetes worker nodes. This This folder is used as a volume by the onap ONAP pods to share data and , so there can only be one copy.
On this page we will attempt to do this by setting up nfs an NFS server on the kubernetes master node Kubernetes Master and then mount the exported shared directory on each of kubernetes’ all Kubernetes worker nodes.
These instruction where written using VMs create created from a ubuntu-16.04-server-cloudimg-amd64-disk1 image.
Any user can be used to run the steps in this page, as all the commands are "sudo".
| Table of Contents |
|---|
...
On the NFS Server
...
VM (Kubernetes Master Node)
The actual /dockerdata-nfs folder will live on the kubernetes master VM.
In our environment we provision an Openstack volume to allocate more space instead of of using the VM default backing store.
This is optional and can be skipped. Create an Openstack Volume
The following script can be used to setup the NFS server on the kubernetes master VMKubernetes Master node which will also be running the NFS server to export this folder.
Set up the /dockerdata-nfs Folder
Choose one of the following to create the /dockerdata-nfs folder on this VM:
| Use local directory | Run the following command as root:
|
|---|
...
|
...
|
...
|
...
|
...
|
...
|
...
| |
| Use separate volume | Following instruction from Create an OpenStack Volume to: (where the VM Instance is the one that you have chosen) |
|---|
Setup the NFS Server and Export /dockerdata-nfs Folder
Execute the following commands as ubuntu user.
| Code Block | ||||
|---|---|---|---|---|
| ||||
sudo apt update sudo apt install nfs-kernel-server sudo vi /etc/exports # append "/exportthe following /dockerdata-nfs *(rw,no_root_squash,no_subtree_check)" sudo mount --bind /home/ubuntu/dockerdata-nfs /export/dockerdata-nfs sudo vi /etc/fstab # append "/home/ubuntu/dockerdata-nfs /export/dockerdata-nfs none bind 0 0" sudo service nfs-kernel-server restart |
Mount the Volume to Other VM Instances
Suggested packages:
watchdog python-doc python-tk python2.7-doc binutils binfmt-support
The following NEW packages will be installed:
keyutils libnfsidmap2 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libtirpc1 nfs-common nfs-kernel-server python python-minimal python2.7 python2.7-minimal rpcbind
0 upgraded, 13 newly installed, 0 to remove and 27 not upgraded.
Need to get 4,383 kB of archives.
After this operation, 18.5 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
…
Replacing config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for systemd (229-4ubuntu19) ...
Processing triggers for ureadahead (0.100.0-19) ...
ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$
...
Modify /etc/exports file to export the /dockerdata-nfs mount point
...
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s:~$ more /etc/exports |
...
Export the /dockerdata-nfs mount point
sudo exportfs -rav
| Expand | ||
|---|---|---|
| ||
exporting sdnc-k8s:/dockerdata-nfs |
...
On the server of the other VM instance
...
Install nfs-common
...
sudo apt install nfs-common
...
Mount the volume as the /dockerdata-nfs directory
...
sudo mkdir -p /dockerdata-nfs
sudo mount <mount point server IP>:/dockerdata-nfs /dockerdata-nfs
...
Validate the mount
...
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s-2:~$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 8209144 0 8209144 0% /dev tmpfs 1643244 9804 1633440 1% /run /dev/vda1 20263528 9778736 10468408 49% / tmpfs 8216216 2460 8213756 1% /dev/shm tmpfs 5120 0 5120 0% /run/lock tmpfs 8216216 0 8216216 0% /sys/fs/cgroup tmpfs 1643244 0 1643244 0% /run/user/1000 10.147.132.10:/dockerdata-nfs 103081984 61440 97761280 1% /dockerdata-nfs |
Tips
...
| Warningexpand | ||
|---|---|---|
| ||
More investigatio needed (as part of multi-nodes kubernetes cluser) as I'm having "Operation not permitted" error in sdnc-dbhost pod when deploying SDN-C cluster with the mounted /dockerdata-nfs from this instruction:
|
...
On the server of the attached VM instance
...
Install exportfs
if exportfs is not installed, install it with the following command:
sudo apt install nfs-kernel-server
...
| title | Example of installing exportfs |
|---|
ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$ sudo apt install nfs-kernel-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
keyutils libnfsidmap2 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libtirpc1 nfs-common python python-minimal python2.7 python2.7-minimal rpcbind
| |
$ ps -ef|grep nfs |
On the other VMs (Kubernetes Worker Nodes)
Mount the /dockerdata-nfs Folder
On each of the Kubernetes worker nodes, mount the /dockerdata-nfs folder. Run the followings as ubuntu user.
| Code Block | ||||
|---|---|---|---|---|
| ||||
sudo apt update
sudo apt install nfs-common -y
sudo mkdir /dockerdata-nfs
sudo chmod 777 /dockerdata-nfs
# Option 1:
sudo mount -t nfs -o proto=tcp,port=2049 <hostname or IP address of NFS server>:/dockerdata-nfs /dockerdata-nfs
sudo vi /etc/fstab
# append the following
<hostname or IP address of NFS server>:/dockerdata-nfs /dockerdata-nfs nfs auto 0 0
# Option 2:
sudo vi /etc/fstab
# append the following line.
<hostname or IP address of NFS server>:/dockerdata-nfs /dockerdata-nfs nfs auto 0 0
# run the following line
sudo mount -a |
Verify it :
Tocuh a file inside /dockerdata-nfs directory on the Kubernetes Master and check to see if the same file is found under /dockerdata-nfs on all Kubernetes worker nodes.
Unmount the share directory
Use the lazy (-l) option on Kubernetes worker nodes to force unmount the mount point.
...