...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
| Tip |
|---|
You can skip this step if |
...
your Kubernetes cluster deployment is on a single VM. |
When setting up a Kubernetes cluster, the folder /dockerdata-nfs directory among must be shared between all of the Kubernetes worker nodes.
| Table of Contents |
|---|
Create Volume
Here is an example of creating an OpenStack volume.
From an OpenStack UI, go to the Volumes tab and click the +Create Volume button to bring up the Create Volume page as shown below. Fill in the fields so that they match the screenshot below (choose your own Volume Name 
):
The volume will be created as shown below:
Attach the Volume to the VM Instance
Underneath the Actions column, click on the down arrow beside Edit Volume to get a drop-down menu; select Manage Attachments:
The Manage Volume Attachments window will pop up:
Underneath Attach to Instance, click the down arrow beside Select an instance to get a drop-down menu; select the desired VM instance, then click on the Attach Volume button.
The volume will be attached to the desired VM instance with the Status changed to In-use, and the Attached To field updated.
The following is an example:
Mount the Volume in the Attached VM Instance
On the attached VM instance server, follow the steps below to mount the volume as the /dockdata-nfs directory:
(See more details at RedHat Enterprise Linux OpenStack Platform Getting Started Guide)
...
Find the volume id
...
ls /dev/disk/by-id
| Expand | ||
|---|---|---|
| ||
virtio-274e55f0-314f-4197-a |
...
Convert to mkfs format
...
sudo mkfs.ext4 /dev/disk/by-id/<volumeId>
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s:~/oom/kubernetes/config$ sudo mkfs.ext4 /dev/disk/by-id/virtio-274e55f0-314f-4197-a mke2fs 1.42.13 (17-May-2015) Creating filesystem with 26214400 4k blocks and 6553600 inodes Filesystem UUID: 8d9e10a2-c28b-4237-b1d5-69bf5c6bec6f Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done |
...
Mount the volume as the /dockerdata-nfs directory
...
sudo mkdir -p /dockerdata-nfs
sudo mount /dev/disk/by-id/virtio-274e55f0-314f-4197-a /dockerdata-nfs
...
Validate the mount
...
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s:~/oom/kubernetes/config$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 8209144 0 8209144 0% /dev tmpfs 1643244 10832 1632412 1% /run /dev/vda1 20263528 5920744 14326400 30% / tmpfs 8216216 2244 8213972 1% /dev/shm tmpfs 5120 0 5120 0% /run/lock tmpfs 8216216 0 8216216 0% /sys/fs/cgroup tmpfs 1643244 0 1643244 0% /run/user/1000 /dev/vdb 103081248 61044 97760940 1% /dockerdata-nfs |
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s:~/oom/kubernetes/config$ mount|grep dockerdata /dev/vdb on /dockerdata-nfs type ext4 (rw,relatime,data=ordered) |
Mount the Volume to Other VM Instances
| Warning | ||
|---|---|---|
| ||
More investigatio needed (as part of multi-nodes kubernetes cluser) as I'm having "Operation not permitted" error in sdnc-dbhost pod when deploying SDN-C cluster with the mounted /dockerdata-nfs from this instruction:
|
...
On the server of the attached VM instance
...
Install exportfs
if exportfs is not installed, install it with the following command:
sudo apt install nfs-kernel-server
...
| title | Example of installing exportfs |
|---|
ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$ sudo apt install nfs-kernel-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
keyutils libnfsidmap2 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libtirpc1 nfs-common python python-minimal python2.7 python2.7-minimal rpcbind
Suggested packages:
watchdog python-doc python-tk python2.7-doc binutils binfmt-support
The following NEW packages will be installed:
keyutils libnfsidmap2 libpython-stdlib libpython2.7-minimal libpython2.7-stdlib libtirpc1 nfs-common nfs-kernel-server python python-minimal python2.7 python2.7-minimal rpcbind
0 upgraded, 13 newly installed, 0 to remove and 27 not upgraded.
Need to get 4,383 kB of archives.
After this operation, 18.5 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
…
Replacing config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for systemd (229-4ubuntu19) ...
Processing triggers for ureadahead (0.100.0-19) ...
ubuntu@sdnc-k8s:~/oom/kubernetes/oneclick$
...
Modify /etc/exports file to export the /dockerdata-nfs mount point
...
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s:~$ more /etc/exports |
...
Export the /dockerdata-nfs mount point
sudo exportfs -rav
| Expand | ||
|---|---|---|
| ||
exporting sdnc-k8s:/dockerdata-nfs |
...
On the server of the other VM instance
...
Install nfs-common
...
sudo apt install nfs-common
...
Mount the volume as the /dockerdata-nfs directory
...
sudo mkdir -p /dockerdata-nfs
sudo mount <mount point server IP>:/dockerdata-nfs /dockerdata-nfs
...
Validate the mount
...
| Expand | ||
|---|---|---|
| ||
ubuntu@sdnc-k8s-2:~$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 8209144 0 8209144 0% /dev tmpfs 1643244 9804 1633440 1% /run /dev/vda1 20263528 9778736 10468408 49% / tmpfs 8216216 2460 8213756 1% /dev/shm tmpfs 5120 0 5120 0% /run/lock tmpfs 8216216 0 8216216 0% /sys/fs/cgroup tmpfs 1643244 0 1643244 0% /run/user/1000 10.147.132.10:/dockerdata-nfs 103081984 61440 97761280 1% /dockerdata-nfs |
Tips
unmount
Use the lazy (-l) option This folder is used as a volume by the ONAP pods to share data, so there can only be one copy.
On this page we will attempt to do this by setting up an NFS server on the Kubernetes Master and then mount the shared directory on all Kubernetes worker nodes.
These instruction where written using VMs created from a ubuntu-16.04-server-cloudimg-amd64-disk1 image.
Any user can be used to run the steps in this page, as all the commands are "sudo".
| Table of Contents |
|---|
On the NFS Server VM (Kubernetes Master Node)
The actual /dockerdata-nfs folder will live on Kubernetes Master node which will also be running the NFS server to export this folder.
Set up the /dockerdata-nfs Folder
Choose one of the following to create the /dockerdata-nfs folder on this VM:
| Use local directory | Run the following command as root:
| |||||
|---|---|---|---|---|---|---|
| Use separate volume | Following instruction from Create an OpenStack Volume to: (where the VM Instance is the one that you have chosen) |
Setup the NFS Server and Export /dockerdata-nfs Folder
Execute the following commands as ubuntu user.
| Code Block | ||||
|---|---|---|---|---|
| ||||
sudo apt update
sudo apt install nfs-kernel-server
sudo vi /etc/exports
# append the following
/dockerdata-nfs *(rw,no_root_squash,no_subtree_check)
sudo service nfs-kernel-server restart |
| Expand | ||
|---|---|---|
| ||
$ ps -ef|grep nfs |
On the other VMs (Kubernetes Worker Nodes)
Mount the /dockerdata-nfs Folder
On each of the Kubernetes worker nodes, mount the /dockerdata-nfs folder. Run the followings as ubuntu user.
| Code Block | ||||
|---|---|---|---|---|
| ||||
sudo apt update
sudo apt install nfs-common -y
sudo mkdir /dockerdata-nfs
sudo chmod 777 /dockerdata-nfs
# Option 1:
sudo mount -t nfs -o proto=tcp,port=2049 <hostname or IP address of NFS server>:/dockerdata-nfs /dockerdata-nfs
sudo vi /etc/fstab
# append the following
<hostname or IP address of NFS server>:/dockerdata-nfs /dockerdata-nfs nfs auto 0 0
# Option 2:
sudo vi /etc/fstab
# append the following line.
<hostname or IP address of NFS server>:/dockerdata-nfs /dockerdata-nfs nfs auto 0 0
# run the following line
sudo mount -a |
Verify it :
Tocuh a file inside /dockerdata-nfs directory on the Kubernetes Master and check to see if the same file is found under /dockerdata-nfs on all Kubernetes worker nodes.
Unmount the share directory
Use the lazy (-l) option on Kubernetes worker nodes to force unmount the mount point.
...