DCM Operation

Owned by Itohan Ukponmwan (Deactivated)
Last updated: Aug 07, 2020 by Igor D.C.
3 min read

The DCM is one of the components of ONAP4K8s, it is responsible for the creation of logical clouds see Distributed Cloud Manager for more detail. This page will describe how the DCM will operate in detail  as well as the components that will make its operation possible.

Logical Cloud Creation Sequence

In order to create a logical cloud, a user call various API as described in Distributed Cloud Manager. The API and sequence are listed below

  1. Logical cloud creation API

  2. Logical cloud cluster API - the user calls this API every time a cluster is added to the logical cloud. (if a logical cloud spans 3 clusters, this API will be called 3 times)

  3. Logical cloud quota API

  4. Apply API

Although there are other APIs for the DCM, the above are absolutely required to create a logical cloud. When the user calls each API (except the Apply API), the API stores the contents in MongoDB, finally the user calls the Apply API. On calling the apply API, the DCM's apply.go code (see https://github.com/onap/multicloud-k8s/blob/master/src/dcm/pkg/module/apply.go), reads from MongoDB, creates the YAML resource string, saves the resources in etcd using the app context library and pings the resource synchronizer over gRPC to trigger the creation of the K8s resources from etcd.

Logical Cloud Creation command Sequence

The steps below describe the kubernetes steps that are done by the rsource syncronizer to create a logical cloud. It also describes the steps that have been implemented in the apply.go code above and the steps that are still being figured out. To create resources in a cluster, the DCM creates a the string representation of the YAML for the resource and saves it in etcd, the resource synchronizer then reads that and creates the resource in the clusters. Currently, the resource synchronizer can only run the equivalent of "kubectl create -f <yaml-file>".

  1. Create Namespace in all clusters

  2.  

    1. kubectl create -f <namespace-yaml>

    2. Resource synchronizer logs into all the clusters specified by the logical cloud and runs the above command



      2. Create User in all clusters (assuming user name is employee)

  1.  

    1. DCM creates a private key for user (e.g user employee) and saved in MongoDB

    2. DCM creates a certificate sign request (CSR) and saves in etcd

    3. The following commands need to be run in the cluster to certificate

      1. kubectl certificate approve lc1-user-cert (TODO)

      2. kubectl get csr lc1-user-cert -o jsonpath='{.status.certificate}' | base64 --decode > employee.crt (TODO)

      3. Generate kubeconfig in DCM:

        1. attach cluster CA cert

        2. attach cluster address

        3. attach user private key

        4. attach user signed cert by CA (TODO - see c.ii.)

    4. DCM Creates Roles YAML using user permissions and saves it in etcd

    5. DCM Creates Role Binding YAML and saves in etcd

    6. Resource Synchronizer creates all the resources

      1. kubectl create -f <csr yaml>

      2. kubectl create -f  <role yaml>

      3. kubectl create -f  <yaml>

Saved in ETCD:

  1. employee.key (TODO)

  2. employee csr yaml

  3. role.yaml

  4. rolebinding.yaml

Opens

Currently the following need to be done to create a user however, we are not sure how to do it yet

  • How to save the user key to be used in the cluster (investigating)

  • How to run the following commands (investigating):

    • kubectl certificate approve lc1-user-cert

    • kubectl get csr lc1-user-cert -o jsonpath='{.status.certificate}' | base64 --decode > employee.crt

    • kubectl config set-credentials employee --client-certificate=/home/employee/.certs/employee.crt  --client-key=/home/employee/.certs/employee.key

    • kubectl config set-context employee-context --cluster=cluster-name --namespace=lc1 --user=employee

3.  Create Quota in all clusters

  1.  

    1. DCM creates quota file for lc1 namespace and stores it in etcd

    2. kubectl create -f <quota-yaml>

    3. Resource synchronizer should log into all the clusters belonging to the logical cloud and run the above command

Saved in ETCD: resource quota yaml

4. Create Service mesh in all clusters   (see istio replicated control plane for ref)   (Not Done)

  1.  

    1. DCM creates a namespace name for the istio namespace using the logical cloud name  e.g istio-lci

    2. DCM queries CA key controller for certificate bundle  and saves it in etcd to be used to create secret

    3. DCM creates charts for to be used to create the istio control plane and saves it in etcd

    4. Resource Synchronizer creates the istio namespace

    5.  

      • kubectl create namespace istio-lci

    6. Resource synchronizer creates the istio secret using the certificate bundle stored in etcd

    7.  

      • kubectl create secret generic cacerts -n istio-system  --from-file=samples/certs/ca-cert.pem --from-file=samples/certs/ca-key.pem  --from-file=samples/certs/root-cert.pem --from-file=samples/certs/cert-chain.pem

    8. Resource synchronizer applies the istio multicluster gateway charts

    9. Resource synchronizer modifies coredns  in each cluster in the logical cloud to add the stub domain for the new logical cloud (Sequence TBD) adds the following to the coredns configmap for each cluster in the logical cloud

  lc1:53 { errors cache 30 forward . $(kubectl get svc -n istio-system istiocoredns -o jsonpath={.spec.clusterIP}):53 }



Saved in ETCD: istio namespace, istio chart, istio secret

Opens

  • The DCM need to install istio on multiple namespaces in the same k8s cluster, currently there are bugs are seen when istio is installed in a namespace aside the default istio-system

  • How to install istio through resource sync without using istioctl command

  • How to update coredns configmap with the stub domain in each cluster every time a new logical cloud is created