Distributed Cloud Manager (DCM)

Owned by Itohan Ukponmwan (Deactivated)
Last updated: Apr 01, 2020
14 min read

Background

The DCM is one of the components of ONAP4K8s. It will run as a microservice exposing Rest APIs, external components will use REST to communicate with the DCM while other microservices will use gRPC. The DCM will perform the following functions;

  1. User creation (currently one user per logical cloud)

  2. Namespace creation (currently one namespace per logical cloud)

  3. Generate intermediate CA key for each edge which is signed by a root or intermediate key

  4. Logical Cloud creation - Create Istio control planes for the logical clouds.

Different components (microservice) work together with the DCM to make the above possible, the components are;

  1. Main DCM Microservice (contains the service mesh Module(formally Logical Cloud Controller), User Module and Namespace Module, Quota Module(Limits resources available to each logical cloud))

  2. CA Key Distribution Controller ( Generate intermediate CA key for each edge which is signed by an root or intermediate key)

Design Overview

Fig 1: DCM Components



DCM Sequence

  1. Client creates logical cloud using logical cloud creation API and the following documents are created in the DCM collection

    • The core module parses the Json and creates a new document for the logical cloud in the mongodb DCM collection.

    • The core module also creates a cluster document in the DCM collection

    • The user module parses the Json and creates a new document for user 

    • The namespace module parses the Json and creates a new document for namespace

  2. Associates logical cloud with clusters (this API is called multiple times)

    • Updates the cluster document with the cluster name, loadbalancer ip every time its called

  3. Add quota for logical cloud

    • The quota module creates a quota document containing the quota details

  4. Apply API is called

    • Service mesh module gets CA bundle from CA controller via gRPC

    • Service mesh module gets names of logical cloud and creates a new namespace name using name of logical cloud name

    • Service mesh module creates helm template/istioctl manifest (WIP)

    • Service mesh module creates service mesh document in the DCM collection and stores the above (CA bundle contents, istio namespace, istioctl manifest) in the document

  5. DCM informs the resource synchronizer to start the logical cloud creation via gRPC and the resource synchronizer starts reading from the DB

  6. The DCM gets status from the resource synchronizer via gRPC

  7.  When the logical cloud creation is complete, the resource synchronizer will store the modified kubeconfig file for each cluster in the cluster document of the logical cloud



The details of the DCM Data Model can be found in DCM MongoDB Data Model

DCM Source Code Directory Structure

dcm
├── core
│      └── main.go
├── namespace-controller
│      └── namespace.go
├── quota-controller
│      └── quota.go
├── service-mesh-controller
│      └── service-mesh.go
└── user-controller
        └── user.go

GO API

func createNamespace(logicalCloudName Namespace string) error //Stores the namespace for the logical cloud in the database func createUser(user logicalCloudName string permissions []map[string]map[string][]string) error //Stores the user details for the logical cloud in the database func createKVpair(name description string userData map[string]string kvPair []map[string]string) error //Stores a new key value pair in the database func addCluster(cluster logicalCloudName string) error //Associates a new cluster with the logical cloud func addUserPermissions(user permissionName string apiGroups resources verbs []string) error func applyConfig(logicalCloudName string) error //Talks to the Resource Synchronizer to start the actual creation of all the resources for the logical cloud func getKVPair(name string) ([]map[string]string error) func getClusterConfig(cluster logicalCloudName string) ([]byte, error) //Returns Kubeconfig for the cluster in JSON format func getNamespace(logicalCloudName string) (string error) func getUser(logicalCloudName string) (string error) func getClusters(logicalCloudName string) ([]string error) func getUserPermissions(user string) (([]map[string]map[string][]string) error) //Sample output [{"permission-1": {"apiGroups": ["stable.example.com"], "resources" : ["secrets", "pods"], "verbs" : ["get", "watch", "list", "create"] }}, {"permission-2": {"apiGroups": [""], "resources" : ["configmaps"], "verbs" : ["*"] }}]



Service Mesh API

Service Mesh API
func create_mesh () { func get_lc_clusters() (map[string]string) //Returns a map containing a mapping of cluster names to load balancer ip address func create_mesh_namespace(logical-cloud-name, []clusters) func create_ca_secrets(logical-cloud-name){ func get_ca_certs(url) } func install_helm([] clusters) (or install istioctl) func create_helm_chart() (or istioctl manifest) }



REST API

API to Create Logical Cloud Name and Associate some components

Create Logical cloud  name for new logical cloud, add user name, namespace and user permissions

Logical Cloud Creation API
URL: /v2/projects/<project-name>/logical-clouds POST BODY: { "metadata" : { "name": "lc-1", //unique name for the record "description": "logical cloud for walmart finance department", //description for the logical cloud "userData1":"<user data>", "userData2":"<user data>" }, "spec" : { "namespace" : "ns-1", // one namespace per logical cloud "user" : { "user-name" : "user-1", //name of user for this cloud (username and logical cloud name would be used as subject for the user key) "type" : "certificate", //type of authentication credentials used by user (certificate, Token, UNPW) "user-permissions" : [ { "permission-name" : "permission-1", "apiGroups" : ["stable.example.com"], "resources" : ["secrets", "pods"], "verbs" : ["get", "watch", "list", "create"] }, { "permission-name" : "permission-2", "apiGroups" : [""], "resources" : ["configmaps"], "verbs" : ["*"] } ] } } } Return Status: 201 Return Body: { "name" : "logical-cloud-1", "logical-cloud-name" : "logical-cloud-1", "namespace" : "ns-1", "user-name" : "user-1" }

PUT (Change logical cloud contents)

PUT Logical Cloud



GET Logical Cloud

GET Logical Cloud

DELETE Logical Cloud

DELETE Logical Cloud

Logical Cloud Cluster API

 POST (Associate cluster with logical cloud )

Associate logical cloud with cluster



PUT

Update cluster in logical cloud

GET

Get Clusters Associated with logical clouds

DELETE (Delete cluster from Logical cloud)

Delete Cluster from logical cloud

Logical Cloud User Permissions API

 Add user permissions

Add User permissions



GET User Permissions

Get User permissions



PUT (Update User permissions)

Update User permissions

 Delete User Permissions

Delete User permissions



Logical Cloud Quota API

Create logical cloud Quota ( quota will be applied to each cluster in the logical cloud)

This allows resources to be tuned for the logical cloud

Create Logical cloud quota



 GET logical cloud Quota

Get Logical cloud quota



Update Logical Cloud Quota

Update Logical cloud quota



Delete Logical Cloud Quota

Update Logical cloud quota



Logical Cloud Key Value API

16. Add Key Value pair to logical cloud database

Add KV pair



17. PUT (Update kv pair)

Update KV pair



18. GET KV pair

Get KV pair



19. DELETE KV pair

Delete KV pair

Kubeconfig API (WORK IN PROGRESS)

Important points to Note

  • cluster CA and cluster CRT will be gotten when a cluster is registered and this will be used to create the user crt after the user csr and user key are created

Kubeconfig will be put in the mongoDB

2.  Get Logical Cloud kubeconfig

Get Logical cloud kubeconfig





Apply API

When the API is called, the resource synchronizer is called  and the resource creation in the cluster begins

Apply all the created configuration, this creates the K8s resources

Apply configuration



Status API

GET (Check status of operation)

Get Operation status