DCAE R6 DL-Handler MicroService
- Vijay Kumar
- Ekko Chang
- Kate Hsuan
- Guobiao Mo
- 1 Overview
- 2 Architecture Diagram
- 3 Artefacts
- 4 Deployment Prerequisite/dependencies
- 5 Deployment Steps
- 5.1 Log-in to the DCAE bootstrap POD's main container
- 5.2 Validate blueprint
- 5.2.1 Validate Blueprint
- 5.3 Upload the blueprint to cloudify manager.
- 5.4 Verify Blueprint Upload
- 5.4.1 Verify Upload
- 5.5 Verify Plugin versions in target Cloudify instance match to blueprint imports
- 5.5.1 Verify Plugin version
- 5.6 Create Deployment
- 5.6.1 Input file
- 5.7 Launch Service
- 5.8 To Un-deploy
- 5.8.1 Uninstall component
- 5.9 Delete blueprint
- 5.9.1 Delete blueprint
- 5.10 Deploy external database
Overview
DataLake is a software component of ONAP that can systematically persist the events in DMaaP into supported Big Data storage systems. It has a Admin UI, where a system administrator configures which Topics to be monitored, and to which data storage to store the data. It is also used to manage the settings of the storage and associated data analytics tool. The second part is the Feeder, which does the data transfer work and is horizontal scalable. In the next release, R7, we will add the third component, Data Exposure Service (EDS), which will expose the data in the data storage via REST API for other ONAP components and external systems to consume. Each data exposure only requires simple configurations.
Architecture Diagram
Data Exposure Service will be available in R7.
Artefacts
Βlueprint (deployment artifact) : k8s-datalake-feeder.yaml, k8s-datalake-admin-ui.yaml
Docker image:
feeder, onap/org.onap.dcaegen2.services.datalakefeeder:1.0.0
admin UI, onap/org.onap.dcaegen2.services.datalakeadminui:1.0.1
Deployment Prerequisite/dependencies
Since datalake can log the message from the DMaap to several different external databases, such as Elasticsearch, Couch Base, MongoDB, Relational databases...etc. Once Datalake is successfully deployed, you can start to configure the external databases through our admin UI. The following sections will guide you to deploy datalake microservice, including cloudify blueprint upload, deployment, and un-deployment.
Deployment Steps
DL-handler consists of two pods- the feeder and admin UI. It can be deployed by using cloudify blueprint. Datalake can be easily deployed through DCAE cloudify manager. The following steps guides you launch Datalake though cloudify manager.
Log-in to the DCAE bootstrap POD's main container
First, we should find the bootstrap pod name through the following command and make sure that DCAE coudify manager is properly deployed.
Login to the DCAE bootstrap pod through the following command.
Login to the bootstrap pod
kubectl exec -it <DCAE bootstrap pod> /bin/bash -n onapValidate blueprint
Validate Blueprint
cfy blueprints validate /blueprints/k8s-dl-handler.yamlUpload the blueprint to cloudify manager.
Upload blueprint to cloudify manager
cfy blueprint upload -b datalake-feeder /bluerints/k8s-datalake-feeder.yaml
cfy blueprint upload -b datalake-admin-ui /blueprints/k8s-datalake-admin-ui.yamlVerify Blueprint Upload
Verify Upload
You can see the following returned message to show the blueprints have been correctly uploaded.
Verify Plugin versions in target Cloudify instance match to blueprint imports
If the version of the plugin used is different, update the blueprint import to match.
Verify Plugin version
Create Deployment
Here we are going to create deployments for both feeder and admin UI.
Input file
Launch Service
Next, we are going to launch the datalake.
Upload and deploy blueprint
To Un-deploy
Uninstall running component and delete deployment