| Table of Contents |
|---|
Overview
...
Data Exposure Service will be available in R7.
...
Artefacts
Βlueprint (deployment artifact) :
Input file (deployment input) :
Docker image : nexus3.onap.org:10001/onap/<>k8s-datalake-feeder.yaml, k8s-datalake-admin-ui.yaml
Docker image:
feeder, onap/org.onap.dcaegen2.services.datalakefeeder:1.0.0
admin UI, onap/org.onap.dcaegen2.services.datalakeadminui:1.0.1
Deployment Prerequisite/dependencies
In R6, the following storage are supported:
MongoDB
Couchbase
Elasticsearch and Kibana
HDFS
To use DataLake, you need to have at least one of these systems ready. Once DataLake is Since datalake can log the message from the DMaap to several different external databases, such as Elasticsearch, Couch Base, MongoDB, Relational databases...etc. Once Datalake is successfully deployed, you can start to configure Topic and storage in the DataLake Admin UI.the external databases through our admin UI. The following sections will guide you to deploy datalake microservice, including cloudify blueprint upload, deployment, and un-deployment.
Deployment Steps
DL-handler consists of two pods- the feeder and admin UI. It can be deployed by using cloudify Helm plug-in. Since the tiller IP address and port should be exposed and accessed by Helm, some additional configuration steps are needed. the detailed pre-configuration steps can be found in the Cloudify Helm Plugin wiki page.
Next, the cloudify input file of datalake should be placed into bootstrap pod. The input file can be found in ONAP git repository. Once you clone the repository, the blueprint file could be copied to the DCAE bootstrap pod through the command line.
...
Validate blueprint
blueprint. Datalake can be easily deployed through DCAE cloudify manager. The following steps guides you launch Datalake though cloudify manager.
Log-in to the DCAE bootstrap POD's main container
First, we should find the bootstrap pod name through the following command and make sure that DCAE coudify manager is properly deployed.
Login to the DCAE bootstrap pod through the following command.
| Code Block | |||||
|---|---|---|---|---|---|
|
...
|
...
cfy blueprints validate /blueprints/k8s-dl-handler.yamlkubectl exec -it <DCAE bootstrap pod> /bin/bash -n onap |
Validate blueprint
| Code Block | |||||
|---|---|---|---|---|---|
|
...
| |||
cfy blueprints |
...
If the version of plugin used are different, update the blueprint import to match.
Deploy Service
validate /blueprints/k8s-dl-handler.yaml |
Upload the blueprint to cloudify manager.
| Code Block | ||||||
|---|---|---|---|---|---|---|
|
...
| |||
cfy blueprint |
...
upload -b |
...
datalake- |
...
feeder |
...
/bluerints/k8s-datalake-feeder.yaml cfy blueprint upload -b datalake-admin-ui /blueprints/k8s-datalake- |
...
admin- |
...
ui.yaml |
To un-deploy
Uninstall running component and delete deployment
Verify Blueprint Upload
| Code Block | |||||
|---|---|---|---|---|---|
|
...
| |||
cfy |
...
Delete blueprint
blueprint list |
You can see the following returned message to show the blueprints have been correctly uploaded.
Verify Plugin versions in target Cloudify instance match to blueprint imports
If the version of the plugin used is different, update the blueprint import to match.
| Code Block | |||||
|---|---|---|---|---|---|
|
...
| |||
cfy |
...
Initial Validation
...
plugins list |
Create Deployment
Here we are going to create deployments for both feeder and admin UI.
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
root@k8s-rancher:~# kubectl get pods -n onap | egrep "dl-handler"
|
...
cfy deployments create -b datalake-feeder feeder-deploy
cfy deployments create -b datalake-admin-ui admin-ui-deploy |
Launch Service
Next, we are going to launch the datalake.
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
|
Functional tests
...
cfy executions start -d feeder-deploy install
cfy executions start -d admin-ui-deploy install |
To Un-deploy
Uninstall running component and delete deployment
| Code Block | ||||||
|---|---|---|---|---|---|---|
| Code Block | ||||||
|
<Add below steps to configure DL-Handler to subscribe and feed into external DL with step-by-step procedure>
Dynamic Configuration Update
As the dl-handler service periodically polls Consul KV using configbindingService api's - the run time configuration of dl-handler service can be updated dynamically without having to redeploy/restart the service. The updates to configuration can be triggered either from Policy (or CLAMP) or made directly in Consul.
Locate the servicename by executing into dl-handler Service pod and getting env HOSTNAME value
| |||
cfy uninstall feeder-deploy
cfy uninstall admin-ui-deploy |
Delete blueprint
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
root@k8s-rancher:~# kubectl exec -it -n onap dep-s78f36f2daf0843518f2e25184769eb8b-dcae-dl-handler-servithzx2 /bin/bash
Defaulting container name to s78f36f2daf0843518f2e25184769eb8b-dcae-dl-handler-service.
Use 'kubectl describe pod/dep-s78f36f2daf0843518f2e25184769eb8b-dcae-dl-handler-servithzx2 -n onap' to see all of the containers in this pod.
misshtbt@s78f36f2daf0843518f2e25184769eb8b-dcae-dl-handler-service:~/bin$ env | grep HOSTNAME
HOSTNAME=s78f36f2daf0843518f2e25184769eb8b-dcae-dl-handler-service |
...
| |||
cfy blueprints delete datalake-feeder
cfy blueprints deltet datalake-admin-ui |
Deploy external database
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
http://<k8snodeip>:30270/ui/#/dc1/kv/ |
...
| |||
docker run -d -p 27017:27017 --name mongodb mongo
docker start mongodb |