Overview

DataFile Collector provides ONAP Operators the ability to upload PM XML Files from PNFs, by sending a fileReady VES Event. These files are published to the DataRouter for further use within ONAP.

Click here to expand...

Bulk PM Flow:

NF notifies DCAE that a PM file is available for upload.
DataFile Collector uploads PM Files from NF and stores them in Data Router.
PM Mapper creates custom Performance events (VES) from the PM File data according to PM Mapping File.
Analytics Applications use these performance events for targeted analysis of network operations.

Blueprint/model/image

dfc blueprint(working) dfc blueprint(El-Alto)	dfc models	dfc image El-Alto (DFC 1.2.3)

Deployment Prerequisite/dependencies

Make sure that cfy is installed and configured to work with the Cloudify deployment.
Make sure the Message Router and Data Router are running.

Deployment Steps

Download the dfc blueprint.
Transfer blueprint file in DCAE bootstrap POD under /blueprints directory.

Execute bash on the DCAE bootstrap POD.

kubectl -n onap exec -it <dev-dcaegen2-dcae-bootstrap> bash

Run Cloudify install command to install dfc.
```
cfy install <dfc-blueprint-path> --blueprint-id <blueprint-name> --deployment-id <deployment-name>
```
Sample output
```
$ cfy install /blueprints/k8s-datafile.yaml --blueprint-id datafile --deployment-id datafile
```
Uploading blueprint /blueprints/k8s-datafile.yaml...
k8s-datafile.yaml |###################################################| 100.0%
Blueprint uploaded. The blueprint's id is datafile
Creating new deployment from blueprint datafile...
Deployment created. The deployment's id is datafile
Executing workflow install on deployment datafile [timeout=900 seconds]
Deployment environment creation is pending...
2019-05-20 11:35:50.030 CFY <datafile> Starting 'create_deployment_environment' workflow execution
2019-05-20 11:35:50.727 CFY <datafile> Installing deployment plugins
2019-05-20 11:35:50.727 CFY <datafile> Sending task 'cloudify_agent.operations.install_plugins'
2019-05-20 11:35:50.727 CFY <datafile> Task started 'cloudify_agent.operations.install_plugins'
2019-05-20 11:35:51.327 LOG <datafile> INFO: Installing plugin: k8s
2019-05-20 11:35:51.327 LOG <datafile> INFO: Using existing installation of managed plugin: c5ce40e9-2d3f-45f4-8fac-479f7de4f146 [package_name: k8splugin, package_version: 1.4.13, supported_platform: linux_x86_64, distribution: centos, distribution_release: core]
2019-05-20 11:35:51.327 CFY <datafile> Task succeeded 'cloudify_agent.operations.install_plugins'
2019-05-20 11:35:51.327 CFY <datafile> Skipping starting deployment policy engine core - no policies defined
2019-05-20 11:35:51.327 CFY <datafile> Creating deployment work directory
2019-05-20 11:35:52.049 CFY <datafile> 'create_deployment_environment' workflow execution succeeded
2019-05-20 11:35:55.034 CFY <datafile> Starting 'install' workflow execution
2019-05-20 11:35:56.265 CFY <datafile> [datafile-collector_lwvvxh] Creating node instance
2019-05-20 11:35:56.985 CFY <datafile> [datafile-collector_lwvvxh.create] Sending task 'k8splugin.create_for_platforms'
2019-05-20 11:35:58.707 LOG <datafile> [datafile-collector_lwvvxh.create] INFO: Added config for dcae-datafile-collector
2019-05-20 11:35:59.214 LOG <datafile> [datafile-collector_lwvvxh.create] INFO: Done setting up: dcae-datafile-collector
2019-05-20 11:35:59.723 CFY <datafile> [datafile-collector_lwvvxh.create] Task succeeded 'k8splugin.create_for_platforms'
2019-05-20 11:35:59.723 CFY <datafile> [datafile-collector_lwvvxh] Node instance created
2019-05-20 11:35:59.723 CFY <datafile> [datafile-collector_lwvvxh] Configuring node instance: nothing to do
2019-05-20 11:36:00.332 CFY <datafile> [datafile-collector_lwvvxh] Starting node instance
2019-05-20 11:36:00.332 CFY <datafile> [datafile-collector_lwvvxh.start] Sending task 'k8splugin.create_and_start_container_for_platforms'
2019-05-20 11:36:02.383 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: Passing k8sconfig: {'tls': {u'cert_path': u'/opt/tls/shared', u'image': u'nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.tls-init-container:1.0.3'}, 'filebeat': {u'config_map': u'dcae-filebeat-configmap', u'config_path': u'/usr/share/filebeat/filebeat.yml', u'log_path': u'/var/log/onap', u'image': u'docker.elastic.co/beats/filebeat:5.5.0', u'data_path': u'/usr/share/filebeat/data', u'config_subpath': u'filebeat.yml'}, 'consul_dns_name': u'consul-server.onap', 'image_pull_secrets': [u'onap-docker-registry-key'], 'namespace': u'onap', 'consul_host': 'consul-server:8500', 'default_k8s_location': u'central'}
2019-05-20 11:36:01.709 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: Starting k8s deployment for dcae-datafile-collector, image: nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:1.1.2, env: {'CONSUL_HOST': u'consul-server.onap', 'CONFIG_BINDING_SERVICE': 'config-binding-service'}, kwargs: {'readiness': {u'endpoint': u'/heartbeat', u'type': u'http', u'timeout': u'1s', u'interval': u'15s'}, 'tls_info': {u'use_tls': True, u'cert_directory': u'/opt/app/datafile/etc/cert/'}, 'replicas': 1, 'labels': {'cfydeployment': u'datafile', 'cfynodeinstance': u'datafile-collector_lwvvxh', 'cfynode': u'datafile-collector'}, 'ctx': <cloudify.context.CloudifyContext object at 0x7f51a5c162d0>, 'always_pull_image': False, 'resource_config': {}, 'log_info': {u'log_directory': u'/opt/app/datafile/logs'}, u'ports': [u'8100:0', u'8433:0'], 'k8s_location': u'central'}
2019-05-20 11:36:02.383 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: k8s deployment initiated successfully for dcae-datafile-collector: {'services': ['dcae-datafile-collector'], 'namespace': u'onap', 'location': u'central', 'deployment': 'dep-dcae-datafile-collector'}
2019-05-20 11:36:02.383 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: Waiting up to 1800 secs for dcae-datafile-collector to become ready
2019-05-20 11:36:57.264 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: k8s deployment is ready for: dcae-datafile-collector
2019-05-20 11:36:57.888 CFY <datafile> [datafile-collector_lwvvxh.start] Task succeeded 'k8splugin.create_and_start_container_for_platforms'
2019-05-20 11:36:58.525 CFY <datafile> [datafile-collector_lwvvxh] Node instance started
2019-05-20 11:36:58.525 CFY <datafile> 'install' workflow execution succeeded
Finished executing workflow install on deployment datafile
* Run 'cfy events list -e 5c39a15a-a30e-4a3e-8196-94139c5ef368' to retrieve the execution's events/logs

Validation

curl <dcaegen2-dcae-healthcheck> and check if datafile-collector is in 'ready' state.

$ curl 10.42.181.66 | jq

{
"type": "summary",
"count": 12,
"ready": 11,
"items": [
{
"name": "dev-dcaegen2-dcae-pm-mapper",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-config-binding-service",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-inventory-api",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-servicechange-handler",
"ready": 0,
"unavailable": 1
},
{
"name": "dev-dcaegen2-dcae-deployment-handler",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-policy-handler",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-ves-collector",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-tca-analytics",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-prh",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-hv-ves-collector",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-datafile-collector",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-snmptrap-collector",
"ready": 1,
"unavailable": 0
}
]
}

Browser not supported