Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand

Gliffy
nameDataFile Collector Sequence Diagram 1
pagePin3

Bulk PM Flow:

  1. NF notifies DCAE that a PM file is available for upload.
  2. DataFile Collector uploads PM Files from NF and stores them in Data Router.
  3. PM Mapper creates custom Performance events (VES) from the PM File data according to PM Mapping File.
  4. Analytics Applications use these performance events for targeted analysis of network operations.



Blueprint/model/image



Deployment Prerequisite/dependencies

  1. Make sure that cfy is installed and configured to work with the Cloudify deployment.
  2. Make sure the Message Router and Data Router are running.

Deployment Steps

  1. Download the dfc blueprint.
  2. Transfer blueprint file in DCAE bootstrap POD under /blueprints directory.
  3. Execute bash on the DCAE bootstrap POD. 

    Code Block
    languagebash
    kubectl -n onap exec -it <dev-dcaegen2-dcae-bootstrap> bash


  4. Run Cloudify install command to install dfc.

    Code Block
    languagebash
    cfy install <dfc-blueprint-path> --blueprint-id <blueprint-name> --deployment-id <deployment-name>


    Expand
    titleSample output
    $ cfy install /blueprints/k8s-datafile.yaml --blueprint-id datafile --deployment-id datafile

    Uploading blueprint /blueprints/k8s-datafile.yaml...
    k8s-datafile.yaml |###################################################| 100.0%
    Blueprint uploaded. The blueprint's id is datafile
    Creating new deployment from blueprint datafile...
    Deployment created. The deployment's id is datafile
    Executing workflow install on deployment datafile [timeout=900 seconds]
    Deployment environment creation is pending...
    2019-05-20 11:35:50.030 CFY <datafile> Starting 'create_deployment_environment' workflow execution
    2019-05-20 11:35:50.727 CFY <datafile> Installing deployment plugins
    2019-05-20 11:35:50.727 CFY <datafile> Sending task 'cloudify_agent.operations.install_plugins'
    2019-05-20 11:35:50.727 CFY <datafile> Task started 'cloudify_agent.operations.install_plugins'
    2019-05-20 11:35:51.327 LOG <datafile> INFO: Installing plugin: k8s
    2019-05-20 11:35:51.327 LOG <datafile> INFO: Using existing installation of managed plugin: c5ce40e9-2d3f-45f4-8fac-479f7de4f146 [package_name: k8splugin, package_version: 1.4.13, supported_platform: linux_x86_64, distribution: centos, distribution_release: core]
    2019-05-20 11:35:51.327 CFY <datafile> Task succeeded 'cloudify_agent.operations.install_plugins'
    2019-05-20 11:35:51.327 CFY <datafile> Skipping starting deployment policy engine core - no policies defined
    2019-05-20 11:35:51.327 CFY <datafile> Creating deployment work directory
    2019-05-20 11:35:52.049 CFY <datafile> 'create_deployment_environment' workflow execution succeeded
    2019-05-20 11:35:55.034 CFY <datafile> Starting 'install' workflow execution
    2019-05-20 11:35:56.265 CFY <datafile> [datafile-collector_lwvvxh] Creating node instance
    2019-05-20 11:35:56.985 CFY <datafile> [datafile-collector_lwvvxh.create] Sending task 'k8splugin.create_for_platforms'
    2019-05-20 11:35:58.707 LOG <datafile> [datafile-collector_lwvvxh.create] INFO: Added config for dcae-datafile-collector
    2019-05-20 11:35:59.214 LOG <datafile> [datafile-collector_lwvvxh.create] INFO: Done setting up: dcae-datafile-collector
    2019-05-20 11:35:59.723 CFY <datafile> [datafile-collector_lwvvxh.create] Task succeeded 'k8splugin.create_for_platforms'
    2019-05-20 11:35:59.723 CFY <datafile> [datafile-collector_lwvvxh] Node instance created
    2019-05-20 11:35:59.723 CFY <datafile> [datafile-collector_lwvvxh] Configuring node instance: nothing to do
    2019-05-20 11:36:00.332 CFY <datafile> [datafile-collector_lwvvxh] Starting node instance
    2019-05-20 11:36:00.332 CFY <datafile> [datafile-collector_lwvvxh.start] Sending task 'k8splugin.create_and_start_container_for_platforms'
    2019-05-20 11:36:02.383 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: Passing k8sconfig: {'tls': {u'cert_path': u'/opt/tls/shared', u'image': u'nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.tls-init-container:1.0.3'}, 'filebeat': {u'config_map': u'dcae-filebeat-configmap', u'config_path': u'/usr/share/filebeat/filebeat.yml', u'log_path': u'/var/log/onap', u'image': u'docker.elastic.co/beats/filebeat:5.5.0', u'data_path': u'/usr/share/filebeat/data', u'config_subpath': u'filebeat.yml'}, 'consul_dns_name': u'consul-server.onap', 'image_pull_secrets': [u'onap-docker-registry-key'], 'namespace': u'onap', 'consul_host': 'consul-server:8500', 'default_k8s_location': u'central'}
    2019-05-20 11:36:01.709 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: Starting k8s deployment for dcae-datafile-collector, image: nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:1.1.2, env: {'CONSUL_HOST': u'consul-server.onap', 'CONFIG_BINDING_SERVICE': 'config-binding-service'}, kwargs: {'readiness': {u'endpoint': u'/heartbeat', u'type': u'http', u'timeout': u'1s', u'interval': u'15s'}, 'tls_info': {u'use_tls': True, u'cert_directory': u'/opt/app/datafile/etc/cert/'}, 'replicas': 1, 'labels': {'cfydeployment': u'datafile', 'cfynodeinstance': u'datafile-collector_lwvvxh', 'cfynode': u'datafile-collector'}, 'ctx': <cloudify.context.CloudifyContext object at 0x7f51a5c162d0>, 'always_pull_image': False, 'resource_config': {}, 'log_info': {u'log_directory': u'/opt/app/datafile/logs'}, u'ports': [u'8100:0', u'8433:0'], 'k8s_location': u'central'}
    2019-05-20 11:36:02.383 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: k8s deployment initiated successfully for dcae-datafile-collector: {'services': ['dcae-datafile-collector'], 'namespace': u'onap', 'location': u'central', 'deployment': 'dep-dcae-datafile-collector'}
    2019-05-20 11:36:02.383 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: Waiting up to 1800 secs for dcae-datafile-collector to become ready
    2019-05-20 11:36:57.264 LOG <datafile> [datafile-collector_lwvvxh.start] INFO: k8s deployment is ready for: dcae-datafile-collector
    2019-05-20 11:36:57.888 CFY <datafile> [datafile-collector_lwvvxh.start] Task succeeded 'k8splugin.create_and_start_container_for_platforms'
    2019-05-20 11:36:58.525 CFY <datafile> [datafile-collector_lwvvxh] Node instance started
    2019-05-20 11:36:58.525 CFY <datafile> 'install' workflow execution succeeded
    Finished executing workflow install on deployment datafile
    * Run 'cfy events list -e 5c39a15a-a30e-4a3e-8196-94139c5ef368' to retrieve the execution's events/logs



...

Validation

curl <dcaegen2-dcae-healthcheck> and check if datafile-collector is in 'ready' state. 

$ curl 10.42.181.66 | jq

{
"type": "summary",
"count": 12,
"ready": 11,
"items": [
{
"name": "dev-dcaegen2-dcae-pm-mapper",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-config-binding-service",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-inventory-api",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-servicechange-handler",
"ready": 0,
"unavailable": 1
},
{
"name": "dev-dcaegen2-dcae-deployment-handler",
"ready": 1,
"unavailable": 0
},
{
"name": "dev-dcaegen2-dcae-policy-handler",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-ves-collector",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-tca-analytics",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-prh",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-hv-ves-collector",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-datafile-collector",
"ready": 1,
"unavailable": 0
},
{
"name": "dep-dcae-snmptrap-collector",
"ready": 1,
"unavailable": 0
}
]
}