Step 1: Design the Control Loop Flow in SDC (DCAE-Design Studio)

What will be the scope of control loop flows in Casablanca:

Which microservices will be used ?
TCA, Holmes (Can be configured in CLAMP and based on pattern matching in the Blueprint)
Will flows contain more than one microservice node?
No
Will microservices be capable of receiving more than one configuration policy?
None
Will the singleton vs. dynamically deployed distrinction distinction be described in the TOSCA?
No
Will collectors be deployed dynamically or statically?
Statically, the VES collector is deployed in DCAE
Will DCAE Design Studio allow for the piecing together of flows in Casablanca?
Which blueprint can be uploaded to SDC ? (Examples)

Code Block

title	Blueprint for TCA on OOM
collapse	true

#
# ============LICENSE_START====================================================
# =============================================================================
# Copyright (c) 2018 AT&T Intellectual Property. All rights reserved.
# =============================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============LICENSE_END======================================================

tosca_definitions_version: cloudify_dsl_1_3

description: >
  This blueprint deploys/manages the TCA module as a Docker container

imports:
  - http://www.getcloudify.org/spec/cloudify/3.4/types.yaml
  - https://nexus.onap.org/service/local/repositories/raw/content/org.onap.dcaegen2.platform.plugins/R3/k8splugin/1.4.3/k8splugin_types.yaml
  - https://nexus.onap.org/service/local/repositories/raw/content/org.onap.dcaegen2.platform.plugins/R2/dcaepolicyplugin/2.3.0/dcaepolicyplugin_types.yaml

inputs:
  aaiEnrichmentHost:
    type: string
    default: "aai.onap.svc.cluster.local"
  aaiEnrichmentPort:
    type: string
    default: "8443"
  enableAAIEnrichment:
    type: string
    default: true
  dmaap_host:
    type: string
    default: message-router.onap.svc.cluster.local
  dmaap_port:
    type: string
    default: "3904"
  enableRedisCaching:
    type: string
    default: false
  redisHosts:
    type: string
    default: dcae-redis.onap.svc.cluster.local:6379
  tag_version:
    type: string
    default: "nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.tca-cdap-container:1.0.0"
  consul_host:
    type: string
    default: consul-server.onap.svc.cluster.local
  consul_port:
    type: string
    default: "8500"
  cbs_host:
    type: string
    default: "config-binding-service.dcae.svc.cluster.local"
  cbs_port:
    type: string
    default: "10000"
  policy_id:
    type: string
    default: "none"
  external_port:
    type: string
    description: Kubernetes node port on which CDAPgui is exposed
    default: "32010"

node_templates:
  tca_k8s:
    type: dcae.nodes.ContainerizedServiceComponent
    relationships:
      - target: tca_policy
        type: cloudify.relationships.depends_on
    properties:
        service_component_type: 'dcaegen2-analytics-tca'
        application_config: {}
        docker_config: {}
        image:
          get_input: tag_version
        log_info:
          log_directory: "/opt/app/TCAnalytics/logs"
        application_config:
            app_config:
                appDescription: DCAE Analytics Threshold Crossing Alert Application
                appName: dcae-tca
                tcaAlertsAbatementTableName: TCAAlertsAbatementTable
                tcaAlertsAbatementTableTTLSeconds: '1728000'
                tcaSubscriberOutputStreamName: TCASubscriberOutputStream
                tcaVESAlertsTableName: TCAVESAlertsTable
                tcaVESAlertsTableTTLSeconds: '1728000'
                tcaVESMessageStatusTableName: TCAVESMessageStatusTable
                tcaVESMessageStatusTableTTLSeconds: '86400'
                thresholdCalculatorFlowletInstances: '2'
            app_preferences:
                aaiEnrichmentHost:
                    get_input: aaiEnrichmentHost
                aaiEnrichmentIgnoreSSLCertificateErrors: 'true'
                aaiEnrichmentPortNumber: '8443'
                aaiEnrichmentProtocol: https
                aaiEnrichmentUserName: DCAE
                aaiEnrichmentUserPassword: DCAE
                aaiVMEnrichmentAPIPath: /aai/v11/search/nodes-query
                aaiVNFEnrichmentAPIPath: /aai/v11/network/generic-vnfs/generic-vnf
                enableAAIEnrichment:
                    get_input: enableAAIEnrichment
                enableRedisCaching:
                    get_input: enableRedisCaching
                redisHosts:
                    get_input: redisHosts
                enableAlertCEFFormat: 'false'
                publisherContentType: application/json
                publisherHostName:
                    get_input: dmaap_host
                publisherHostPort:
                    get_input: dmaap_port
                publisherMaxBatchSize: '1'
                publisherMaxRecoveryQueueSize: '100000'
                publisherPollingInterval: '20000'
                publisherProtocol: http
                publisherTopicName: unauthenticated.DCAE_CL_OUTPUT
                subscriberConsumerGroup: OpenDCAE-c12
                subscriberConsumerId: c12
                subscriberContentType: application/json
                subscriberHostName:
                    get_input: dmaap_host
                subscriberHostPort:
                    get_input: dmaap_port
                subscriberMessageLimit: '-1'
                subscriberPollingInterval: '30000'
                subscriberProtocol: http
                subscriberTimeoutMS: '-1'
                subscriberTopicName: unauthenticated.SEC_MEASUREMENT_OUTPUT
                tca_policy: '{"domain":"measurementsForVfScaling","metricsPerEventName":[{"eventName":"vFirewallBroadcastPackets","controlLoopSchemaType":"VNF","policyScope":"DCAE","policyName":"DCAE.Config_tca-hi-lo","policyVersion":"v0.0.1","thresholds":[{"closedLoopControlName":"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta","thresholdValue":300,"direction":"LESS_OR_EQUAL","severity":"MAJOR","closedLoopEventStatus":"ONSET"},{"closedLoopControlName":"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta","thresholdValue":700,"direction":"GREATER_OR_EQUAL","severity":"CRITICAL","closedLoopEventStatus":"ONSET"}]},{"eventName":"vLoadBalancer","controlLoopSchemaType":"VM","policyScope":"DCAE","policyName":"DCAE.Config_tca-hi-lo","policyVersion":"v0.0.1","thresholds":[{"closedLoopControlName":"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta","thresholdValue":300,"direction":"GREATER_OR_EQUAL","severity":"CRITICAL","closedLoopEventStatus":"ONSET"}]},{"eventName":"Measurement_vGMUX","controlLoopSchemaType":"VNF","policyScope":"DCAE","policyName":"DCAE.Config_tca-hi-lo","policyVersion":"v0.0.1","thresholds":[{"closedLoopControlName":"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value","thresholdValue":0,"direction":"EQUAL","severity":"MAJOR","closedLoopEventStatus":"ABATED"},{"closedLoopControlName":"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value","thresholdValue":0,"direction":"GREATER","severity":"CRITICAL","closedLoopEventStatus":"ONSET"}]}]}'
        service_component_type: dcaegen2-analytics_tca
    interfaces:
      cloudify.interfaces.lifecycle:
        start:
          inputs:
            envs:
                DMAAPHOST:
                    { get_input: dmaap_host }
                DMAAPPORT:
                    { get_input: dmaap_port }
                DMAAPPUBTOPIC: "unauthenticated.DCAE_CL_OUTPUT"
                DMAAPSUBTOPIC: "unauthenticated.SEC_MEASUREMENT_OUTPUT"
                AAIHOST:
                    { get_input: aaiEnrichmentHost }
                AAIPORT:
                    { get_input: aaiEnrichmentPort }
                CONSUL_HOST:
                    { get_input: consul_host }
                CONSUL_PORT:
                    { get_input: consul_port }
                CBS_HOST:
                    { get_input: cbs_host }
                CBS_PORT:
                    { get_input: cbs_port }
                CONFIG_BINDING_SERVICE: "config_binding_service"
            ports:
              - concat: ["11011:", { get_input: external_port }]
  tca_policy:
    type: dcae.nodes.policy
    properties:
      policy_id:
           get_input: policy_id

Code Block

title	Blueprint for TCA on HEAT
collapse	true

tosca_definitions_version: cloudify_dsl_1_3
imports:
  - "http://www.getcloudify.org/spec/cloudify/3.4/types.yaml"
  - https://nexus.onap.org/service/local/repositories/raw/content/org.onap.dcaegen2.platform.plugins/R3/dockerplugin/3.2.0/dockerplugin_types.yaml
  - https://nexus.onap.org/service/local/repositories/raw/content/org.onap.dcaegen2.platform.plugins/R3/relationshipplugin/1.0.0/relationshipplugin_types.yaml
  - https://nexus.onap.org/service/local/repositories/raw/content/org.onap.dcaegen2.platform.plugins/R3/dcaepolicyplugin/2.3.0/dcaepolicyplugin_types.yaml

inputs:
  dh_override:
    type: string
    default: "dockerhost"
  dh_location_id:
    type: string
    default: "zone1"
  aaiEnrichmentHost:
    type: string
    default: "none"
  aaiEnrichmentPort:
    type: string    
    default: 8443
  enableAAIEnrichment:
    type: string
    default: false
  dmaap_host:
    type: string
    default: dmaap.onap-message-router   
  dmaap_port:
    type: string
    default: 3904    
  enableRedisCaching:
    type: string
    default: false    
  redisHosts:
    type: string      
  tag_version:
    type: string
    default: "nexus3.onap.org:10001/onap/org.onap.dcaegen2.deployments.tca-cdap-container:1.1.0"
  consul_host:
    type: string
    default: consul-server.onap-consul
  consul_port:
    type: string
    default: "8500"
  cbs_host:
    type: string
    default: "config-binding-service.dcae"
  cbs_port:
    type: string
    default: "10000"
  policy_id:
    type: string
    default: "none"
  external_port:
    type: string
    description: "Port for CDAPgui to be exposed"
    default: "32010"
  scn_name: 
    default: dcaegen2-analytics_tca_clampinstance_1
    type: string
node_templates:
  docker_service_host:
    properties:
      docker_host_override:
        get_input: dh_override
      location_id:
        get_input: dh_location_id
    type: dcae.nodes.SelectedDockerHost
  tca_docker:
    relationships:
       - type: dcae.relationships.component_contained_in
         target: docker_service_host
       - target: tca_policy
         type: cloudify.relationships.depends_on        
    type: dcae.nodes.DockerContainerForComponentsUsingDmaap
    properties:
        application_config:
            app_config:
                appDescription: DCAE Analytics Threshold Crossing Alert Application
                appName: dcae-tca
                tcaAlertsAbatementTableName: TCAAlertsAbatementTable
                tcaAlertsAbatementTableTTLSeconds: '1728000'
                tcaSubscriberOutputStreamName: TCASubscriberOutputStream
                tcaVESAlertsTableName: TCAVESAlertsTable
                tcaVESAlertsTableTTLSeconds: '1728000'
                tcaVESMessageStatusTableName: TCAVESMessageStatusTable
                tcaVESMessageStatusTableTTLSeconds: '86400'
                thresholdCalculatorFlowletInstances: '2'
            app_preferences:
                aaiEnrichmentHost: 
                    get_input: aaiEnrichmentHost
                aaiEnrichmentIgnoreSSLCertificateErrors: 'true'
                aaiEnrichmentPortNumber: '8443'
                aaiEnrichmentProtocol: https
                aaiEnrichmentUserName: DCAE
                aaiEnrichmentUserPassword: DCAE
                aaiVMEnrichmentAPIPath: /aai/v11/search/nodes-query
                aaiVNFEnrichmentAPIPath: /aai/v11/network/generic-vnfs/generic-vnf
                enableAAIEnrichment: 
                    get_input: enableAAIEnrichment
                enableRedisCaching: 
                    get_input: enableRedisCaching
                redisHosts: 
                    get_input: redisHosts
                enableAlertCEFFormat: 'false'
                publisherContentType: application/json
                publisherHostName: 
                    get_input: dmaap_host
                publisherHostPort: 
                    get_input: dmaap_port                  
                publisherMaxBatchSize: '1'
                publisherMaxRecoveryQueueSize: '100000'
                publisherPollingInterval: '20000'
                publisherProtocol: http
                publisherTopicName: unauthenticated.DCAE_CL_OUTPUT
                subscriberConsumerGroup: OpenDCAE-c12
                subscriberConsumerId: c12
                subscriberContentType: application/json
                subscriberHostName: 
                    get_input: dmaap_host
                subscriberHostPort:
                    get_input: dmaap_port                                  
                subscriberMessageLimit: '-1'
                subscriberPollingInterval: '30000'
                subscriberProtocol: http
                subscriberTimeoutMS: '-1'
                subscriberTopicName: unauthenticated.SEC_MEASUREMENT_OUTPUT
                tca_policy_default: '{"domain":"measurementsForVfScaling","metricsPerEventName":[{"eventName":"vFirewallBroadcastPackets","controlLoopSchemaType":"VNF","policyScope":"DCAE","policyName":"DCAE.Config_tca-hi-lo","policyVersion":"v0.0.1","thresholds":[{"closedLoopControlName":"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta","thresholdValue":300,"direction":"LESS_OR_EQUAL","severity":"MAJOR","closedLoopEventStatus":"ONSET"},{"closedLoopControlName":"ControlLoop-vFirewall-d0a1dfc6-94f5-4fd4-a5b5-4630b438850a","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta","thresholdValue":700,"direction":"GREATER_OR_EQUAL","severity":"CRITICAL","closedLoopEventStatus":"ONSET"}]},{"eventName":"vLoadBalancer","controlLoopSchemaType":"VM","policyScope":"DCAE","policyName":"DCAE.Config_tca-hi-lo","policyVersion":"v0.0.1","thresholds":[{"closedLoopControlName":"ControlLoop-vDNS-6f37f56d-a87d-4b85-b6a9-cc953cf779b3","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.vNicUsageArray[*].receivedTotalPacketsDelta","thresholdValue":300,"direction":"GREATER_OR_EQUAL","severity":"CRITICAL","closedLoopEventStatus":"ONSET"}]},{"eventName":"Measurement_vGMUX","controlLoopSchemaType":"VNF","policyScope":"DCAE","policyName":"DCAE.Config_tca-hi-lo","policyVersion":"v0.0.1","thresholds":[{"closedLoopControlName":"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value","thresholdValue":0,"direction":"EQUAL","severity":"MAJOR","closedLoopEventStatus":"ABATED"},{"closedLoopControlName":"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e","version":"1.0.2","fieldPath":"$.event.measurementsForVfScalingFields.additionalMeasurements[*].arrayOfFields[0].value","thresholdValue":0,"direction":"GREATER","severity":"CRITICAL","closedLoopEventStatus":"ONSET"}]}]}'
        service_component_type: dcaegen2-analytics_tca    
        docker_config:
            healthcheck:
               endpoint: /
               interval: 15s
               timeout: 1s
               type: http
        image:
            get_input: tag_version   
        service_component_name_override: 
            get_input: scn_name            
    interfaces:
      cloudify.interfaces.lifecycle:
        start:
          inputs:
            envs:
                DMAAPHOST: 
                    { get_input: dmaap_host }
                DMAAPPORT:
                    { get_input: dmaap_port }
                DMAAPPUBTOPIC: "unauthenticated.DCAE_CL_OUTPUT"
                DMAAPSUBTOPIC: "unauthenticated.SEC_MEASUREMENT_OUTPUT"
                AAIHOST: 
                    { get_input: aaiEnrichmentHost }
                AAIPORT: 
                    { get_input: aaiEnrichmentPort }
                CONSUL_HOST: 
                    { get_input: consul_host }
                CONSUL_PORT: 
                    { get_input: consul_port }
                CBS_HOST: 
                    { get_input: cbs_host }
                CBS_PORT: 
                    { get_input: cbs_port }
                CONFIG_BINDING_SERVICE: "config_binding_service"                
                SERVICE_11011_NAME: 
                    { get_input: scn_name }
                SERVICE_11015_IGNORE: "true"                
            ports:
              - concat: ["11011:", { get_input: external_port }]        
        stop:
          inputs:
            cleanup_image: true              
  tca_policy:
    type: dcae.nodes.policy
    properties:
      policy_id:
           get_input: policy_id

Step 2:

...

Distribute artifacts

Cloudify TOSCA is different from standard TOSCA
Many details of deployment are not relevant to CLAMP
For these reasons, there should be parallel artifacts: Service Template and Blueprint
- Service Template is for use by CLAMP and is not Cloudify-specific
- Based on Service Template, any deployment of the parallel Blueprint should be possible from CLAMP
- Service Template and Blueprint must define the same inputs
Will DMaaP topics be provisioned at deploy time?
- If a new topic is created, this would require CLAMP dashboard awareness in order to monitor the messages on that topic
- How will credentials for DMaaP be provisioned?

Step 3: Distribute artifacts

The artifact distribution must be done from SDC to CLAMP (see CLAMP videos), in order to CLAMP videos#DistributiontoCLAMP

The artifact distribution must be done from SDC to CLAMP, in order to distribute the Blueprint and service/VF associated.

This will create Closed Loops associated to VF and Service in Clamp database

Step

...

3: CLAMP Processes artifact

Parsing of the SDC notification is done by CLAMP and is based on string patterns found in the Blueprint.

Only Holmes and TCA are currently supported by default, but this can be customized by the Clamp config files.

The default config file is bundled in the JAR: src/main/resources/clds/templates/blueprint-parser-mapping.json

But this can be overridden by specifying the spring config "clamp.config.sdc.blueprint.parser.mapping" in clamp.env for docker-compose (set by default to "classpath:/clds/blueprint-parser-mapping.json")

Configure

How will this be affected by changes in Policy Lifecycle APIs? -mapping.json")

Configure

Configure the Closed Loop in the CLAMP UI by opening the Closed loop created by the SDC distribution. The Guard/Operational and configuration policies must be configured in the UI.

CLAMP videos#ConfigureClosedloopinCLAMPUI

Once done, the closed loop can be submitted to Policy by clicking on Manage → Submit

CLAMP videos#SubmitConfigurationandOperationalPoliciestoPolicyEngine

Deploy

The micro-service deployment can be done onhttps://wiki.onap.org/display/DW/Policy+Platform+API

Deploy

Update

...

CLAMP+videos#CLAMPvideos-Deploy/ControltheClosedLooponDCAE DCAE from CLAMP, a query is sent from CLAMP to DCAE with the deployment parameters configured in CLAMP.

CLAMP videos#Deploy/ControltheClosedLooponDCAE

Update / Suspend

The policies can be updated at runtime, microservice "Stopped/Undeployed"

CLAMP videos#Deploy/ControltheClosedLooponDCAE

Versions Compared

Old Version 10

New Version 11

Key

Step 1: Design the Control Loop Flow in SDC (DCAE-Design Studio)

Step 2:

Distribute artifacts

Step 3: Distribute artifacts

Step

3: CLAMP Processes artifact

Configure

Configure

Deploy

Deploy

Update

Update / Suspend

Page Comparison

Versions Compared

Old Version 10

New Version 11

Key

Step 1: Design the Control Loop Flow in SDC (DCAE-Design Studio)

Step 2:

Distribute artifacts

Step 3: Distribute artifacts

Step

3: CLAMP Processes artifact

Configure

Configure

Deploy

Deploy

Update

Update / Suspend