2020-01-15 LF ONAP Prague DDF Policy Framework - Frankfurt Improvements

2020-01-15 LF ONAP Prague DDF Policy Framework - Frankfurt Improvements

New component capabilities for Frankfurt, i.e. the functional enhancements.

Frankfurt release targets the completion of the work started in Dublin to re-build the Policy Framework infrastructure. The new capabilities were to complete missing functionality that was previously in the legacy components, or was targeted to be part of the platform and was not able to be finished in Dublin:

Consolidation of Policy Health Check - China Ericsson Team

https://lf-onap.atlassian.net/browse/POLICY-2025 - Per Integration team, one single health check requested that gives status for the whole of Policy Framework components.

  • New API implemented in PAP : /policy/pap/v1/pdps/healthcheck

Example API result:

{ "pdps": { "xacml": [ { "instanceId": "CN-00000994", "pdpState": "ACTIVE", "healthy": "HEALTHY" } ], "drools": [ { "instanceId": "CN-00000994", "pdpState": "ACTIVE", "healthy": "HEALTHY" } ], "apex": [ { "instanceId": "CN-00000994", "pdpState": "ACTIVE", "healthy": "HEALTHY" } ] }, "healthy": true, "api": { "name": "Policy API", "url": "http://policy-api:6969/policy/api/v1/healthcheck", "healthy": true, "code": 200, "message": "alive" }, "distribution": { "name": "Policy SSD", "url": "http://policy-distribution:6969/healthcheck", "healthy": true, "code": 200, "message": "alive" }, "pap": { "name": "Policy PAP", "url": "http://policy-pap:6969/policy/pap/v1/healthcheck", "healthy": true, "code": 200, "message": "alive" } }

 

PDP Statistics - China Ericsson Team

https://lf-onap.atlassian.net/browse/POLICY-2025 - The same epic also covers the work for collecting various statistics of registered PDP engines and storing them in the Policy DB.

  • Policy models and related provider classes created for storing PDP Statistics in DB.

  • REST API created in PAP to fetch the PDP Statistics saved in Policy DB.

  • PdpStatus heartbeat messages from apex-pdp has been updated to include current statistics in every interval. Xacml-pdp & drools-pdp will be updated shortly.

  • PAP handlers has been updated to collect & store the statistics coming in heartbeat messages to Policy DB.

PDP Monitoring GUI - China Ericsson Team

https://lf-onap.atlassian.net/browse/POLICY-1751 - This epic covers the work to create a GUI that polls the PDP statistics from PAP for monitoring PDP engines. This is currently work in progress.

A new UI is being developed to show

  • the list of PDP engines registered with PAP

  • the statistics of an individual PDP engine.

  • the cumulative statistics of a group of PDP engines.

  • the cumulative statistics of all PDP engines for a particular domain.

Integration of CDS As an Actor @Former user (Deleted) @Rashmi Pujar

https://lf-onap.atlassian.net/browse/POLICY-2087 - Started in Dublin as dark code, finishing the support for this new actor to be fully available for Control Loop Operational Policies

  • CDS is added as an actor in Policy to take actions

  • Integration is done by implementing a gRPC call to CDS

  • Validation is done against vFirewall use case

  • Sample Operational Policy YAML for vFirewall usecase:

controlLoop: version: 2.0.0 controlLoopName: ControlLoop-vFirewall-7e4fbe9c-d612-4ec5-bbf8-605aeabdb677 trigger_policy: unique-policy-id-1-modifyConfig timeout: 60 abatement: false policies: - id: unique-policy-id-1-modifyConfig name: modifyconfig-cds-actor description: actor: CDS recipe: modify-config target: resourceID: 7e4fbe9c-d612-4ec5-bbf8-605aeabdb677 type: VNF payload: artifact_name: vFW-CDS artifact_version: 1.0.0 data: '{"active-streams":"7"}' retry: 0 timeout: 30 success: final_success failure: final_failure failure_timeout: final_failure_timeout failure_retries: final_failure_retries failure_exception: final_failure_exception failure_guard: final_failure_guard
  • REST equivalent of the gRPC request from Policy to CDS to execute a CBA:

curl -X POST \ 'http://{{ip}}:{{port}}/api/v1/execution-service/process' \ -H 'Authorization: Basic Y2NzZGthcHBzOmNjc2RrYXBwcw==' \ -H 'Content-Type: application/json' \ -H 'cache-control: no-cache' \ -d '{ "commonHeader":{ "subRequestId":"14384b21-8224-4055-bb9b-0469397db801", "requestId":"d57709fb-bbec-491d-a2a6-8a25c8097ee8", "originatorId":"POLICY" }, "actionIdentifiers":{ "mode":"sync", "blueprintName":"vFW-CDS", "blueprintVersion":"1.0.0", "actionName":"config-deploy" }, "payload":{ "config-deploy-request":{ "resolution-key":"6128eb53-0eac-4c79-855c-ff56a7b81141", "config-deploy-properties":{ "service-instance.service-instance-id":"40004db6-c51f-45b0-abab-ea4156bae422", "generic-vnf.vnf-id":"8d09e3bd-ae1d-4765-b26e-4a45f568a092", "data":{ "active-streams":"7" } } } } }'

Policy Update Notifications @James Hahn

https://lf-onap.atlassian.net/browse/POLICY-1840 - When policies are deployed/undeployed (eg new version or removal), then some ONAP components need more immediate notification. Notably, DCAE microservices need to respond quickly to policy changes. @James Hahn

When a policy has been pushed (or "unpushed") to all relevant PDPs, a notification is published to the DMaaP topic, POLICY-NOTIFICATION, a sample of which is shown below:

{ "deployed-policies": [ { "policy-type": "onap.policies.monitoring.cdap.tca.hi.lo.app", "policy-type-version": "1.0.0", "policy-id": "onap.scaleout.tca", "policy-version": "2.0.0", "success-count": 3, "failure-count": 0 } ], "undeployed-policies": [ { "policy-type": "onap.policies.firewall", "policy-type-version": "1.0.0", "policy-id": "onap.firewall.tca", "policy-version": "6.0.0", "success-count": 3, "failure-count": 0 } ] }

 

Policy Validation @Liam Fallon

https://lf-onap.atlassian.net/browse/POLICY-2027 - Enhances the validation of TOSCA Policy Types and provides tools to Policy Designers. @Liam Fallon

Better support for TOSCA Data Types, Policy Types and Policies. Currently (Dublin/El Alto), policies are validated by PDPs at load time, and the Policy Framework checks that the policy type of a policy is valid and exists when it is stored over the API

In Frankfurt, we will:

  • Validate that the parent policy types, and referenced data types in policy types and data types exist

  • Return fully dereferenced policy types and data types on get requests

  • Vaidate consistent create and update on policy types and data types

  • Validate references to policy tpyes and data types prior to deletion of policy types and data types

  • Validate versions of policy types and policies

  • Validation of policies against their policy types on create and update of policies (Stretch)

See R6 Data Type, Policy Type, and Policy Referencing and Versioning for more information.

Support for Native Policies  @Chenfei Gao @Ram Krishna Verma @Pamela Dragosh

https://lf-onap.atlassian.net/browse/POLICY-1845 - Some support for this in the legacy components, this work greatly enhances the ability to load PDP native policies that either execute alone or in conjunction with TOSCA Policy Types. Adds in Apex Native Policy Types in addition to XACML, Drools.

  • Designed TOSCA-compliant solution to create/read/update/delete(CRUD), deploy/undeploy and store native PDP policies and their artifacts

  • Designed native PDP policy types off which native policies executable in different PDP engines (i.e. Drools, XACML, APEX) can be created and deployed

  • Build a validator for the API payload provided to create native PDP policies to avoid unnecessary inconsistency between policy artifacts in the nexus and TOSCA policies in the database

  • Build Drools PDP (PDP-D) support for native Drools policy execution

  • Build XACML PDP (PDP-X) support for native XACML policy enforcement

  • Build APEX PDP (PDP-A) support for native APEX policy execution

Native XACML policy type
tosca_definitions_version: tosca_simple_yaml_1_0_0 policy_types: onap.policies.Native: derived_from: tosca.policies.Root description: a base policy type for all native PDP policies version: 1.0.0 onap.policies.native.Xacml: derived_from: onap.policies.Native description: a policy type for native xacml policies version: 1.0.0 properties: policy: type: String required: true description: The XML XACML 3.0 PolicySet or Policy metadata: encoding: URL

 

Native Drools policy type
tosca_definitions_version: tosca_simple_yaml_1_0_0 policy_types: onap.policies.Native: derived_from: tosca.policies.Root description: a base policy type for all native PDP policies version: 1.0.0 onap.policies.native.Drools: derived_from: onap.policies.Native description: a policy type for native drools policies version: 1.0.0 properties: rule_artifact: type: onap.datatypes.native.rule_artifact required: true description: the GAV information of the maven artifact controller: type: onap.datatypes.drools.controller.relation required: true description: the drools controller to which the current native policy is assigned data_types: onap.datatypes.native.rule_artifact: derived_from: tosca.datatypes.Root properties: groupId: type: string required: true description: the groupId of the maven artifact artifactId: type: string required: true description: the artifactId of the maven artifact version: type: string required: true description: the version of the maven artifact onap.datatypes.drools.controller.relation: derived_from: tosca.datatypes.Root properties: controllerPolicyName: type: string required: true description: the name of drools controller policy controllerPolicyVersion: type: string required: true description: the version of drools controller policy

 

Native APEX policy type
tosca_definitions_version: tosca_simple_yaml_1_0_0, policy_types: onap.policies.Native: derived_from: tosca.policies.Root description: a base policy type for all native PDP policies version: 1.0.0 onap.policies.native.Apex: derived_from: onap.policies.Native description: a policy type for native apex policies version: 1.0.0 properties: engine_service: type: onap.datatypes.native.apex.EngineService description: APEX Engine Service Parameters inputs: type: map description: Inputs for handling events coming into the APEX engine entry_schema: type: onap.datatypes.native.apex.EventHandler outputs: type: map description: Outputs for handling events going out of the APEX engine entry_schema: type: onap.datatypes.native.apex.EventHandler environment: type: list description: Envioronmental parameters for the APEX engine entry_schema: type: onap.datatypes.native.apex.Environment data_types: onap.datatypes.native.apex.EngineService: derived_from: tosca.datatypes.Root properties: name: type: string description: Specifies the engine name required: false default: "ApexEngineService" version: type: string description: Specifies the engine version in double dotted format required: false default: "1.0.0" id: type: int description: Specifies the engine id required: true instance_count: type: int description: Specifies the number of engine threads that should be run required: true deployment_port: type: int description: Specifies the port to connect to for engine administration required: false default: 1 policy_model_file_name: type: string description: The name of the file from which to read the APEX policy model required: false default: "" policy_type_impl: type: string description: The policy type implementation from which to read the APEX policy model required: false default: "" periodic_event_period: type: string description: The time interval in milliseconds for the periodic scanning event, 0 means don't scan required: false default: 0 engine: type: onap.datatypes.native.apex.engineservice.Engine description: The parameters for all engines in the APEX engine service required: true onap.datatypes.native.apex.EventHandler: derived_from: tosca.datatypes.Root properties: name: type: string description: Specifies the event handler name, if not specified this is set to the key name required: false carrier_technology: type: onap.datatypes.native.apex.CarrierTechnology description: Specifies the carrier technology of the event handler (such as REST/Web Socket/Kafka) required: true event_protocol: type: onap.datatypes.native.apex.EventProtocol description: Specifies the event protocol of events for the event handler (such as Yaml/JSON/XML/POJO) required: true event_name: type: string description: Specifies the event name for events on this event handler, if not specified, the event name is read from or written to the event being received or sent required: false event_name_filter: type: string description: Specifies a filter as a regular expression, events that do not match the filter are dropped, the default is to let all events through required: false synchronous_mode: type: bool description: Specifies the event handler is syncronous (receive event and send response) required: false default: false synchronous_peer: type: string description: The peer event handler (output for input or input for output) of this event handler in synchronous mode, this parameter is mandatory if the event handler is in synchronous mode required: false default: "" synchronous_timeout: type: int description: The timeout in milliseconds for responses to be issued by APEX torequests, this parameter is mandatory if the event handler is in synchronous mode required: false default: "" requestor_mode: type: bool description: Specifies the event handler is in requestor mode (send event and wait for response mode) required: false default: false requestor_peer: type: string description: The peer event handler (output for input or input for output) of this event handler in requestor mode, this parameter is mandatory if the event handler is in requestor mode required: false default: "" requestor_timeout: type: int description: The timeout in milliseconds for wait for responses to requests, this parameter is mandatory if the event handler is in requestor mode required: false default: "" onap.datatypes.native.apex.CarrierTechnology: derived_from: tosca.datatypes.Root properties: label: type: string description: The label (name) of the carrier technology (such as REST, Kafka, WebSocket) required: true plugin_parameter_class_name: type: string description: The class name of the class that overrides default handling of event input or output for this carrier technology, defaults to the supplied input or output class required: false onap.datatypes.native.apex.EventProtocol: derived_from: tosca.datatypes.Root properties: label: type: string description: The label (name) of the event protocol (such as Yaml, JSON, XML, or POJO) required: true event_protocol_plugin_class: type: string description: The class name of the class that overrides default handling of the event protocol for this carrier technology, defaults to the supplied event protocol class required: false onap.datatypes.native.apex.Environmental: derived_from: tosca.datatypes.Root properties: name: type: string description: The name of the environment variable required: true value: type: string description: The value of the environment variable required: true onap.datatypes.native.apex.engineservice.Engine: derived_from: tosca.datatypes.Root properties: context: type: onap.datatypes.native.apex.engineservice.engine.Context description: The properties for handling context in APEX engines, defaults to using Java maps for context required: false executors: type: map description: The plugins for policy executors used in engines such as javascript, MVEL, Jython required: true entry_schema: description: The plugin class path for this policy executor type: string onap.datatypes.native.apex.engineservice.engine.Context: derived_from: tosca.datatypes.Root properties: distributor: type: onap.datatypes.native.apex.Plugin description: The plugin to be used for distributing context between APEX PDPs at runtime required: false schemas: type: map description: The plugins for context schemas available in APEX PDPs such as Java and Avro required: false entry_schema: type: onap.datatypes.native.apex.Plugin locking: type: onap.datatypes.native.apex.plugin description: The plugin to be used for locking context in and between APEX PDPs at runtime required: false persistence: type: onap.datatypes.native.apex.Plugin description: The plugin to be used for persisting context for APEX PDPs at runtime required: false onap.datatypes.native.apex.Plugin: derived_from: tosca.datatypes.Root properties: name: type: string description: The name of the executor such as Javascript, Jython or MVEL required: true plugin_class_name: type: string description: The class path of the plugin class for this executor

 

Optimization and Naming Policies - Better Use of the Decision API @Pamela Dragosh 

https://lf-onap.atlassian.net/browse/POLICY-2067 - In working with the OOF team, we are moving some of the decision making that the legacy engine could not support into the new xacml PDP engine which supports a Decision API. Previously, OOF had to make 2 separate API calls. The addition of "matchable" attributes in the Optimization and Naming policies allow the XACML PDP to dynamically translate TOSCA policies to XACML for more fine-grained policy Decisions.

The OOF optimize uses a "best matches" algorithm on top of the XACML engine fine-grained decision in order to further refine the decision to return only the "best matching" policy given the attributes in the Decision API call.

Base Optimization Policies - including "matchable"
tosca_definitions_version: tosca_simple_yaml_1_0_0 policy_types: onap.policies.Optimization: derived_from: tosca.policies.Root version: 1.0.0 description: The base policy type for all policies that govern optimization properties: scope: description: Scope for the policy - could be for a specific release. type: list metadata: matchable: true required: true entry_schema: type: string geography: description: One or more geographic regions type: list metadata: matchable: true required: true entry_schema: type: string constraints: - valid_values: - US - International identity: description: Used internally for identification type: string required: true

The OOF optimization Decision includes the ability to ask the decision to take in "context" when returning policy decisions. Eliminates one less call the OOF project had to make to the Policy Decision API to retrieve policies during runtime.

Example Decision API call Payload
{ "ONAPName": "OOF", "ONAPComponent": "OOF-component", "ONAPInstance": "OOF-component-instance", "context" : { "subscriberName": [] }, "action": "optimize", "resource": { "scope": [], "services": [], "resources": [], "geography": [] } }

 

TOSCA Compliant Operational and Guard Policies @Pamela Dragosh 

https://lf-onap.atlassian.net/browse/POLICY-2118 - Finishing the work started in Dublin to fully move ALL the Control Loop Policy Types into TOSCA compliance with CLAMP team. Previously our operational and guard policies used a YAML format defined a few years ago. In this effort we are translating it to be fully compliant with TOSCA and cleaning up some of the syntax.

Operational Policy Type
tosca_definitions_version: tosca_simple_yaml_1_0_0 policy_types: onap.policies.controlloop.operational.Common: derived_from: tosca.policies.Root version: 1.0.0 description: Operational Policy for Control Loop execution properties: id: type: String description: The unique control loop id. required: true timeout: type: Integer description: | Overall timeout for executing all the operations. This timeout should equal or exceed the total timeout for each operation listed. required: true abatement: type: Boolean description: Whether an abatement event message will be expected for the control loop from DCAE. required: true default: false trigger: type: String description: Initial operation to execute upon receiving an Onset event message for the Control Loop. required: true operations: type: List description: List of operations to be performed when Control Loop is triggered. required: true entry_schema: type: onap.datatype.controlloop.Operation onap.policies.controlloop.operational.common.Apex: derived_from: onap.policies.controlloop.operational.Common type_version: 1.0.0 version: 1.0.0 description: Operational policies for Apex PDP properties: onap.policies.controlloop.operational.common.Drools: derived_from: onap.policies.controlloop.operational.Common type_version: 1.0.0 version: 1.0.0 description: Operational policies for Drools PDP properties: controllerName: type: String description: Drools controller properties required: false data_types: # TBD if this is needed onap.datatype.controlloop.operation.Failure: derived_from: tosca.datatypes.Root description: Captures information of an operational failure performed for control loop properties: messages: type: String description: error message required: true category: type: String description: | The category the error occurred in. Whether this is a general error from the actor, or the operation timed out, retries were exhausted in trying to execute the operation, a guard policy prevented the operation from occuring, or an exception in the system caused the failure. constraints: - valid_values: [error, timeout, retries, guard, exception] onap.datatype.controlloop.Target: derived_from: tosca.datatypes.Root description: Definition for a entity in A&AI to perform a control loop operation on properties: targetType: type: String description: Category for the target type required: true constraints: - valid_values: [VNF, VM, VFMODULE, PNF] entityIds: type: Map description: | Map of values that identify the resource. If none are provided, it is assumed that the entity that generated the ONSET event will be the target. required: false onap.datatype.controlloop.Actor: derived_from: tosca.datatypes.Root description: An actor/operation/target definition properties: actor: type: String description: The actor performing the operation. required: true operation: type: String description: The operation the actor is performing. required: true target: type: String description: The resource the operation should be performed on. required: true metadata: clamp_possible_values: <string:see clamp project for syntax> payload: type: Map description: Name/value pairs of payload information passed by Policy to the actor required: false entry_schema: type: String onap.datatype.controlloop.Operation: derived_from: tosca.datatypes.Root description: An operation supported by an actor properties: id: type: String description: Unique identifier for the operation required: true description: type: String description: A user-friendly description of the intent for the operation required: false operation: type: onap.datatype.controlloop.Actor description: The definition of the operation to be performed. required: true metadata: clamp_possible_values: <string:see clamp project for syntax> timeout: type: Integer description: The amount of time for the actor to perform the operation. required: true retries: type: Integer description: The number of retries the actor should attempt to perform the operation. required: true default: 0 success: type: String description: Points to the operation to invoke on success. A value of "final_success" indicates and end to the operation. required: false default: final_success failure: type: String description: Points to the operation to invoke on Actor operation failure. required: false default: final_failure failure_timeout: type: String description: Points to the operation to invoke when the time out for the operation occurs. required: false default: final_failure_timeout failure_retries: type: String description: Points to the operation to invoke when the current operation has exceeded its max retries. required: false default: final_failure_retries failure_exception: type: String description: Points to the operation to invoke when the current operation causes an exception. required: false default: final_failure_exception failure_guard: type: String description: Points to the operation to invoke when the current operation is blocked due to guard policy enforcement. required: false default: final_failure_guard

Much discussion for this was done in the Control Loop sub committee meetings. As captured here: Control Loop Policy Type Design Proposal#1: Simple upgrade of legacy policy types to TOSCA

Policy API Enhancements @Chenfei Gao

https://lf-onap.atlassian.net/browse/POLICY-2028 - Finished work such as added "application/yaml" as Content-Type.

  • Added the support of "application/yaml" as Content-Type in policy lifecycle API

  • Created a new simplified API for creating multiple policies in one shot

  • Made policy types preloading configurable in the helm charts

  • Added new version validation for POST API payload to constrain clients to carefully specify a reasonable version for policy or policy type

  • Preload default policies for ONAP components

 

PAP ↔ PDP Communication @Ram Krishna Verma @James Hahn

https://lf-onap.atlassian.net/browse/POLICY-2026 - Clarified PAP ↔ communication of Policy Types, Health Status, PDP Group Details, Passive vs Active modes. @Ram Krishna Verma

  • Mapping of kubernetes deployment to PDP Subgroup

  • Re-synch of PAP and PDP when the supported policy type changes

  • Change the PAP Group Deploy/Undeploy API to only create/update PdpGroups

  • Create new API to deploy/undeploy policies into PdpGroups

  • Handle derivation in PDP supported types

 

RECORDING

Recording