Controller Design Studio-- F2F August 26th to 28th 2019

Prerequisite

https://lf-onap.atlassian.net/wiki/display/DW/Controller+Design+Studio+%5BCDS%5D+Documentation

Face 2 Face Controller Design Studio Agenda


  • The ONAP CDS developer communities met at Bell Canada facility in Montreal Canada. 

 Register for the Event

Contact Information:

Name

Email

Session Virtual/In-Person

ali.daher@bell.caIn-Person

In-Person

In-Person

In-Person

In-Person

In-Person

In-Person

In-Person

In-Person




Yuriy.Malakov@att.comIn-Person

In-Person
Eric Multanen
In-Person
Adil Mir
In-Person

In-Person

In-Person
Kirill Lukashin (Deactivated)
Virtual

Agenda for the week of August 26th 2019


NOTE: Monday and Tuesday session will happen in Downtown Montreal for ease of travel to most of those joining out of town. Wednesday session will take place at Nuns Island to allow a go and see to the Bell campus where the ONAP squads are located.  

Monday 


Tuesday 


Wednesday 


NEWTON ROOM 8th floor- 671, rue de la Gauchetière O. MONTREAL PQNEWTON ROOM 8th floor- 671, rue de la Gauchetière O. MONTREAL PQRoom E1.2 1, carrefour Alexander-Graham-Bell VERDUN - CAMPUS MONTREAL PQ, H3E3B3,
09:00AM

Welcome, Introduction, logistics, overview of Agenda and CDS roadmap.  Adjust Agenda if needed 


No Recording

Dis-aggregation of CDS Resource Resolution from Blueprintprocessor


Link to recording


Go and See of Bell ONAP squads 



09:30AM

Designer screen for CBA package + feedback from Bell Network Programmability team carrie.lau

No Recording 

10:00AM
10:30AM Break Break Break
10:45AM

CDS Test Strategy

Align test coverage with CDS features upgrades

Add CDS Health Check for Robot

Add a black box test case for CDS via Robot

Add a use case test case for CDS via Robot

UAT test 


No Recording


CDS: Queue Mechanism/Priority Execution


Link to recording

Resource LCM  Resolution

VNF LCM Resolution, SO and CDS tighter integration


No Recording 


11:30AM
12:00PMLunch

Lunch


 Lunch

 

12:30PM
1:00PM

AAF Integration with Blueprint processor /Controller blueprint

Refactor Logging in CDS components to integrate full error management (error handler and error code)


Link to recording


CDS. Atomic Transaction for Multi Device level connection 


Link to recording

CDS capabilities in Frankfurt to meet 5G requirements


Requirement link


1:30PM

2:00PM
2:30PM
3:00PMBreak Break Break 
3:30PM

CDS security and gRPC TLS/Authentication 



(Buffer: DSL topic, Consolidate Blueprint processor and controller blueprints MS) 


Link to recording


CDS Mediation Layer: Store the configlet content that is pushed to network via Ansible/Python in a database as a service. Retrieve configlet content form Ansible/Chef etc.. 


Link to recording



CDS Policy and roadmap. 



Requirement link

4:00PM

4:30PM
5:00PMGeneration of the OPEN-API google protobuff specification for CBA Packages. 

CDS and MultiCloud Integration


Link to recording

5:30PM
6:30-8PMTeam Dinner: Moxies, 1207 Boulevard Robert-Bourassa, Montréal, QC H3B 0C3Retrospective around the workshop (check and adjust) + Team Dinner 

Retrospective around the workshop

Meeting Minutes: 

 8/26-CDS-Meeting Minutes

Meeting Minutes:

  1. gRPC CDS Integration [primary priority]
    • Action Item:
      • Yuriy to create a US for Frankfurt to move all the CDS UI API leveraging the blueprint processor for enrich , save, publish, download
    • NOTE: Work remaining for the integration of the controller blueprints capabilities in blueprint processor
    • In Dublin release gRPC not supported by Controller blueprints
    • Frankfurt release there is an effort to combine the controller blueprints with blueprint processor. Blueprint processors support gRPC endpoints.
  2. Event Based Message integration with CDS
    • CDS à Kakfa/Dmaap
    • Kafka/Dmaap à CDS
    • Action Item:
      • Prat to provide a the information for show casing this capability  in ONAP.
      • Brinda to work with Prat on the improvement to the kafka implementation. 
    •  Use Case need to be show cases in ONAP. Bell C has up streamed the Kafka and Dmaap support in ONAP with CDS integration.
  3. Health Check Strategy
    • Action Item:
      • Yuriy to work with Abdel to support the health check integration with new health check API
      • Alexis to provide guidance on the standardize on the health check implementation for all the endpoint and provide the output via a single api. Janani??
    • The current health check API only checks the self service API. In Frankfurt release, the proposal is to expose endpoint that will be able to do all the health check api for all the cds design time and run time functions.
  4. Testing Strategy: 
    • 1. Verify Job integration the sonar check to prevent new code from being submitted  --> refining the JJ-Job --to define the Jenkins job builder. Prototype....
      • Yuriy to setup a call with Jessica include Ebo, Alexis, Dan T, Prat
      • Ebo to look into the tool to help provide a clear view on the health of the project before integration.
      • Ebo add the capability for testing UAT Test as part of the run time. (Mock vs real ENV). combination of the run time output and Mock expectation output using the run time results. --SELF SERVICE CSIT for enriched CBA as part of run time execution.
      • Yuriy to work with Ebo to extended the UAT Test file to vLB and vFW use case.
      • Ebo to extended the declarative testing to enrichment validation for CBA Package.
        • No Enrich Blueprint > Validate Enrichment process;
      • Arundathi and Ebo to enable the Testing folder to allow a list of UAT files for verifying list of actions.
        • Multiple file for UAT--is expecting --> we should have a pattern for each of the UAT and actions.
      • NOTE:
        • Automatically create serverless function --running a build --Docker Image deploy > Kick off testing and returning test results.
        • Providing mock information FOR EXTERNAL SYSTEM. Mock Packages.
        • Technology YAML > YAML error prone?  DSL Domain Specific language. Brinda recommendation
        • General Comment from Dan T: No permission to disable false positive discovery by sonar. --talk to Jessica
        • Move integration test to unit test
        • integration test –
        • Unit Test --- complimentary to acceptance test
        • Acceptance test --- ete test with external testing.
        • UAT Testing of the CBA critical to verify the backwards compatibility and introduction of new features.
        • Select the ENV to run---
      • Linux foundation support might be needed...to have sonar analysis the new code coverage
      • Action Item:
    • Test Coverage -->
    • ideas   
  5. Heat Bridge
    • Munir to contribute the heat bridge so implementation for create A&AI object which include the pserver object creation.
    • Munir  contribute the heat bridge so implementation to delete the A&AI object as part of the delete flow.
    • Action Items:
      • Munir to contribute the fix. 


  1. CDS & AAF Integration – 
    • CBA support the notion of adding new action which needs to be correlated to the list of users that might be able to executed as part of the design time activity. 
    • Brinda to provide update to Mike on the AAF/Service Meshing integration. Brinda to share AAF documentation for CBA Package integration notion.
    • Yuriy to add Mike to the thread for AAF Integration
    • Yuriy to setup a call with Ajay S on the call NPM Implementation of user role management via AAF REST API layer.
    • Yuriy to add a roadmap item for CDS UI and AAF Integration for User Role Management –ADMIN User needed??? Placeholder
    • Henry to add a flag to turn off aaf
      • If aaf is disable then http is supported.
    • NOTE:
      • New Servlet version à Intercept concept is supported with version 3.0.
      • Bell Canada is leading effort for service mesh/ISTO
      • Need to be bound to AAF, Service Meshing and ISTO
      • Mike à ISTO/Service Meshing support à What is possible Frankfurt ???
      • Endpoint role management à Web flux (non-block integration) ???
      • Service mesh and ISTO
    • Namespace Creation and User Role Management is managed with AAF
    • CBA Package should include reference to the AAF Namespace for role and users that are able to execution the CDS action within the CBA context. Each CBA might have a unique or shareable namespace. This implementation should be implemented via REST Layer.
    • AAF Integration with CBA for G release or H release. 
    • Action Items:


  1. Platform CDS Logging, Tracing, and Error Modeling
    • Error Code to Error message mapping within a component
    • All Error code and messages need to be document with hints of common remediation/resolution
    • Aggregated all the errors and make sure if an error occurs its tied to error code.
    • Platform Error à  Derived from NodeTypes.
    • Parallel process request error à capture multiple error and aggregate back to the users.
    • Ebo provide link to à
    • Event based logging needed for CDS Platform. Logging message event. (Request to maintain all the chain) (Diary of events) Request , Blueprint name, version.
    • Application performance monitoring à Tracing tracking the run time actions within the CBA. 
    • Benefits: what are the benefits.
    • ELK à provides but not opensource.
    • Alexis looking at alternative.  à Zipkin is opensource. (Elizar provide this as well)
    • Resource à Required ???     
    • Error Modeling  à —  Design review for the platform and CBA (component). Yuriy to setup a review with Brinda , Munir, Serge, Alexis etc..
    • Logging:  logger wrapper? Brinda has added a notion in the latest code.  Yuriy to setup a design review with Brinda  Munir, Serge,, Alexis etc..
    • Tracing à – Zipkin Alexis poc


  1. CBA Specification Generation [Janani??]
    • As part of the enrichment process generate a spec using a template engine.
      • Create a folder to store
      • Template Engine for a Version
        • OPEN-API
        • Google Protobuf (gRPC Client)
        • TOSCA à
        • Any model
      • Which version to generate à
        • Action Item: Rashmi to bring this topic up with CLAMP UI Team.
          • Options: gRPC and OPEN-API
        • CLAMP UI support TOSCA rendering???
      • NOTE:
      • Spec generation
      • Yuriy and Rashmi shall follow up with clamp team and the expectations. 


  1. Frankfurt Controller Blueprints implementation consolidation into controller blueprint processor.
    • Yuriy to create placeholder US to cover the below UI Scope planned for Frankfurt release:
      • Serverless function create an dynamically image and deploy with native k8s API. [Roadmap Item]
      • Spin up new blueprint processor only in testing environment and not in production is expected.
      • Serverless function allows you to build image on the fly.
      • Assumption: Production will have a stable set of containers/pod.
      • CDS UI to call the CDS Blueprints designer apià     
      • CDS UI to discover the blueprint process instance à Configuration management  [roadmap.  item]
      • CDS UI to spin up run time.  [Roadmap item]
      • Remove t
      • NOTE:
    • Action Item:
      • G release - Yuriy to create a user story to for OOM to remove the controller blueprints ms. 
        • Abdel 


    • Function Module outside of the blueprint processor from code perspective --
      • Follow up required with community. 


  • blueprints management -- Pending work items
    • Combine the two table for where we store the design time and runtime…. Frankfurt Release à
      • One instance of blueprint processor to manage the package for all CBA às in a file structure Master  [Frankfurt Roadmap]
      • One instance of blueprint processor to manage the database based on the property–bootable property . 
        • If you have multiple instance the none db instance wont be managing the db connection as part of the boot up. 
    • Design time & Run time validation  is standalone  today but need to combine validation for design time and run time.
    • Need a flag to retrieve the CDS Blueprint cba package from a file path [mounted to a file path ]
    • Action Item:
      • Brinda is finalizing the implementation for following items:  
      • Janani to help in this area as well. 


  • DSL Concept à Multiple CBA file
    • DSL and JSON file should keep constant reference for cds_model_name and cds_model_version.
    • As part of the distribution the onap component should extract the cds model name and version from a constant sdc tosca service.
    • Keep the constant name from SDC, SO, and CDS.
    • Work with Nokia on this implementation.
    • SDC and CDS Integration à Frankfurt


    • Action Item:
      • CDS UI should enforce the populate using the tosca meta file.
      • For CBA Package reference based on the tosca meta data.
      • Brinda to change template_model and template version to
        • CBA_MODEL_NAME and CBA_MODEL_VERSION. --design on a name. SO, SDC, Policy, CLAMP
        • Model-Name and Model-Version in CDS tosca. 
      • Yuriy to create requirement for UI team:
      • Alexis to support Migration of existing CBA in Bell from json blueprints tosca meta data to tosca file meta section for frankfurt.


  1. Other TOPICS:
  • Enable grpc and rest both? Brinda and Alexis ? Is this possible. Proxy Port.
  • Netty Listerner Proxy
  • One Server with two instances???one for gRPC and one for REST?? Challenge ? Benefits
  • Kafka  ? Can we remove spring layer?
    •  Alexis to check the idea.
    • Action Item: Brinda to provide doc to Alexis.


  1. MultiCloud/VIM and CDS Integration  (Eric/Lukasz)
    1. Instantiation Alignment
      1. agreement is that the VM and CNF Building Block orchestration should be common with the only difference is that SO as part of Create Building Block shall call trigger MultiCloud/Vim adapter for helm creation for the pod. 
    2. LCM Action
      1. CDS shall call the multiCloud/VIm Openstack Keystone v3 API or k8S based API for LCM Cloud actions. 
 8/27-CDS-Meeting Minutes

Meeting Minutes:

  1. Disaggregation à resource resolution ms from blueprint processor  (Primary --Alexis)
    • Resource resolution engine
    • Template engine generation
    • Decouple the Database management for persisting the resource resolution. [nice to have]
      • Database is outside of the mS For resource resolution. (mariagaleriadb)
    • Resource Resolution Context Key à
      • The resource tables shall be extended to store the resource keys  and configlet id
      • Alexis to make the change: Default to true for storing the configlet generation in the tables;
      • Resource resolution Component  need to be extended to support unassignment based on the data dictionary capabilities for unassign.
    • Alexis to provide POC implementation and provide feedback.
    • Revisit Brinda idea of device modeling using json path.
    • Resource Resolution mS shall be created
    • Self Service API shall forward the call to resource resolution mS
    • Action Item:


  1. Atomic Transaction—
    • Each Transaction  support rollback /Failure
    • Rollback include previously successful configlet download
    • All Device are lock until all the action are implemented.
    • Atomic Transaction should be supported in parallel; 
    • For a given action within support configlet generation  and download for multiple


    • Alexis Solution:
      • Commit confirm capability supported
        • Commit fails on 3rd.
        • Commit Confirm success
        • Commit works for the first 2
        •  
        • Action
          • Support the capability of Commit Id Rollback (also can be supported but the team felt that disc is more likely—this implementation can be modeled.
          • Yuriy to create a US for testing the rollback activities as a roadmap item.
          • Dan T to provide a WIP patch of Om implementation of atomic transaction for Alexis to review.
          • General premise is to support a disconnect flow for rollback use case.
      • NetConf Protocol
      • CDS Platform capability for rollback is supported via imperative workflow and/or DG CDS failed action.



  1. Resource Resolution for Ansible/Pyton: (read me on the flow to be provided by Alexis and functionality is there in onap). 
    • Action Item:
      • Yuriy to create a placeholder US for supporting the capability for persisting resolved resource using Ansible/Python script. The Ansible/Pything script shall call the resource resolution API to store the resource resolve context. Notion is that CDS is the source of truth for all resolved resource that is pushed to the network.
      • Alexis to upstream it.


 

  1. Building Block Orchestration using CDS Resource Assignment:
    • Assign:
      • have SO retrieve the CDS context using the A&AI self-link.
        • Sub-Option 1 
          • Retrieve the Resolved name/value pair context
        • Sub-Option 2 (Preferred)
          • Retrieve the Cloud Meshed Context 
        • Sub-Option 1 & 2
          • CDS will provide right link for retrieving the correct sub-option. 
      • have CDS Create either MD-SAL structure for SO to extract context as part of create
      • Option 1:  (preferred)
      • Option2:
    • Bell , AT&T and other is looking a way to eliminate the resource accumulator template and simplify the resource resolution for cloud orchestration.
    • The approach is to use the directly the CDS Building block for Service, VNF, VF Module                Assign and Unassign


        • Option 1 & 2:
          • Partial assignment should be supported. Meaning that if the capability is already resolved and it has a value instance then it resource-resolution component shouldnt call the capability component for resolution.
            • Can be model in CDS similar to the current DG implementation.


      • Unassign/Delete
        • Have CDS unassign the capability using the unassign capability reference in the data dictionary.


  1. SDNC Helm Chart Update
    • Update CDS chart to be standalone.
      • Yuriy to create US for Abdel.
      • Default CDS Chart to be Y
      • Action Item:



  1. MultiCloud , SO and CDS Integration
    • Leverage the SO, SDNC, and CDS Building
    • Multicloud to support the onboarding of the values.yaml via SDC
    • Override values.yaml file will be a template
    • CDS shall resolved resource in name value pair and SDNC shall persist in mD_SAL
    • SO shall retrieve the SDNC context and pass it to MultiCloud
    • MultiCloud shall mesh the override .yaml template with sdnc context and trigger HELM.
    • Extended the CDS Integration to LCM Action and Helm Creation/Deletion.
    • Action Item:
      • Eric/Lukasz to help with the integration strategy for Frankfurt release.
 8/28 CDS-Meeting Minutes

Meeting Minutes:

5G Use Cases:
https://lf-onap.atlassian.net/wiki/display/DW/5G+Use+Cases+in+R6+Frankfurt


SO and CDS Integration comment---:

  • SO to CDS need to make sure we pass the namespace/endpoint.
  • Extend Marco Building block flows to existing flow architecture with the hooks of scope and action. 
  • Discuss Strategy towards SO Tosca Driven orchestration and integration with CDS (long strategy --Frankfurt)
    • Benjamin M, Steve S, Seshu S, Alexis , Dan T , Damian, Henry, Mac, Munir, Eric M , Yuriy , Shawn


Policy and CDS Integration: 

  • # 1. Alignment between on the AAI notation used in policy (node.attribute) and CDS (attribute) - AAI properties can be extracted out of config-deploy-properties into its own aai-properties as a workflow input under config-deploy-request 
  • # 2. CLAMP rendering possible actions supported by CDS + rendering of the input fields from CBA - Open API generated from the protobuff - Ingest the TOSCA yaml from CDS
  • # 3. SO response back to CDS for selected params in order to store them: e.g. in cases where VNF management IP is not a static IP. - Use heatbridge information
  • # 4. Distributed CDS or for multi-tenancy: placeholder for the name of CDS or an identifier to resolve a specific CDS instance (namespace) # 5. Spec for SLA for an end-to-end closeloop automation, existing data??



CDS UI Feedback

  • Node Type --> Should have Tags/Labels that the UI should express as a function.
    • Node Type rename to Function.
  • Workflow > Action
    • Do we need a MODEL of a workflow or should the UI implement the tosca model with the output that CDS uses? Verify with Brinda/Alexis.
    • Is there a way to hide the get_attribute and reuse the model from the modeling-->

   4   "attributes": {

   5     "assignment-params": {

   6       "required": true,

   7       "type": "string"

   8     }



  • Template Artifact --> Do we have a plugin for velocity, Jinga, XML, Json,
  • Future --> Integration accessing the XML content from DG builder.
  • DG Builder support XML should be converted to json--> Dan T



  • Run Time Experience -->
    • Add a new UI for the Configlet and resource resolution [Nirvan].
      • Support life cycle management of parameter resolution using this ui.
    • Sandbox --> ADD A NEW REQUIREMENT; (Samsaung)
    • Who is the users from operation--> Configlet history tied to the users? Audit trial.
    • CDS UI Diary of the execution logs for a given resources and the users/systems that executed for the CBA Package.


  • Reconciliation à Support required (roadmap)





Retrospective: 

What did we do well?

  • Very informative session 
  • Good to see everyone in person
  • UAT test framework looks promising 
  • DSL was a good discussion 
  • Good overall vision of the product and features under our radar
  • Good introduction to CDS made by Alexis 
  • Lunch catering and time keeping was well done 
  • Webex recording and Meeting minutes are well done
  • Finalized Frankfurt CDS Architecture Evolution captured below.

What should we have done better?

  • Lack of prep work to get everyone at same technical level
  • Many terminologies being used with lack of understanding by some attendees
  • More people should ask questions and challenge 
  • Need to add more focus on helping user adoption 
  • Bring onsite those who joined remote. 

CDS Architecture Evolution from Dublin to Frankfurt

 Frankfurt

 Dublin

Montreal onsite Logistics

Monday and Tuesday: 

https://www.google.com/maps/place/671+Rue+de+la+Gauchetière+O,+Montréal,+QC+H3B+2M8/@45.5014373,-73.5675914,17z/data=!4m13!1m7!3m6!1s0x4cc91a5b4471d7b7:0x88612655a88e21d0!2s671+Rue+de+la+Gauchetière+O,+Montréal,+QC+H3B+2M8!3b1!8m2!3d45.5014373!4d-73.5654027!3m4!1s0x4cc91a5b4471d7b7:0x88612655a88e21d0!8m2!3d45.5014373!4d-73.5654027

Wednesday: 

https://www.google.com/maps/place/Campus+Bell+(de+la+Pointe-Nord%2FJ.-Le+Ber)/@45.4723884,-73.6100177,12z/data=!4m16!1m10!4m9!1m1!4e2!1m6!1m2!1s0x4cc9100a7e06f11f:0x3f23d9830b59c387!2sbell+campus+nuns!2m2!1d-73.5399779!2d45.4724098!3m4!1s0x4cc9100a7e06f11f:0x3f23d9830b59c387!8m2!3d45.4724098!4d-73.5399779