Configuration Persistence Service Jakarta Meeting Notes & Recordings

Meeting Info

When: Fridays (weekly) 13:00-14:00 Irish Standard Time

Where: https://zoom.us/j/836561560?pwd=TTZNcFhXTWYxMmZ4SlgzcVZZQXluUT09

OrganizerToine Siebelink

Recurring Agenda Items

Software Status

Minutes continued in Configuration Persistence Service Kohn Meeting Notes & Recordings

For minutes before Jakarta please go to Configuration Persistence Service Meeting Notes & Recordings 2021

Minutes

DateAttendeesAgenda / Notes

 

  • Software Status
    • CPS-322 - Getting issue details... STATUS validation only required in Service-Layer. REst layer wil be handled using exception handling. Persistence Layer will depend on validation depend beforehand.
    • CPS-829 - Getting issue details... STATUS Priyank Maheshwari and Sourabh Sourabh are working together on this and with help of Former user (Deleted) they can now test using Kafka docker image approach which is good enough for now. Any (future) Strimzi Kafka integration is Integration Team responsibility (Fiachra Corcoran) and onto for CPS team.

 

  • Software Status
    • Lee Anjella Macabuhay  is on holidays next week and needs to wrap up  https://gerrit.onap.org/r/c/cps/+/128013 can merge with comments but not closing user story so someone else can take over next week,
    • Impact of CPS-961 - Getting issue details... STATUS agreed that xPath stored in DB is for 'internal use' only and not exposed to end-user. We can normalize storage for better compatibility and adherence to xpath conventions when retrieving it. Lathishbabu Ganesan will work out prosed solution and do some PoC for this.
    • CPS-962 - Getting issue details... STATUS has been split of from CPS-960 will be handed over from Joseph Keenan to renu kumari when she has time (can be paused for now)
  • Propose Joseph Keenan to become committer for CPS-Core and DMI-Plugin
  • A.O.B.
    • Luke Gleeson (Unlicensed)  presented walkthrough and demo re.  CPS-911 - Getting issue details... STATUS
    • Start of DST in Europe this weekend! Please check your local time for that start of this meeting next week

 

  • Software Status
    •   CPS-928 - Getting issue details... STATUS not blocking,  can be closed. Functionality wil be updated as part of  CPS-896 - Getting issue details... STATUS
    • Sourabh Sourabh  will demo CPS-828 - Getting issue details... STATUS
      • invalid topic should return 4xx as per A/C. Sourabh wil update
    • Lathishbabu Ganesanwill demo findings on CPS-340 - Getting issue details... STATUS
      • System works as expected but Lathish will propose some documentation updates before closing

A.O.B.

 

 

 

Cancelled 

Cancelled 

 

 

 

  • Sourabh Sourabh will demo CPS-855 - Getting issue details... STATUS
    • The logging demonstrated is only to be turned on for specific functional tets to determine time spent during functional execution. Load test wil be affected too much by this.
    • Logging (configuration) guidelines will be documented as part of  CPS-604 - Getting issue details... STATUS
    • The logging introduced here probably needs re-start of service as it is enable using an annotation. We'll await client-feedback to see if this suffices.
  • Lee Anjella Macabuhay  will demo (initial findings) CPS-802 - Getting issue details... STATUS
    • 'restconf' responses are somewhat smaller then equivalent 'rests' responses. This needs some further investigation. CSIT test using the new version of SDN-C will determine if there are significant issues but this is not expected
    • Some small changes were seen comparing legacy (SDNC 1.8) output with current output on the 'rests' interface. But these could be related to small model and data instance updates that have been part of the new SDN-C Deployment
  • Follow up E2E Slicing use-case impacts with Niranjana Y (now limited to  CPS-461 - Getting issue details... STATUS )
    • Niranjana has prepared CPS-TBDMT Output transformation and existing template update. Required change are well understood but are dependent on CPS delivering CPS-865 - Getting issue details... STATUS
    • Niranjana point out an 'additional prefix ' in the output from SDN-C (using NCMP passthrough). CPS-870 - Getting issue details... STATUS has been created to track and address this issue
  • ONAP Best practice related work items: Jakarta Impact View per Component
    • CPS-869 - Getting issue details... STATUS has been created to address logging details adherence to ONAP requirements

 

 

 

Toine Siebelink Peter Turcsanyi (Ericsson) Lee Anjella Macabuhay Tony Finnerty Joe K Dylan Byrne Gergely Molnar (Ericsson) Csaba Eder (Ericsson) Balazs Papp (Ericsson) kieran mccarthy  Rafael Rocha (Ericsson)  Priyank Maheshwari Aditya Puthuparambil Sourabh Sourabh Michael Pruss Former user (Deleted) Jamil Fares (Dell) Kunal Patel renu kumari 

Happy New Year!

  • Discuss Dell requirement for XML Payload with Michael Pruss 
    • Interface overview: https://docs.onap.org/projects/onap-cps/en/latest/architecture.html 
    • For 'passtrough' (bypassing cache)  CPS-E-05 can already support XML (using JSON wrapping)
    • Cache not yet enabled in NCMP (planned for Jakarta). CPS-Core can be used to store any data and 'cache' it but currently only support json payload for data
    • Dell would like CPS to also support XML payload on CPS-E-05,  CPS-E-02 and CPS-E-03 interfaces (probably does not apply to CPS-E-01 (TBC)
      • CPS-E-02 could easily be made to 'accept' XML format too 
      • CPS-E-03 would required an additional parameter? to request XML output
      • DMI interface might be affect to, needs further investigation
    • DB storage is de-coupled from REST input format (interfaces are using java map objects instead) Postgress solution uses json format internally but this is not related to/affected by payload format
    • Toine Siebelink  and Michael Pruss will schedule a follow up meeting next week
  • Present findings on Horizontal Scaling by Lee Anjella Macabuhay  
    • Conclusion; CPS-NCMP can scale horizontally without code changes
    • Lee Anjella Macabuhay  will add some information on how to increase the DB connection pool size for 'vertical scaling of a singel instance. as it was observed the pool size became a bottleneck when more then 5 simultaneous requests were made.
    • as part of discussion future possibilities for domain based (manual) setup of groups of CPS instances were discussed. Sharding could be used to distribute cm-handles.
    • kieran mccarthy  confirm spike goal was achieved and wil contact Peter Turcsany for follow up load testing. 
  • Software Status 
    • nothing of special interest

 

No meeting, New Year's Eve

 

No meeting, Christmas Eve

 

 

Toine Siebelink Aditya Puthuparambil Lee Anjella Macabuhay Dylan Byrne Luke Gleeson (Unlicensed) Tony Finnerty Joe K Former user (Deleted)
  • Software Status
  • Discussion of CSIT CI Issue for parallel changes
  • Team agreed to NOT submit (backward incompatible) DMI-plugin updates until CPS-NCMP changes are approved too. This mean CSIT test can only be ran successful locally until the first merge.


 

  • Software Status
  • Demo of Passthrough write use-case Dylan Byrne 
    • registration sometimes returns 500 even though is it successful. Dylan Byrne  will create bug
  • Demo of Delete Dataspace Lee Anjella Macabuhay 
  • Discuss (drop) release strategy and impacts on CSIT tests
    • remove --latest'
    • verify jobs do NOT push to nexus
    • will depend on 'merge' of depend (dmi-plugin) repo
    • remove DMI-VERSION property to avoid double maintenance (in docker compose for CSIT)
  • AOB: need Jira to describe test-setup using CSIT  scripts (incl. tips for WSL users): CPS-796 - Getting issue details... STATUS  

 

 

  • Software Status
  • CSIT Test failing, https://jenkins.onap.org/job/cps-review-verification-maven-master/948/
    • Same failure seen in a few jobs now 
    • cps-and-ncmp is restating before test: "Restarting (1) 23 seconds ago"
    • 1 test passes (verify modules which ONYL uses DMI plugin, all other tests depend on cps-ncmp service, which seems down)
    • Readiness probe port 8887 (cps-ncmp) start failing just after "Node mounted in 0"
    • Same test pass locally.
    • Theory: Mounting node starves cps-ncmp of resources, kubernetes eventually thinks it is dead and restarts it. Troubleshooting steps:
      • run locally
      • verify resource usage (htop) when logs report "Node mounted in 0", try to determine if the is a process that hogs all resources.


 

N/A

Meeting cancelled due to Zoom login issue

 

 

  • Software Status
  • Delete List / List Element / DataNode discussion with Dylan Byrne 
    • Team agreed to use the same refactored code to delete List(element) and/or Container. For backward compatibility the rest endpoint that deletes lists (element) only it will thrown an exception if the target is a normal container 
  • Code Walkthrough https://gerrit.onap.org/r/c/cps/ncmp-dmi-plugin/+/125284 by renu kumari 
  • Change of Meeting time.
    • For European participants this meeting weel be 1 hour earlier local time.

 

  • JSON Parser Choice by Luke Gleeson (Unlicensed) 
    • renu kumari pointed out that larger file used for Yang Data (but less frequent) Luke will add to study
    • Team confirmed preference for Jackson parser mainly because of GSON numeric issue and performance impact not significant enough to mix both throughout our code.
  • Passthrough 'root' support. Rahul Tyagi documented his findings in CPS-745 - Getting issue details... STATUS .  This will be taken as input for the root-function support items planned for Jakarta. While Toine Siebelink wil start conversation with SDN-C team (PTL Dan Timoney ) about possible fixes in SDN-C and/or ODL. Tony Finnerty suggested a future proof workaround where the system will only try the restconf interface on SDN-C if the original request using the rests interface failed and the resource identifier equates to root. 
  • Model-Resync issues. Rahul Tyagi confirmed in CPS-741 - Getting issue details... STATUS that an easy work-around is available. The ensuing discussing made clear that the stakeholders might need to consider a cm-handle sync-state & watchdog in NCMP to ensure model and dat sync of registered nodes (with attributes to enable/disable this functionality)

 

 

  • Software Status
  • Review Process / Release Process
    • Tony can help with Reviews
    • Release (get ready to build images)
    • Prioritize Niamhs & Renu's commits (try to merge today)
    • Toine will cerate task to deliver images, bump version for each repo

  • Discussion on Configuration Parameters
    • Username/passwords (sensitive info) as Secrets variables by convention
    • In DMI we have defined SDNC password as we're not the owner
    • Properties are set in the values.yml oom file under config or secrets
    • The properties are used in the helm charts
    • Each cps component has its own values.yml and helm chart

 

  • Software Status
    • 4 People involved in documentation using group reviews of generated HTML
    • Logging and monitoring admin guide done by Former user (Deleted) for CPS Temporal will be moved to CPS-Core and made generic by Luke Gleeson (Unlicensed) 
    • CPS-645 - Getting issue details... STATUS renu kumari wil investigate option. Stakeholder want clear separation between 'client' and 'internal' interfaces. Although both need to be available for publication, just different target audiences. Preferred Internal interface pre-fix is /ncmpDmi
  • Discuss adding CM-Handle for SDN-R (network Slicing) with Ahila P 
    • Proposal for SDN-R integration: SDN-R Feature for adding cm-handle
    • Toine Siebelink clarified that instead of Async flow steps 2&3 a 'temporary' sync option is available right now on DMI-Plugin. Toine wil mail specification links to Ahila
    • Steps 4 & 5 are also already developed and test in Jakarta using SDN-C. These might need to be tested in SND-R environment.
    • CPS teams prefers NOT to have a third option for adding CM Handles. Instead we agreed on the following possible evolution path:
      1. SDN-R use synchronous method delivered in release I, mentioned above (can be done now)
      2. SDN-R publishes event of the same schema and same topic as PNF. This release CPS  is delivering the relevant event based API in J which has already been investigated , see CPS-393 - Getting issue details... STATUS
      3. eventually, post Jakarta, both SDN-R wil use the current A&AI event based interfaces
    • At any stage Topic and or Event bus provide could be made configurable as long as te event schema stays the same
    • PNF Registration flow is detailed here

 

 

  • Istanbul demo
    • Model Sync
    • Read pass-through datastores
    • Write  pass-through datastore
    • CPS Temporal
  • Jakarta Roadmap
  • Branching Strategy
    • team agreed not to cerate I branch until we need to deliver Jakarta specific functionality or affect stability i.e. interface split
  • CSIT Improvements
    • Aditya Puthuparambil  started discussion re. CSIT testing with Morgan Richomme to include in CPS CSIT in OOM build. Not yet clear how exactly this will work (in Jakarta) Aditya will follow up

    • Team wil continue to deliver CSIT test as planned for Istanbul using current process/framework

 

  • Istanbul Demo by Joe K and Rahul Tyagi 
    • Some feedback on Demo
      • Show explain (local) deployment/docker view first
      • No need to mount node during demo (prerequisite)
      • Execute query on (target) Node before it is added (show failure on registration)
      • Add dummy model (not DMI Registration) to showcase write use case and possible diff between  datastore 'running' and 'operational' (config true/ all)
      • optional: demo sync 2nd node with same models, 3rd node with 1 additional model
      • GREAT DEMO! really cool to see all Istanbul work in action!
  • Software Status
    • Aditya Puthuparambil experimenting with installing Python 3 for CSIT test (new Jira to be added). This might make   CPS-658 - Getting issue details... STATUS redundant (low priority J release)
    • CPS-634 - Getting issue details... STATUS has been re-opened! Rahul Tyagi will work on it

 

Recordings

DateZoom Recording

 

 

 

 

 

 

 

 

 

CPS-211203.mp4

 

 

 

 

 

CPS-210917.mp4

 

CPS-210910.mp4