1. Action point follow-up
  2. Admin
    1. SECCOM reco / baseline images
      1. Patch created based on SECCOM reco
      2. Baseline images created (currently in gate) https://gerrit.onap.org/r/c/integration/docker/onap-java11/+/121545, https://gerrit.onap.org/r/c/integration/docker/onap-python/+/121544
    2. repo cleanup
      1. #action all cleanup the repo (integration/testsuite/Demo) and remove in master what is not used for istanbul/honolulu
    3. versioning
      1. xtesting images can be created now - 1.7.0 created for guilin..bad version 7.0.0 shall be created as well as 8.0.0 for honolulu (shall be possible now..)
        1. in progress....
      2. onapsdk => new version for guilin (7.6.0) and for honolulu (8.0.0)
        1. agreed => 2 versions needed #action michal create the 2 tags (7.6.0=8.0.0)
    4.  Future PTL: how to move forward: houston.pdf
  3. Honolulu
    1. integration of pnf_macro
      1. issue with mongo service when run in // with pnf-registrate, #action Krzysztof and Michal specify the service to be unique and linked to the use case
    2. stability integration in CI
      1. now included in weekly master chain
        1. currently limited to 24 because the system is breaking before 72h
        2. DB tuning in progress
        3. no more Maria DB issue, issues on ONAP side (SO or subsequent component for instantiation tests and data middleware for SDC)
    3. move back to pythonsdk xtesting version corresponding to the chain version (was master for all) => guilin backporting required
      1. that is why we need a "good" version for onapsdk..guilin is consuming guilin xtesting, honolulu/honlulu, master/master
    4. resiliency follow-up -teasing for next week presentation by Natacha Chéreau
      1. see you next week for a short presentation and a demo
  4. Istanbul
    1. discussions on CI/CD category:
      1. shall we split smoke into smoke (E2E test to validate the release) and use cases
      2. stability category added in honolulu, shall we add a resiliency category or put them all together
      3. postponed to next week
    2. onap-k8s update
      1. add the generation of a onap_versions.json to automate the container tracking
      2. onap_versions.json generated => shall be able to track the docker version changes on daily
    3. onapsdk main dev for Istanbul Michał Jagiełło
      1. #action michal send a mail to indicate main topic for onapsdk in istanbul
    4. dcae test update Krzysztof Kuzmicki
      1. postponed
    5. Epic for istanbul to be created
  5. AoB
    1. Thanks Lasse Kaihlavirta for the great work done since more than 2 years for integration project.

  1. Action point follow-up

  2. Admin
    1. committers
    2. baseline images
    3. repo cleanup
  3. Status on CSIT Lasse Kaihlavirta : CSIT status after Honolulu.pptx
    1. CPS = Showcase example, everything is fine
    2. CCSDK previous showcase, still requires distributtion of docker builds, not possible to rebuild locally
    3. DMaaP (dmaap/buscontroller moved), OPTF (optf/odf and optf/has already moved), SDNC (almost finished but still some failures) transition in progress
    4. AAI mumtitenancy done outside ONAP planned to be incoprorated into aai/resources and aai/traversal => use of scala+gatling (examples based on robot so far)
    5. Policy: CSIT moved to reir repositories but tests do not follow reco (test after merge, not project specific, jjb fully specific
    6. Still some active projects with old design (DCAEGEN2, Modeling, Multicloud, OOM-platform-cert-service, SO)
    7. Usecase #action Morgan Richomme dig a little bit to understand if it corresponds to active use cases
    8. VFC and VID do not seem really maintained
    9. VNFSDK: only images for frankfurt => not maintained
    10. CSIT tests disabled (aaf, appc, clamp, music)
    11. clamp moved to SDC => what does it mean from a CSIT perspective
  4. Demo vFW CNF Lukasz Rajewski Konrad Bańka
    1. slide deck: #action Lukasz Rajewski vFW CNF Use Case Demo
    2. use case partially automated but would require a KUD (test cannot use a ns of the existing k8s hosting ONAP)
    3. As the use case is maintained and includes regularly feature changes, it woudl be interesting to include it in CI..to show that it is red/green as part of the use cases (more than smoke tests)
    4. #action Morgan Richomme Konrad Bańka Lukasz Rajewski estimate what is missing for integration in CI
    5. Open question and where the status of the xNF shall be available for ONAP admin..not clear status of the service/vnf/module was available in VID, some test status was available in SDC but not sure to know where the info on the status of the xNF would be the more accurate #action Morgan Richomme send a mail to the community to get feedback on this xNF status page
  5. AoB

  1. Action point follow-up
    1. AP1: Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
      1. not done yet wait for release of the xtesting dockers
    2. AP2: all move or closed the JIRA in honolulu
      1. done
    3. AP3: report your feedback on honolulu lesson learned
      1. to be done
    4. AP4: participants indicate their top 3 priorities for honolulu (https://etherpad.anuket.io/p/onap_integration_istanbul)
      1. done
  2. Admin
    1. Admin status (PTL, election, INFO.yaml)
      1. congratulations Bartosz Gardziejewski elected as integration committer
      1. all the INFO.yaml actions will be performed once
      2. PTL no candidate => escalation done to TSC
    2. xtesting docker release
      1. new way, need to twist the scripts but could be OK. Need to fix guilin CI chain first (issue due to new pbr value)
    3. baseline images: wait for seccom reco for istanbul
  2. Honolulu
    1. regression on multicloud-k8s / pnf-macro
      1. issue due to helm format, no error with the helm charts sued for Lukasz's use case. Fix done, docker images must be regenerated
      2. action Lukasz Rajewski  prepare a demo on test automation managed by Lukasz..why it could not be integrated in CI for the moment (reflexion for Istanbul)
    2. master-weekly
      1. tern
        1. should be fully OK next week
        2. last results : https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/07_22-21/security/tern/index.html
        3. Former user (Deleted) indicates that anothe resource from Saùmsung is working on the topic to improve the images. No feedback from the PTL.Morgan Richomme indicates that he reports the issue to PTL during PTL meeting. Not normal to have no feedback (e.g. CLI improvement patch in gate for a long time), David McBride sent a mail to Former user (Deleted)  #action  morgan sent a mail to PTL on good practice. Reminder that no patch in gate oldes than x months is a criteria for milestones..it is clearly not respected.
      2. stability tests
        1. update on MariaDB tuning => reduce the replica of non core components/removal of appc/... see stability_follow-up 2-2.pdf
          1. redimensioning is enough to avoid DB crash but recommendation to TSC to reduce replica for non core components 15 instances of Maria DB => 15x30 Go of additional RAM needed to avoid crash in load conditions...9 nodes cassandra...
          2. stability tests are now performed on each weekly => results are here: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-05/07_22-21/stability_tests/
        2. choice of the figures for weekly
          1. start with instantiation to avoid lots of models in SDC
          2. 10 instantiation in // during 24h (basic_vm / a la carte bpmn)
          3. then 5 onboarding in // during 24h
          4. needs to add basic_vm_macro if stability reached on first instantiation
        3. DB tuning
          1. suggestions from Samuel Liard  to modify Maria Db and Cassandra DB default configuration: https://gerrit.onap.org/r/c/oom/+/121229/10
          2. all integration committers are using teh default DB values on their lab, no specific tuning, only the nb of compoentns may be reduced (because not used)
    3. CSIT status after Honolulu (we may have to reserve some time for this so not necessarily today):  Integration
      1. introduction to the status of CSIT at the end of Honolulu
      2. #action Lasse Kaihlavirta plan a 20' section for a full session

Présentation

  1. Istanbul
    1. New functional tests
      1. Do we have resources / Shall we consider the existing list and select some to automate?
        1. scale out
        2. vfw CL
        3. Software upgrade
        4. MDONS
        5. 5G slicing
        6. ...
      2. pythonsdk
        1. Current Open issues: https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk/-/issues
        2. onaptests priorities
          1. update pnf_macro with Eli's simulator wrapper?
        3. to be reprogrammed whith Michał Jagiełło Andreas Geißler Illia Halych
    2. Backup and Restore
      1. tooling: Velero? other solutions? Any feedback
      2. Scenario? what can we automate?
      3. #action Morgan Richomme send a mail to the community/task force to converge on a scenario of tests that coudl be used for istanbul
    3. Operations
      1. Upgrade of a pod / chart
    4. Stability
      1. include a test with basic_vm_macro
      2. tuning of the DB
    5. Resiliency
      1. Integration of a k8s chaos framework?
  2. AoB
    1. if ready Krzysztof Kuzmicki could also make a demo of the simulator

  1. Action point follow-up
    1. Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
      1. not done yet..several issues on LF (SSO, Nexus, gerrit,..)
    2. Morgan Richomme create a JIRA for basic_vm_macro
      1. not needed last results with CCSDK fix look good
    3. Former user (Deleted) finalize push to LF back end
      1. test in progress on weekly master..config is good
    4. Morgan Richomme create resiliency chapter in doc and initiate Jira for confirmed issue (appears at least 2 times), restart of a controller still to be done to complete the test.
      1. done see next section
    5. Krzysztof Kuzmicki create a wiki page in Istanbul page for robot refactoring follow-up
    6. Krzysztof Kuzmicki create a wiki page in Istanbul page for DCAE migration
  2. Admin
    1. INFO.yaml updated needed (NS repo have no yaml + status of the project)
    2. Commiter Promotion Request for Bartosz Gardziejewski: results 10 Yes on 10
    3. Commiter Promotion Request for Alexander Mazuruk: results 4 Yes on 4
    4. PTL election: call for self nomination sent
      1. all operations on INFO.yaml will be done at the same time
      2. note Lasse Kaihlavirta indicated that he also will step down in 1 month
  3. Honolulu
    1. status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
      1. few JIRA still open # action all move or closed the JIRA in honolulu
    2. release testsuite 1.8.0 done
      1. just merged
    3. doc updated
      1. done
    4. Remaining work on tests
      1. macro issues Lukasz Rajewski

      2. stability/resiliency tests

        1. Maria DB galera troubleshooting

        2. honolulu_sign-off.pdf
        3. stability_follow-up.pdf
    5. Honolulu: Lessons Learned
      1. #action report your feedback on honolulu lesson learned
  4. Istanbul
    1. New Jira board created: https://jira.onap.org/secure/RapidBoard.jspa?rapidView=233
    2. brainstorm on new use cases for Istanbul open => https://etherpad.anuket.io/p/onap_integration_istanbul
      1. participants indicate their top 3 priorities for honolulu
  5. AoB

  1. Action point follow-up
    1. Morgan Richomme review Integration simu and release versions if possible + update doc accordingly
      1. not done, will be done after doc, stability and resiliency testing #reaction
    2. Lukasz Rajewski properly tag SO bugs to reference them as blocking in Integration blicking table
      1. SO bugs do not appear in Honolulu Integration Blocking Points => to be crosscheck with Lukasz Rajewski
    3. if SDNC-1515 merged and pnf-simu OK in honoluluMorgan Richomme add pnf-macro to CI
      1. honolulu daily shows that the basic_vm_macro issue is not really fully fixed => #action Morgan Richomme create a JIRA for basoc_vm_macro
    4. Morgan Richomme grant access to ChrisC to check the status of SDC/DB
      1. done see section on SDC stability test
    5. Morgan Richomme retry and wait for the 5 minutes (Eviction timeout) then investigate on the networking issue
      1. done, Jira completed
    6. Morgan Richomme check ansible image availability for chained-ci
      1. doneAndreas Geißler gave a new try, still issue but not related to teh image. open question: shall we also reference the image in the CI page of the release?
  2. Admin
    1. INFO.yaml updated needed (NS repo have no yaml + status of the project)
    2. Commiter Promotion Request for Bartosz Gardziejewski: please vote before end of the week. BTW if other committer proposal => feel free to suggest (to replace Pawel, Marcin and partly Thierry)
  3. Honolulu
    1. status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
    2. release testsuite 1.8.0 under built
      1. in progress https://gerrit.onap.org/r/c/testsuite/+/120960, once merged, submit a patch to OOM
    3. Remaining work on tests
      1. macro issues Lukasz Rajewski
        1. pnf-macro test to be done on master before integration in CI (working fine in guilin)
        2. basic_vm_macro good results on guilin but not so good on master/honolulu => JIRA created
      2. tern: test run in CI but results not pushed yet to LF Backend => WIP to manage properly the push to lf backend. On last honolulu and master-weekly, artifacts were produced and pushed manually #action Former user (Deleted) finalize push to LF back end

      3. stability/resiliency tests

        1. Resiliency test

          1. worker node restart (thanks Bartek Grzybowski for the full analysis) => how do we follow, shall we create Jiras?
          2. test done, eviction looks good. some issues detected on some pods but not always trivial to reproduce may depend on the associated evicted pods. sometimes Init error, sometimes pod Running but exception..sometimes first eviction look good, but second seems to be fail... different cases referenced in Honolulu Resiliency and Backup and Restore test
          3. for SDC issue, the restart may be due to teh fact that the SDC is trying to recreate tables that already exist in cassandra
          4. #action Morgan Richomme create resiliency chapter in doc and initiate Jira for confirmed issue (appears at least 2 times), restart of a controller still to be done to complete the test.

        2. stability tests

          1. SDC tests: 72h 5 parallel on boarding INT-1912 - SDC Stability test: success rate drops after ~ 500 onboarding Open
          2. ChrisC reports that it was useful to detect new issues, especially the continuous time increasing that my be lead to teh DB and/or a middle ware layer (explicit exception seen)
          3. plan to integrate such test in weekly as part of benchmark test
          4. instantiation tests
            1. 48h 5 parallel basic_vm (initial issue on SDNC (fixed by restart) then regular timeout value then internal openstack issues INT-1918 - Stability tests: // instantiation Open
            2. more timeout than in guilin, need to be confirmed with a 1 test (no parallelization), mariadbgalera issue observed on some replica
            3. 72h 1 test basic_vm => problem with mariadb galera, results will not be as good as in guilin run
            4. issue with mariadbgalera after 24h (but tests run after the 48h tests..) once finished (thurday morning => reinstallation of honolulu weekly to reproduce the tests with the last dockers
            5. mariadb galera workaround reported by Bell to OOM, seems to improve the overall quality of gate but trigger an new issue on CDS. tradefoff between redundancy and efficiency not clear
    4. documentation update started but not finished => https://gerrit.onap.org/r/c/integration/+/120236
    5. estimation 2-3 days still needed to compelte the doc / release note + testsuite integration
  4. Istanbul
    1. New Jira board created: https://jira.onap.org/secure/RapidBoard.jspa?rapidView=233
    2. robot refacroting..it eems we have a plan (Krzysztof Kuzmicki )
    3. discussion by mail, #action Krzysztof Kuzmicki crerate a wiki page in Istanbul page
    4. migration of DCAE from cloudify to helm chart..first analysis by Krzysztof Kuzmicki , impacts to be planned on several tests
    5. discussion by mail, #action Krzysztof Kuzmicki crerate a wiki page in Istanbul page
    6. New CSIT (AAI, CDS,..)
    7. AAI is folowwing the SDC way (Maven based), for CDS it shoudl be possible to reuse what Lasse Kaihlavirta suggested. AAI team contacted SDC
    8. brainstorm on new use cases for Istanbul open => https://etherpad.anuket.io/p/onap_integration_istanbul
    9. all review the page, it is an etherpad..so do not refrain yourself..put everything you have in mind
  5. AoB
    1. Lasse Kaihlavirta closed the CSIT refactoring EPIC
    2. Lasse Kaihlavirta also disabled useless SO CSIT tests (with very old images) - the Jira was closed without action by Seshu Kumar Mudiganti so we remove some jobs as they are meaningless (test with casablanca images)
    3. Lasse Kaihlavirta add a browser cleanup on robot healthcheck (vid) to save resource, it will be integrated in 1.8.0


  1. Action point follow-up
    1. Illia Halych review the official simu page (pythonsdk wrapper)
      1. done, no example for Honolulu, to be completed in Istanbul
    2. Morgan Richomme refine the information about the conditions of the resiliency tests of worker restart -
      1. See Honolulu topic
    3. Lasse Kaihlavirta Offline discussions with Lasse about AAI CSIT
      1. Done
    4. Krzysztof Kuzmicki Next week proposal about what we can do with robot
      1. Postponed to next week
    5. Krzysztof Kuzmicki INT-1907 - browser_setup.robot does not provide proper teardown Open  no teardown for browser-based checks - high consumption of resources at the end
      1. used by VID test, workaround to be confirmed
    6. Maybe we need more information about DMaaP simulator use and to write it better in the simulator doc → Reported by Lasse Kaihlavirta

      1. Globally we are not well structured on the simulator, on the release note of version X we should list all the versions of the simulators we used for the validation. It shall be done by the use case teams that developed a simu and:or the integration team. The difficulty ois to get the full view on the simulators, some are hidden in repositories, some are unmaintained,...# action Morgan Richomme review Integration simu and release versions if possible + update doc accordingly

    7. Need of consolidation of versioning of images of simulators used in CSIT  → Reported by Lasse Kaihlavirta
      1. sure it is a bit messy...and impossible to maintain, integration can cleanup time to time..but the best way for functional tests => bring functional tests including simu in their own repo. And if the simu can be used more widely create a dedicate repo under integration/simulators..
  2. Admin
    1. INFO.yaml updated needed (NS repo have no yaml + status of the project)
      1. strange INFO.yaml is needed to create the repo => to be xchecked. Morgan Richomme indicates to Thomas Kulik that it will be done with the next INFO update (needed as at least 2 committers announced they will stop after Honolulu)
    2. thanks to Pawel and Marcin for their contributions and all the best for the next challenges
  3. Honolulu
    1. status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229
    2. Remaining work on tests
      1. pnf_macro => OK on daily guilin, what do we do regarding CI integration

        1. kudos to Michał Jagiełło  - first pythonsdk tests with simulator - let's wait for the merge of Dan fix for SDNC-1515, if OK #action Morgan Richomme add pnf-macro to CI

        2. stil SO bugs preventing Lukasz Rajewski to complete his test .. deja-vu?
        3. #action Lukasz Rajewski properly tag SO bugs to reference them as blocking in Integration blicking table
      2. tern: test run in CI
        1. test results available (launched manually) => https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-04/19_18-36/security/tern/index.html
        2. next CI run shall be good..wait and see

      3. stability/resiliency tests

        1. I started the first resiliency tests: TEST-308 - [Resiliency] Evaluate ONAP behavior on k8s worker node restart In Progress , replay done, mail sent to the community (network issue?)

          1. behavor different on Nokia (pod stuck for ever in Terminating state when stopping a working) and Samsung RKE2 cluster, continue discussion on the ticket Bartek Grzybowski  indicates that we shall be careful especially with statefulset, it coudl explain some issues
          2. #action morgan retry and wait for the 5 minutes (Eviction timeout) then investigate on the networking issue

        2. stability tests

          1. SDC tests: 72h 5 parallel on boarding started on the 20th of April..wait and see...
            1. First tests were all OK but duration ontinuously increases. No error rate is high. Wait for the end of the tests to get the graphs.
            2. #action Morgan Richomme grant access to ChrisC to check the status of SDC/DB
          2. instantiation tests to be planned after onboarding tests

      4. consolidation of versions

        1. better exception catching + integration of the GUI => still action Morgan

      5. removal of the submodule: done

    3. documentation update started but not finished => https://gerrit.onap.org/r/c/integration/+/120236
    4. release testsuite 1.8.0 to be done?
      1. next week if no objection
    5. simulators: whuch dockers are available in nexus today? could we clarify the build chain?
      1. see previous topic
  4. Istanbul
    1. New Jira board created: https://jira.onap.org/secure/RapidBoard.jspa?rapidView=233
    2. discussion on future tests
      1. follow up integration AAI tests in CSIT
      2. CDS in CSIT?
  5. AoB
    1. Andreas Geißler mentioned that his CI chains were affected by the non availability of an ansible image used in the runner => could be a problem, need to release also such images somewhere to avoid such issues #action Morgan Richomme check ansible image availability for chained-ci (2.7.13)
      1. issue also on Orange chains last week (build chain of ansible image was broken, chain fixed)
    2. ChrisC raises a question on python2.7 in jenkins CI and in robot. Partly addressed by the initiative of robot refactoring. note xtesting dockers also still using python2.7 due to dependency to python-utils that has not evolved for a long time.

  1. Action point follow-up
    1. Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
      1. in progress
    2. Michał Jagiełło complete basic_vm_macro
      1. done
    3. all review the wiki page and add the links of the GUI you are using
    4. all review/complete the wiki on process to integrate new external tests in CSIT or CI
    5. Krzysztof Kuzmicki complete the official simulator page with NS illustration
      1. documentation is done
    6. Illia Halych review the official simu page (pythonsdk wrapper)
      1. no feedback
    7. all review your Jira ticket move to instanbul or close
    8. Morgan Richomme review the patch dealing with sumbmodule removal +  cherry + adapt xtesting-onap
      1. done
    9. Morgan Richomme complete wiki instanbul page to track this AAI scenario
      1. done, thanks to Lasse Kaihlavirta to ahve provided more examples.
  2. Admin
    1. good occasion to discuss on the future organization and the future PTL..
    2. instanbul renaming (instead of istambul) done in seccom and oparent repo
  1. Honolulu
    1. status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229&selectedIssue=INT-1842
    2. Daily Honolulu CI/CD chain created
      1. still sometimes some issues at SDNC startup (, Dan Timoney already added check mechanism but it seems that there are still issue in mariadbgalera, workaround (restart of SDNC seems to be OK)
      2. basic_vm_macro does not work due to SDNC ()
      3. basic_vm and basic_cnf fail randomly - mostly due to SDC problem or timeout in SO - this looks less stable than in guilin
      4. observed problem with config-assign/config-deploy in vFW-CNF integration test  - we need to cover config-assign/config-deploy in our smoke tests
      5. to sum up: we have two major problems with service-macro-create
    3. Remaining work on tests

      1. basic_clamp: OK

      2. basic_vm_macro: OK show a regression in SDNC ()

      3. pnf_macro => Michał Jagiełłois working on the latest comments and needs further tests

      4. refactoring of 5gbulkpm => OK (additional patch required submitted by Krzysztof Kuzmicki and cherry-picked)

      5. tern: test declared in CI => we went further on the weekly-honolulu (https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/jobs/1165577708), ansible-playbook was missing on the runner...fix merged (we need to use image with already ansible installed) ...to be continued but at least it is properly triggered

      6. stability tests

        1. I started the first resiliency tests: , replay today - we need to add remark this is only for stateless services - what about DB?

        2. stability I planned to regive a try to Natacha's suite as soon as the other tests are OK

      7. consolidation of versions

        1. better exception catching

        2. integration of the GUI part to generate an html reporting (for the moment I usually replay manually the tests, get the results and apply the GUI processing manually)

      8. removal of the submodule: test in progress (seems there was an import in the simu step), once done we could review our docker build (to remove the developer mode) but we shall not forget to adapt xtesting-onap according ly (/Src/onaptests/src/onaptests => /usr/lib/python3.8/sites-available/onaptests => action morgan for the week end..to avoid breaking everything before my PTO..

    4. documentation update started => https://gerrit.onap.org/r/c/integration/+/120236
    5. release testsuite 1.8.0 to be done
  2. Istanbul
    1. discussion on future tests
    2. follow up integration AAI tests in CSIT 
  3. AOB


  1. Action point follow-up
    1. Michał Jagiełło pnf_macro add processing for pnf_macro to wait for a clean startup of the simulator (currently time based, and it seems that it takes more time than expected)
      1. Not done
    2. Morgan Richomme cleanup basic_clamp and re-run it on a daily master
      1. Done
    3. Michał Jagiełło complete basic_vm_macro
      1. In progress, issue in SDNC:
    4. Morgan Richomme create wiki page to reference the GUI endpoints to be tested
      1. Done Honolulu UIs
    5. all review the wiki page and add the links of the GUI you are using
    6. Morgan Richomme create a wiki page to improve the process on test integration in CSIT and/or CI from third parties
      1. Done How to help projects to integrate their tests in Integration?
    7. all review/complete the wiki on process to integrate new external tests in CSIT or CI
    8. Morgan Richomme initiate an official page on simulator
      1. Done https://docs.onap.org/projects/onap-integration/en/latest/integration-simulators.html
    9. Krzysztof Kuzmicki complete the official simulator page with NS illustration
      1. To be Done
    10. Bartek Grzybowski review the official simu page (list of usable simu)
      1. Done
    11. Illia Halych review the official simu page (pythonsdk wrapper)
      1. To be Done
  2. Admin
    1. Illia Halych officially new Integration commiter, welcome !
    2. No update on xtesting docker release
    3. LF IT backend , not possible to move logs, no cleanup mechanism (manual cleanup needed)
  1. Honolulu
    1. status:board review https://jira.onap.org/secure/RapidBoard.jspa?rapidView=229&selectedIssue=INT-1842
      1. action #all review your Jira ticket move to instanbul or close
    2. Daily Honolulu CI/CD chain created
      1. https://logs.onap.org/onap-integration/daily/onap_daily_pod4_honolulu/2021-04/
    3. Remaining work on tests

      1. basic_clamp: integrated in CI, several successful runs, but improvements needed to manage exceptions (Policy/DCAE/CLamp) => https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/120234

      2. basic_vm_macro (replace vfw_macro): in progress still some issues on the daily honolulu

      3. pnf_macro => add mechanism to have a better control of the pnf simu launch (timer is not enough) => # action Michal

      4. refactoring of 5gbulkpm => #action Krzysztof => test done this afternoon => merged

      5. tern: test declared in CI, but issue => # action morgan & Alexander

      6. stability tests

        1. I started the first resiliency tests: TEST-308 - [Resiliency] Evaluate ONAP behavior on k8s worker node restart

          1. Other labs are experiencing different issues
            1. Nokia => pod remain in stuck
            2. #action Daniel, Andreas create a JIRA ticket for each resiliency test
            3. #action morgan create a wiki page to consolidate resiliencey tests

        2. stability I planned to regive a try to Natacha's suite as soon as the other tests are OK

          1. to be tried asap: if enough resources => weekly Honolulu to be launched on Wednesday

      7. consolidation of versions: I was waiting for the come back of Pawel but as far as I understood, you will move to othe r challenges soon.. :) so I need to review that

        1. better exception catching

        2. integration of the GUI part to generate an html reporting (for the moment I usually replay manually the tests, get the results and apply the GUI processing manually)

      8. removal of the submodule: test in progress (seems there was an import in the simu step), once done we could review our docker build (to remove the developer mode) but we shall not forget to adapt xtesting-onap according ly (/Src/onaptests/src/onaptests => /usr/lib/python3.8/sites-available/onaptests

        1. #action morgan review the patch dealing with sumbmodule removal +  cherry + adapt xtesting-onap

      9. CDS / CSIT => will not have the resource to do it for Honolulu => move to Istanbul

      10. GUI tests: wait for completion of the Wiki page and see if enough bandwidth => move to instanbul

    4. documentation update started
    5. honolulu branching done
    6. release testsuite 1.8.0 to be done
  2. Istanbul
    1. requests from new project (OOF, AAI) "on how to bring new tests to integration" => How to help projects to integrate their tests in Integration?
      1. Discussion on the support for AAI gatling tests. By default the multitenancy support is disabled and they require a keycloak third party
      2. first issue is the test env => we coudl imagine to have a daily scenario daily_master_multitenancy but we need to take care of the resources. Until we cannot rely on the windriver lab, it will be hard to create new scenarios...
      3. installation of keycloak shall be done in a tooling namespace through a helm chart => OOM
      4. Mohammad Hosnidokht indicated that gatling part is almost automated (docker verifying the roles)
      5. William Reehil indicated that it will probably break the other tests
      6. it is probably more relevant to start by creating a CSIT test (fonctional test) with AAI + keycloak rather than a daily.. 
      7. #action Morgan Richomme complete wiki instanbul page to track this scenario
    2. run robot tests out of onap namespace: How to run the tests out of the ONAP cluster?
  3. AoB

  1. Action point follow-up
    1. Morgan Richomme action morgan initiate Illia Halych committer promotion procedure
      1. done
    2. Morgan RichommeMichał Jagiełło develop basic_vm_macro and basic_cbf_macro
      1. basic_vm ready to be tested, basic_cnf shall be very closed
    3. Morgan Richommecontact Lasse Kaihlavirta to see if CDS regression could be integrated in CSIT
      1. done, no bandwidth available, Jira created
    4. Morgan RichommeAndreas Geißlerlist UI and create a healthcheck test to check the availability of the UI
      1. done, see next section
  2. Admin
    1. Committer promotion: formal vote initiated Committer Promotion Request for Integration Illia Halych
      1. vote in progress, please committer vote before the end of the week
    2. testing docker discussion (formating, unexpected removal of daily build dockers,..)
      1. 2 tickets created on LF
        1. "fresh" xtesting dockers disappear time to time on the nexus https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-21812
        2. "impossible to release because the tag format of the snapshot docker does not respect some rules" => https://jira.linuxfoundation.org/plugins/servlet/theme/portal/2/IT-21710
        3. xtesting dockers are rebuilt daily based on the branches (guilin/master end of branches of all the components embedded in the docker), CI or end users cannot guess the format imposed by the LF rules (used for components e.g. 6.0-STAGING-20210315T185806Z)
  1. Honolulu
    1. status: INT-1883 - Reccurent various errors in smoke tests In Progress
      1. better results on daily master
      2. last patch for the release was the SDNC patch, currently in gate - shall be merged before the TSC
      3. dockers version (on Monday Master) - SO integrated meanwhile, wait for SDNC but otherwise it looks good: https://docs.google.com/spreadsheets/d/1t3GNRtabdkVxG4ZAxqBJ-X7OO-Zv6xRwd8KrR52V9zU/edit?usp=sharing
      4. weekly versions test show also some progress but we still have bad versions: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-03/29_08-09/security/versions/versions.html, results transfered to seccom (Amy Zwarico )
    2.  lab
      1. azure staging resinstallation initiated
        1. done, keys pushed to the lab, ready to use: https://logs.onap.org/onap-integration/daily/onap_oom_staging_azure_1/2021-03/31_07-06/
      2. windriver: feedback from Intel => recontact them after Honolulu for Hardware upgrade
    3. Automated tests
      1. pnf_macro
        1. issue with the start of the simulator, wait 30s but it takes more time to start in master, a mechanism to check that everything shall be implemented #action Michał Jagiełło add mechanism in simu step to verify that everything is well started before sending a VES event.
      2. basic_clamp
        1. some regression had been introduced in master. Cleanup in progress, tests to be done after the meeting # action morgan re-run a clean basic_clamp on daily
      3. basic_vm_macro
        1. test almost ready, a config-deploy file was requested to complete the deployment, Michał Jagiełło wil include an empty file (nothing to do..) and re-test #action Michał Jagiełło complete basic_vm_macro tests
      4. tern integration in CI
        1. long duration test seen in https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-03/29_08-09/security/tern/index.html but not sure the processing has been executed, no ternenv seen in the master-weekly lab. troubleshooting to be done (same mechanism will be reused for the stability tests.)
      5. Krzysztof Kuzmicki indicates that there are issues on cmpv2 and hv-ves on IPv4/Ipv6 lab due to proxy issues in Nokia lab, errors can be ignored
    4. stability and resiliency testing
  2. Istanbul
    1. first tests to execute the robotframework based tests in external ns
      1. several current robotframework, k8s job healthcheck
    2. url checker for GUI
      1. mail sent, no feedback #action Morgan Richomme create a wiki page to share the link before building the test (could be introduced in Honolulu)
      2. #action all review and complete this patch
    3. requests from new project (OOF, AAI) "on how to bring new tests to integration" => discussion on the best way..
      1. Q&A + official exist but are probably not enough
      2. #action Morgan Richomme create a wiki page to collect integration contributer view
      3. #action all review and complete the page
  3. AoB
    1. Do we have/should we have some Best Practices laid down for various simulators? (Lasse)
      1. the answer is no, we do not have, and yes we should have
      2. #action Morgan Richomme initiate a official doc page on simulator
      3. #action Krzysztof Kuzmicki use NS refactoring to complete this page

  1. Action point follow-up
    1. AP1: Morgan Richomme  amend pythonsdk_tests to include the workaround
      1. new patch submitted yesterday (first one not efficient as no exception was raised by the SDK at this stage, just a 404 but no exception ResourceNotFound)
      2. https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/119655
    2. AP2: Santosh bayas  identify inputs to see if parameters are missing + publish robot code in testsuite for review
    3. AP3: Morgan Richomme  Krzysztof Kuzmicki  check why we have a path difference between execution in the robot pod and in the xtesting docker
      1. not done
    4. AP4: Morgan Richomme  review MR from Former user (Deleted) for the integration of tern in ci-weekly
      1. first attempt done last friday but faield, new attempts to be done, see next section
    5. AP5: Morgan Richomme  contact use case owner to collect the possible needs and remind the need to update the doc
      1. done see next section
    6. AP6: Morgan Richomme  Krzysztof Kuzmicki  create Epic and tasks to detail expectations on robot pod refactoring (alpine, split web/robot, python3, execution from outside the cluster, use python baseline image,....)
      1. done
  2. Admin
    1. update on the use cases (mail)
      1. Use Case for ONAP-based SON for 5G networks: "We do the testing in Winlab (Rutger’s University) within our use case team. So there is no requirement from Integration team for resources (human as well as lab infra). Some automation has been done – see https://gerrit.onap.org/r/gitweb?p=testsuite.git;a=tree;f=robot/testsuites/usecases;h=1690b9d4b7222f3748211df3a539e2a7d48e2a88;hb=HEAD."

      2. E2E Network Slicing use case requirements for Honolulu release: "We do the testing in CMCC lab, Winlab (Rutger’s University), and at Windriver (OOF tenant) within our use case team. So there is no other requirement from Integration team for resources (human as well as lab infra). Automation – we expect to make some progress in Istanbul release."

      3. CCVPN - Transport Slicing use case requirements for Honolulu release  Henry/Lin (some tests automated, dedicated repo created) "We are using our local lab in Ottawa to test the CCVPN use-case.  And in parallel, the integration test of Network Slicing and Transport Slicing is done together with the Network Slicing use case team (i.e., using CMCC lab and Win lab)."
    2. doc updated
    3. Time to think to renew committers (some are no more very active) => suggest to promote Illia Halych
      1. #action morgan initiate promotion procedure
  1. Honolulu
    1. Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
      1. good progress on Master, still lots of patches in gate
      2. on daily Master still regular timeouts on basic_vm|cnf|network
        1. these 2E2 basic tests are leveraging A lacarte bpmn, it seems that most of the Service Providers priviledge the macro bpmn
        2. pnf_macro under integration in CI
        3. it would make sense to keep 1 test with a la carte workflow (basic_network) and move basic_vm and basic_cnf in macro Michał Jagiełło and Morgan Richomme started working on this task
      3. Daily Guilin very stable - no so many timeouts for the basic_tests...so for sure the change of DB had an impact..we are not very good in evaluating the DB performance...important to address before B&R tests
      1. VID issue fixed
      2. Certificates at large fixed (openstack, AAI,..)
      1. Daily master of yesterday looks fine: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/2021-03/23_11-52/
      2. Majority of the issues due to timeout on SO requests
        1. Issues could be due to SO or any other components SO is discussing with including DB
        2. On Guilin it is much more stable and for a full workflow it take usually 2 times less average (1100 versus 500s)
        3. Code did not changes that much => could be due to the DB upgrade
        4. #action morgan/michal: write basic_vm_macro, basic_cnf_macro
        5. Lukasz Rajewski indicates that we must keep the test of a la carte bpmn because they are usefull to specify vnf order and complete macro mode
        6. the goal is not to exclude these tests but to create new one on macro mode which is the main used bpmn
        7. Krzysztof Kuzmicki indicates that some SO slow down are also affecting the pf-registrate, wich is using te macro mode
    3. Automated tests
      1. update on CDS regression test (session with JC Morgan Richomme
        1. session done last week, it was possible to launch the mock and start running the tests
        2. some issues due to the fact that the test code was not fully up to date regarding master version
        3. #action morgan contact Lasse Kaihlavirta and see if integration in CSIT will nto make more sense
      2. DCAEmod integrated in CI Krzysztof Kuzmicki
        1. integrated in CI
        2. false negative (icon red but test green) due to the fact that the test was not declared in the test DB. Test added, so at teh end of xtesting the status is pushed ot the DB http://testresults.opnfv.org/onap/api/v1/results?case_name=dcaemod
        3. From a dashboard perspective => test in healthcheck = 1 project / Smoke = test involved several projects BUT test still executed as part of smoke test in CI (avoid race condition with DCAE healthcheck)
      3. tern integration in CI failed in last weekly (gitlab-ci under review)
        1. use of a bad env var for the weekly rules change merged to be retested
  2. Istambul
    1. Move Robot pod out of ONAP cluster (including robot)
      1. certificate and robot pod to be executed in onap-testing cluster
        1. deals only with xtesting (not the robot pod)
        2. the idea is to keep the onap namespace cleaned
        3. to be tested, only open question = cm shall be recreated in teh new onap-testing namespace? url seems OK in cm to allow x-namespace exchanges
    2.  PythonSDK
      1. which tests? what is poorly covered?
        1. vFWCL? scaleout?
        2. NBI
        3. basic loop (basic_clamp extended)
        4. ETSI bpmn?
        5. use cases?
    3. Robot
      1. Bell/apex (if not done in Honolulu)
    4. Add a retry capability in CI? other rules (if onap-helm or onap-k8s Fail "too much" => do not execute E2E tests to save CI time?)
      1. Open discussion on the missing tests
        1. Bartek Grzybowski Illia Halych  Backup & Restore
        2. Michał Jagiełło Krzysztof Kuzmicki Service Sanity check. Healthcheck are not enough, they may be PASS when the system is not working at all..Problem reported already (lots of Healthcheck tests do not provide more information that the onap-k8s checking that the pods are up&running). We have several examples when the pods are up&running but it does not wrok. Exception shall not caught to hide the issues. We would need to formalize a little bit to discuss with the PTL.
        3. Andreas Geißler we are missing a simple test checking the UI endpoint
          1. #action andreas morgan send a mail to PTL to collect admin and user UI and create a simple test checking that the UI are available
        4. Former user (Deleted) add clair scan, even if it shall be done at LF, usually showing it in weekly is the fastest way for adoption
    5. CI dashboard: improved dashboard to identify regression over time (see xls graphs) - multi platforms?
  3. AoB

  1. Action point follow-up
    1. AP1 Illia Halych : ask OOM for the right place to host helm chart
      1. unformal exchange. For CDS helm charts are stored intestsuite/cds/mock-* until we coudl move the helm charts to a helm registry
    2. AP2 Andreas Geißler add warning in doc for backup and restore
      1. patch done
    3. AP3 Morgan Richomme contact TSC regarding B&R task force and possible scenario for Honolulu
      1. done, mail sent, cl664y@att.com answered and agreed on shortmid term plans
    4. AP4 Morgan Richomme test basic-pnf (macro) on daily master before integrating thsi use case in CI
      1. done, test issues reported to Michał Jagiełło  and added in
  2. Admin
    1. congratulation to the team for the S3P..now it will be cool to fix the issues in Master/Gating (smile)
    2. windriver re-installation: to be planned after honolulu => answer sent to Intel team
    3. Azure Staging lab reinstalled
      1. #acrion Morgan Richomme  send a mail to the community to indicate that staging azure is back (guilin maintenance release as master is broken)
  3. Honolulu
    1. Troubleshooting campaign status: INT-1883 - Reccurent various errors in smoke tests In Progress
      1. several issues found
        1. SDC races => including offset to move to pseudo sequential seems to reduces the number of SDC issues. Still one where ceritification seems to take to much time => possibility to add a workaround in the test to retry and/or delay the creation/certification
          1. #action Morgan Richomme amend pythonsdk_tests to include the workaround
        2. VID 500 due to the presence of a "U" char in random password (which explains why we did not have always the error...à - deobuscated by VID even if not obfuscated using jetty library interprating U as unicode char breaking everything. Problem found and fixed by Krzysztof Opasiak => last gating seems reasonably better
        3. SO error (so-bpmn stuck) was alsready observed time to time in guilin, so it is not a new problem...it explains why sometimes an instantiation takes 3 minutes sometimes 20, a jIRA had been created on this aspect
        4. rebase/remerge in gate in progress...gates will burn over the next days...but it means that master chains should be back and usable
    2. Automated tests
      1. demo Apex pdp Policy Santosh bayas  Prudence Au
        1. test done on Bell labs, adaptation needed to be intgrated in CI (config in a dedicated file not leveraging CM information)..
          1. #Action Santosh bayas  identify inputs to see if parameters are missing + publish robot code in testsuite for review
      2. CDS regression test (session with JC  planned on thursday to give a try on Orange lab and see how we can integrate in CI - helm chart finally with docker file in mock repositories)
        1. mock in gates
        2. troubleshooting planned on thursday
      3. DCAEmod, 5Gbulkpm evolution (Krzysztof Kuzmicki ) → Morgan Richomme 
        1. Could we move this point to next week when bulkpm over HTTPS server will be finished and DCAEMOD in daily CI ?
        2. #action Morgan Richomme Krzysztof Kuzmicki check why we have a path difference between execution in the robot pod and in the xtesting docker
      4. pythonsdk (pnf, clamp, .., removal of the submodule, patch to allow test replay)
      5. tern integration in CI Former user (Deleted)  Morgan Richomme - how to test/troubleshoot ansible par
        1. Former user (Deleted) provides follow up in the MR https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/-/merge_requests/37
          1. #action Morgan Richomme reriew MR for the integration of tern in ci-weekly
          2. #agreed moved weekly reinstallation on Friday next week
    3. S3P tests
      1. back on B&R mail to the community => only resilience tests to be planned for Honolulu
      2. stability => target?
        1. wait for master stabilization
        2. Figures to be decided: 5 // onboarding during 48, 20 // instantiation during 48?
      3. feed back on first test executed with litmus
        1. k8s chaos monkey framework
        2. it was possible to anote onap component and generate chaos..coudl be used for resiliency testing to check that components are recovering
    4. use cases
      1. who knows the list of Honolulu use cases?..
        1. Lukasz Rajewski  shared the link with the Honolulu Release Requirements
        2. no requests received so far
        3. no documentation changes seen
        4. #action Morgan Richomme contact use case owner to collect the possible needs and remind the need to update teh doc
  4. Istambul => for information to be discussed next week
    1. Move Robot pod out of ONAP cluster (including robot)
      1. certificate and robot pod to be executed in onap-testing cluster
    2. CI dashboard: improved dashboard to identify regression over time (see xls graphs) - muti platforms?
    3.  PythonSDK
      1. which tests? what is poorly covered?
        1. vFWCL? scaleout?
        2. NBI
        3. basic loop (basic_clamp extended)
    4. Add a retry capability in CI? other rules (if onap-helm or onap-k8s Fail "too much" => do not execute E2E tests to save CI time?)
    5. Robot refactoring: target (slot planned after the meeting if we are short in time)
      1. #action Morgan Richomme  Krzysztof Kuzmicki  create Epic and tasks to detail expectations on robot pod refactoring (alpine, split web/robot, python3, execution from outside the cluster, use python baseline image,....)
  5. AoB

  1. Action point follow-up
    1. AP1: Andreas Geißler check if doc of the official use cases are excplictely referencing the robot pod
      1. not done yet
    2. AP2:  Morgan Richomme  include Nokia pod in onap-integration web site
      1. done => http://testresults.opnfv.org/onap-integration/
    3. AP3: all help reviewing the OOM patches to give Integration feedback
      1. in progress see next section
  2. Admin
  3. Honolulu
    1. pythonsdk
      1. smoke test issues (basic_vnf, basic_cnf) 10'
        • sequential versus parallel
          • tests planned to move to E2E sequential to avoid side effect due to parallelization. Parallel tests could be done on weekly...need to fist master and gating chains first.
      2. tests update (Michał Jagiełło 15' )
        1. pnf (macro flow)
          1. ready to be tested (logger fixed)
          2. #action morgan test basic-pnf on daily
        2. exceptions not caught
          1. patch provided
        3. basic_clamp
          1. almost there: last issue due to DCAE TCA, seems tehre is a regression on dcae due to python2.7 / Python3 => . Wait for fix before retrying.
        4. wrapper
          1. OK, question on where to host the helm charts for the simulator (pnf and future one)
          2. #action contact OOM team => similar question than the for CDS mock objects
          3. it is possible to remove the submodule - just be carefull, once removed => possible to deploy onapsdk as a "standard module" no more as developer module => impact on xtesting-onap as the paces will be different
    2. Backup/restore Morgan Richomme 15' Backup_and_restore_discussion.pdf
      1. discussion on a realistic use case...
        1. backup and restore seems too optimistic
        2. consensus to work on the resiliency of the core component
        3. #action Andreas Geißler warning shall be added in the documentation, as aarna network is not really usable for a real backup/restore (more some examples to use velero)
        4. Companies that tried velero has issues due to teh fact that ONAP is not really cloud native
  2. AoB
    1. cmpv2 about to be moved from healthcheck to smoke
    2. Krzysztof Kuzmicki indicates that 5Gbulkpm will be updated ..may have some turbulences during the update

Actions

  1. Action point follow-up
    1. no action point from last week (several docs)
    2. action point from the 17/02
      1. AP1: Morgan Richomme : contact vcpe tosca team to get their position in Honolulu ()
        1. done no feedback on the JIra
      2. AP2: Former user (Deleted) : get the list of projects using the baseline images
        1. done
      3. AP3: Illia Halych amend https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/117192 for helm3 support
        1. done
      4. AP4: Morgan Richomme test basic_pnf on Orange daily master with onaptests:master
        1. not done PTO..but I can reaction myself;.
      5. AP5: Michał Jagiełło Morgan Richomme test carefully before submitting until we got a gate to do it...
        1. not really an action point but the advise is wise...be carefull after removing the submodules + change the xtesting installation (using standard python module not developer way...xtesting-onap shall follow)
      6. AP6: Krzysztof Kuzmicki prepare a 10 minutes demo on pnf-simu to nf-simu (explaning the graph shared by mail)
        1. done
      7. AP7: Marcin Przybysz plan a demo of VID new UI, how to launch it, how it is used in PNF macromode + see with VID how we can manage tests that are using legacy VID..
        1. done
      8. AP8: Morgan Richomme contact Kenny Paul  for the status of the slack chan
        1. done it is a free chan, no idea how LF decides to pay for a non free chan..if community thinks that we shall be able to keep the history and then take a non free chan, it is possible to report it to the TSC
  2. Admin
    1. A1 policy enforcement created
      1. https://gerrit.linuxfoundation.org/infra/c/releng/info-master/+/66923
    2. nf repo created (Krzysztof Kuzmicki )
    3. M3 is over..JIRA review needed to know what is possible (support/test cases/) what shall be postponed...
      1. release cadence is new, some projects were surprised. No new code will be accepted in OOM gate. Lots of patches submitted...gating system under pressure..wait for merge to reach the RC0
  3. Honolulu (M3 is over..)
    1. CSIT issue with python2.7/Python3 (Lasse Kaihlavirta Morgan Richomme )
      1. fix submitted by Former user (Deleted)
      2. list of jenkins images  known in CSIT repo but not sure it is documented. Some issues due to python 2.7/3.5/3.6/3.7/3.8/3.9...lib dependency issues NB the seccom recommended version is the 3.9
    2. Refactoring of testsuite
      1. python2.7/micro service/root
      2. cert
      3. helm chart
      4. Lukasz Rajewski and Andreas Geißler suggest to put the effort on migrating to pythonsdk rather than maintaining the pod
      5. Morgan Richomme reminds that robot tests are run through pythonsdk, the question is linked to the robot pod not the ability to run robotframework test cases, which is always possible. Who is using the e2e scripts that trigger processing on the robot pod. Lasse Kaihlavirta indicates that some usecases (under upstreaming (A1)) are in this case. For Honolulu we may ask for a waiver and remove the pod in Istambul (is nobody is using it). Morgan Richomme sent a mail to the community  but no feedback so far. Scaleout and vFWCL still probably requires the pod, other usercases requires also the init offered by the bash scripts #action Andreas Geißler look in the doc which use cases explicitely make reference to the scripts in the official documentation
    3. status on CI/CD (delta guilin) (Morgan Richomme ) 
      1. New Nokia dualstack lab added in the pool of lab reporting results http://testresults.opnfv.org/onap/api/v1/results?last=5&case_name=full&pod=ci-dualstack-master-daily
        1. IPv4/IPv6 server, ONAP Master daily => results shared with the community in LF backend + Test DB
      2. shall we update http://testresults.opnfv.org/onap-integration/ ?
        1. Daniel Milaszkiewicz OK #action morgan add Nokia in http://testresults.opnfv.org/onap-integration/
      3. Master is currently not that bad: https://logs.onap.org/onap-integration/daily/onap_daily_pod4_master/03-01-2021_06-03/
        1. error due to internal cert executed in onap ns (shall be moved to an external ns)
        2. 1 error on enhanced policy => policy anticipated the OOM merge and already adapted the test, it shall be back OK once the new policy dockers woudl be merged
        3. 1 error in E2E due to SDC (exception not caught)
        4. lots of patch in OOM gate..if lots of errors it is not normal as the master is reasonably stable...
        5. #action all help reviewing the OOM patches to give Integration feedback
      4. Guilin is not good
        1. same CI errors in DT and Orange
        2. runnign the tests directly is usually good => issue in the way to launch the test in guilin
        3. "old" ansible version in the xtesting-onap branch
        4. tests in progress to use xtesting-onap master (assuming that the test version is provided by a env variable already) => no need of specific branch
      5. delta guilin/current master shared with the community
        1. https://docs.google.com/spreadsheets/d/1t3GNRtabdkVxG4ZAxqBJ-X7OO-Zv6xRwd8KrR52V9zU
        2. as usual some projects anticipate and provided features continuously (Policy, DCAE, NBI, OOF, OOM), some did not...which explains the gating bottlneck due to the M3
    4. security test update
      1. tern (Former user (Deleted) )
        1. last results: https://logs.onap.org/onap-integration/daily/onap_weekly_pod4_master/02-25-2021_02-02/security/tern/
        2. patch submitted to fix some projects ..but they have no gating so manual test needed
          1. first patch submitted
          2. advantage, it will be easier to chase CVE if we can rely on baseline images
          3. issues with Python (and lib dependencies) but few components dealing with python
        3. next steps work on teh integration as a weekly job in xtesting-onap to run automatically tern everyweek
      2. versions still some issue in weekly chain + reporting to be integrated
        1. patch merged for the bug
        2. results of last week shared => https://logs.onap.org/onap-integration/daily/onap_weekly_pod4_master/02-25-2021_02-02/security/versions/versions/index.html
      3. core/internal nodeports tests can be long ..any idea how to parallelize
        1. wait for Paweł Wieczorek come back as same issue than for versions
    5. Backup & restore
      1. postponed to next week
    6. simu question on the availability of the E// pnf simu availability in nexus AJAY SINGH
      1. answer sent by mai, changes done broke the built chain => after 40 days the snampshot images are removed, as the image was not released and not rebuilt, expiration date was reached => need to fix the build or to release an old version (pretty sure that jenkins history is also cleaned..so probably not possible)
  4. AoB
  5. Action points for next meeting


  1. Action point follow-up
  2. Admin
  3. Honolulu
    1. demo simulator Core NSSMF simulator (Zhang Min ) 20'
    2. demo CSIT of VNF/NFVO package management and LCM  Gareth Roper , Andrew Lamb Byung-Woo Jun 10'
    3. nf-simulator (pnf-simu evolution) Krzysztof Kuzmicki 10'
    4. VID new UI Marcin Przybysz VID_MODERN_UI.pptx
  4. AoB

  1. Action point follow-up
    1. AP1: morgan_orange publish video and slideck on the how to
      1. done: Youtube video link, discussions in progress at LFN level to suggest how to publish such videos per community channel
    2. AP2: morgan_orange plan a new slot with praveen
      1. no feedback from praveen v , need to replan a slot
    3. AP3: morgan_orange send mail to suggest repo, and if OK => create teh repo
      1. mail sent to Konrad Bańka , wait for green light before creating the new repo
    4. AP4: morgan_orange anser to the mail for pnf-simu evolution and plan a formal agreement from committer next week
      1. see next section
    5. AP5: natacha add the _v to manage the settings path in order to retrieve the pythonsdk logs
      1. done but side effect due to the fact that the settings in master are in /Src/onaptests/src/onaptests and not under /usr/lib/python3.8 in xtesting
  2. Admin
    1. new ietf repo created, Bartek Grzybowski added JS linter, first upload in progress
    2. Guilin CI chain
      1. ~ OK http://testresults.opnfv.org/onap-integration/results/index-onap_daily_pod4_guilin-ONAP-oom.html
      2. Certificates CDS-UI expires in 3 days and VID in 29 days
      3. Error due to 500 from SDC
    3. Master chain
      1. pythonsdk tests broken during 2-3 days due to the introduction of onapsdk 7.4 (onapsdk custom exceptions were no more caught in onaptests)
      2. problem affecting all the gating chains
    4. Comments from Lukasz Rajewski on the slack channel => quid about the history, free account does not provide long retention period #action 8
  3. Honolulu
    1. Discussion on a recovery pipeline / resiliency tests (Marek Szwałkiewicz Morgan Richomme
      1. Epic created on Honolulu Resiliency and Backup and Restore test
        1. several tasks to perform tests
        2. Please do not hesitate to use the JIRA if you are performing such tests in your lab
    2. Discussion on CI/CD chain evolution (Morgan Richomme )honolulu_ci_evolution_08022021.pdf
      1. Krzysztof Kuzmicki indicates that dcae-mod shall be tested also through a new healthcheck test
      2. FWCL not possible to integrate in CI straightforward - preprovisionned resources needed, dependencies on installation (documented),
      3. Not sure if FW (macromode) will bring more than basic_VM + PNF and note sure Andreas Geißler will have resource to deal with it
      4. VCPE TOSCA (CLI) was not possible to run it on daily labs (running onwly on tester lab), improvements were planned in Guilin (seems to be done). #action 1
      5. #agreed No objection from the new categories and new tests for Honolulu branch
      6. for tern/scancode: watch out the versions of the OS, Debian9 is too old. #action 2
    3. Security
      1. versions: last update
        1. test created by Paweł Wieczorek
        2. waiver list introduced in Honolulu (patch merged)
        3. New GUI suggested more explicit that the json file: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/02-11-2021_22-02/security/versions/versions/
          1. still lots of java 8 and python 2.7...
          2. very few projects seem to use the baseline images (that are up to date..)
    4.  Pythonsdk
      1. removal of the submodule?
      2. #agree remove the pnf submodule as soon as possible, be careful with the side effects on xtesting dockers / xtesting-onap ansible roles as we coudl move back from src/onaptests to /usr/lib/python3.8 path
      3. status on the wrapper (helm2/helm3)
        1. Illia Halych indicates that there is a client supporting helm2 and one supporting helm3 but not both
        2. #agree consider only helm3
        3. Add the statement in the README (as for the python min version 3.7 (same as pythonsdk)
        4. #action 3
      4. status on the tests basic_pnf, basic_clamp
        1. problem for basic_pnf seems to be fixed, patch merged on SDC
        2. see if the basic_pnf could be introduced in CI (test it first manually on daily weekly then add teh job xtesting-onap project that defines the CI chain) #action 4
      5. Improve reporting
        1. reporting may be misleading as it indictaes only the results of resource creation. We recently got an issue when deleting the resources => all the reporting is green even if there was an error during vfModule deletion. Michał Jagiełło submitted a patch to consider deletion in reporting #action 5
      6. Solution to limit issues on master gate (gating on pythonsdk?, versioning,..)
    5. pnf-simu > nf-simu: formal review
      1. #agreed
      2. demo to be planned next week #action6
  4. AoB
    1. Marcin Przybysz indicates that a discussion is needed on VID. Vid new UI is needed for PNF macromode tests but old UI is needed for vFWCL..how do we manage that
    2. Michał Jagiełło tried to setup the new VID uI (VID 7.0.2) but has issue, Marcin Przybysz indicates that some flags have to be tuned #action 7
  5. actions for next meeting
    1. AP1: Morgan Richomme : contact vcpe tosca team to get their position in Honolulu ()
    2. AP2: Former user (Deleted) : get the list of projects using the baseline images
    3. AP3: Illia Halych amend https://gerrit.onap.org/r/c/testsuite/pythonsdk-tests/+/117192 for helm3 support
    4. AP4: Morgan Richomme test basic_pnf on Orange daily master with onaptests:master
    5. AP5: Michał Jagiełło Morgan Richomme test carefully before submitting until we got a gate to do it...
    6. AP6: Krzysztof Kuzmicki prepare a 10 minutes demo on pnf-simu to nf-simu (explaning the graph shared by mail)
    7. AP7: Marcin Przybysz plan a demo of VID new UI, how to launch it, how it is used in PNF macromode + see with VID how we can manage tests that are using legacy VID..
    8. AP8: Morgan Richomme contact Kenny Paul  for the status of the slack chan


  1. Action point follow-up
  2. Admin
  3. Honolulu
    1. demo 5g-core-nf simu (praveen v ) 20'
    2. demo ORAN A1-based Closed Loop (Konrad Bańka ) 20' A1 Policy Enforcement PoCv1.pptx
    3. demo stability test launcher (Natacha Chéreau )10'
  4. AoB

(18:02:16) collab-meetbot: Meeting ended Wed Feb 10 17:02:15 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
(18:02:16) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_10_2_2021/onap-int-integration_weekly_meeting_10_2_2021.2021-02-10-16.44.html
(18:02:16) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_10_2_2021/onap-int-integration_weekly_meeting_10_2_2021.2021-02-10-16.44.txt
(18:02:16) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_10_2_2021/onap-int-integration_weekly_meeting_10_2_2021.2021-02-10-16.44.log.html

  1. Action point follow-up
  2. Admin
    1. new repo follow-up
  3. Honolulu
    1. demo PNF (Michał Jagiełło )
    2. discussion on simulators
    3. simulator wrapper in pythonsdk-tests (Illia Halych)
  4. Preparation vEvent
  5. AoB

(18:57:32) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_27_01_2021/onap-int-integration_weekly_meeting_27_01_2021.2021-01-27-17.29.html
(18:57:32) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_27_01_2021/onap-int-integration_weekly_meeting_27_01_2021.2021-01-27-17.29.txt
(18:57:32) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_27_01_2021/onap-int-integration_weekly_meeting_27_01_2021.2021-01-27-17.29.log.html

  1. Action point follow-up
  2. Admin
    1. new repo follow-up
    2. new Real time support channel: slack (already exists), publick rocket chat, mattermost, ....
  3. Honolulu
    1. demo PNF (Michał Jagiełło )
    2. discussion on simulators
    3. simulator wrapper in pythonsdk-tests (Illia Halych)
    4. CSIT redesign updates (Lasse Kaihlavirta)
    5. stability/resiliency testing (Natacha Chéreau )
    6. Update on Tern tests (Former user (Deleted) )
  4. Preparation vEvent
  5. AoB

(17:40:10) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_20_01_2021/onap-int-integration_weekly_meeting_20_01_2021.2021-01-20-16.06.html
(17:40:10) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_20_01_2021/onap-int-integration_weekly_meeting_20_01_2021.2021-01-20-16.06.txt
(17:40:10) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_20_01_2021/onap-int-integration_weekly_meeting_20_01_2021.2021-01-20-16.06.log.html

  1. Action point follow-up
  2. Honolulu
    1. demo CDS / how it is used in Bell and how to upstream (JC ) 10' +10' discussions

    1. demo tests DataFileCollector (Krzysztof Gajewski ) 10'
  2. Guilin Maintenance Release
  3. Preparation vEvent
  4. AoB

(16:26:02) collab-meetbot: Minutes: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_13_1_2021/onap-int-integration_weekly_meeting_13_1_2021.2021-01-13-14.51.html
(16:26:03) collab-meetbot: Minutes (text): http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_13_1_2021/onap-int-integration_weekly_meeting_13_1_2021.2021-01-13-14.51.txt
(16:26:03) collab-meetbot: Log: http://ircbot.wl.linuxfoundation.org/meetings/onap-int/2021/integration_weekly_meeting_13_1_2021/onap-int-integration_weekly_meeting_13_1_2021.2021-01-13-14.51.log.html

Canceled quorum not reached (lots of public holidays)

Integration weekly meeting 2020 archives

Integration weekly meeting 2019 archives

Integration weekly meeting 2018 archives

Integration weekly meeting 2017 archives