LOG Meeting Minutes 2017-12-05

Owned by Michael O'Brien
Last updated: Dec 05, 2017
4 min read

Agenda

  • Get branch for docs in

  • Backlog/Board https://jira.onap.org/secure/RapidBoard.jspa?rapidView=53&view=planning&selectedIssue=LOG-85

  • Guidelines exposure on onap-discuss need examples

  • JIRA work on OOM for LOG 

  • Discuss Integration testing of OOM in Beijing - LOG is part of this

  • Discuss vFW status

  • Discuss F2F

  • Discuss KubeCon

    • Discuss new members: @Shishir Thakore @sanjay Agraharam

  • Discuss Alexis work stabilizing release-1.1.0 oom

  • Discuss work moving log config out of the config container - MSO prototype

  • Committer review for the TSC - 1 or 2 more possible committer

  • New committers discussion in prep for Beijing

  • Focus on requestId flow for the new 
    Notes

  • post new committer template

  • Request for @Dave Williamson as committer for next meeting - michael

  • Borislav raising sync issue for OOM and Logging sync - 

    • check on lost logging configuration on the logging-analytics repo moved to oom 

    • specifically docker/src in the config container

    • Fix for current 1.1.0 

    • mso jetty container inside the war (logger) has issues unless put in after war explode - verify post start hook

    • OOM Meeting Notes - 2017-12-06

  • how to verify this in the future - wider review and doing a deployment - at least one review - need Acceptance Testing - and post results to the JIRA

  • review all commits on logging since the 22nd to 1.1 - check the jiras - likely reopen at least one of them - start with a regression test suite for logging run on the daily CD before cherry picking to the other branch.


Analysis of the onap/ecomp issue with the ELK stack raised by Borislav

Analysis of the onap/ecomp issue with the ELK stack raised by Borislav

A lot of the fixes since Dec 30 were done during the 3 day blitz to stabilize ONAP in prep of KubeCon and the F2F - during validating the vFW

Message: Regression test – even though MSO flows did not work until the change – we can still run rest calls to force out logs



If you look at some of the merges – we might get hammered on being light on reviewers for master – but you would need to know this was a cherry picked form 1.1 where the fixes were done first - and tested on 2 systems –which does have all the reviewers. The issue is a lack of regression testing for logs.

We need to have a plan to address changes that are approved but still cause design contention and are raised later – in the logging meeting the issue is phrased more of a design refactor issue not something that is actually fully broken - again need to retest the MSO logs - which we can now that vf-module creation works because of @Alexis de Talhouët

https://gerrit.onap.org/r/#/q/status:merged+project:+oom

For example

https://gerrit.onap.org/r/#/c/25053/

was flagged as busted – but reviewed as ok because a separate patch below reverted the onap/onap back to onap/ecomp for filebeat/logback

the fix

https://gerrit.onap.org/r/#/c/25219/1

fixed this issue and was signed off – if we are talking about another change then we may need to create a new JIRA for it when we identify it

I will add an agenda item to the OOM meeting tomorrow to get this ongoing issue resolved – as in identify exactly the issue in a JIRA and add all reviewers before we adjust the yamls again –and get Borislav’s signoff.

Which brings up the fact that we likely need to consume public onap daily and run a log sanity on it asap – especially after a large number of commits and between the port to master.

We need a process for this before being put into integration testing after the F2F.

Overflow

  • Backlog/Board https://jira.onap.org/secure/RapidBoard.jspa?rapidView=53&view=planning&selectedIssue=LOG-85

  • Guidelines exposure on onap-discuss need examples

  • JIRA work on OOM for LOG - LOG-101 - OOM Logging fixes tracking Epic DONE
    Please code-review (Alexis is helping us out) -  OOM-462 - Adjust mso artifacts to latest (Amsterdam) SUBMITTED

  • Helm 2.7 upgrade from 2.3 reverted yesterday - 3 Log ELK containers back up for OOM consumers - OOM-427 - log containers fail to create since 20171213 - using tpl template function on helm 2.3 not 2.6 IN PROGRESS

  • Doc updates (R1 branch) -

    LOG-107 - Add wiki detailing diff between original PDF logging guidelines and current R2 wiki page IN PROGRESS

  • R1 vFW triage (all hands on) at 1200EDT daily until 6 Dec Kubecon and 11 Dec ONAP F2F - for OOM and HEAT
    Please join the call:  and contribute anything vFirewall related or stabilizing the R1 branch for OOM or HEAT
    Vetted vFirewall Demo - Full draft how-to for F2F and ReadTheDocs



Notes

  • New Work Items:

  • Luke to work with doc team on high level what's new and MDC details as part of LOG-107 - Add wiki detailing diff between original PDF logging guidelines and current R2 wiki page IN PROGRESS

  • Check on closing sprint - michael

  • version using branch on our RTD - michael

  • Filebeat easier in HEAT because all work is in integration team - michael (get epic)

    • Discuss how to get http request/session key:value pairs all the way down to the DAO layer - REST API should handle this - verify the code abstraction from example JAX-RS 1/2
      (No manditory adoption: servlet filter - maybe no - give the projects an RI on a single project and reference this 

    • Need to audit all the project - unavoidable to enumerate what API is used for HTTP REST access - in the terms of plugging in our requestID passing

    • need to look at the code - 

need to review - and prioritize and merge with OOM R2 release - for resiliency (how is logging part of this) 
- OOM for Carrier Grade Deployments for example clustering persistence LOG R2 (Beijing) Candidates - for example how do we handle the distributed FS on a distributed ELK stack
both ELK infrastructure and a scaled SDNC cluster (a filebeat per instance and its PV)