LOG Meeting Minutes 2017-11-07
Agenda
Review guidelines in general
Review line formats
Comparison of line formats: from Luke
Before:
11/08-02:11:28.213|org.onap.example.component1.subcomponent1.LogbackTest|b88319fb-966e-4eb9-ad7f-a604da21c55a||main|||INFO||||||here's an info it has newlines and tabs
After
org.onap.example.component1.subcomponent1.LogbackTest 2017-11-07T15:11:28.213Z INFO here's an info\nit has newlines\tand tabs key1=value1, key2=value2 with space, RequestId=b88319fb-966e-4eb9-ad7f-a604da21c55a, key5=value5"with"quotes, key6=value5>with<chevrons, key3=value3\nwith\nnewlines, key4=value4\twith\ttabs main
TODO: need to correlate the position of the thread name "main" in its new position in the guidelines, also the MDC comma separated block. We need to specify the exact tab structure preceding the mdc block
Introduce 2 new documentation contributors: Akanksha Khosla (OOM) , Anuj Kapoor (LOG)
discuss doc work - todo
discuss doc release notice work
discuss carrier grade work in OOM -OOM for Production-Grade Deployments around OOM-109: Platform Centralized LoggingClosed
JIRA review - https://jira.onap.org/secure/RapidBoard.jspa?rapidView=53&view=planning&selectedIssue=LOG-85
Triage log issues in other components, AAI, OOM - introduce Logging label
AAI-454: LoggingContext.requestId required NULL handling in aai/aai-common (20170607) - during demo.sh init_customerClosed, OOM-403: Logs are not properly initialized in SDC-FE podClosedDiscuss overall ONAP stability as it is affecting our log traceability - INT-106: Stabilizing the ONAP master branchClosed please share general SDC, MSO, AAI workarounds here
F2F Dec 2017 presentations - Transaction Traceability using ELK by James
Minutes
Discussed
Discuss delimiter, new line, mdc name/value pairs
Discuss producers (devs) as well as consumers (ops) AT&T and Bell - need to handle ops as well as NBI clients - from an indexing/parsing perspective
Discuss use case where indexing fails and ops reads the logs directly
Allow for default and new format toggling - could have dual appenders/or subdirs for both formats until R2? - see MSO for example in ONAP Log Locations
maintain backward compatabilty with EELF
Standardize in the future on logger names not different files
Need to rework standard directory issue
Review log eviction policy for emptyDir and dockerdata-nfs outputs
Check ELK container logstash HD
Compare overhead of logging over NFS - need some volumetrics state before/after log duplication
part of log audit - verify eelf appenders in logback.xml
Post Meeting
Epic for dual appenders (eelf, current) design issues, Resiliency/rotation changes, migration to single appender in the future - Mike
Epic for default configuration being applied
Finalize tabs vs pipes for R2 doc - vote on tue
Implementation (slf4j/eelf) is less of a concern over verifying that indexed content of the log format is standardized for consumers - however an additional API wrapper would benefit abstractONAP-component-wide changes over direct SLF4J calls.
Enumerate new ONAP projects as candidates for guideline enforcement.
Need volumetrics performance stats for end to end log emission all the way to ELK indexing/query - to be able to answer for more real time requestID tracking - like a DMaaP queue?
Committers have ideas about a 3rd appender to drop into the message queue DMaaP - a secondary, perhaps more performant view of the events.
Design Issue Epic for above LOG-94: Investigate DMaaP appender as 3rd appenderClosed
AOP annotation support for Logging - check for existing code and raise (custom EELF annotations on top of Spring ?) - LOG-95: Add AOP logging annotation supportClosed
Rollover Actions
From last week
vote on ways to edit the guidelines next meet
vote meeting agenda flow - guide first, JIRA second ?
onap config is split - oom is working merging heat and oom
(oom config in oom project, heat it is in each project distributed across components)
discuss (raise epic) combining config - or raise a hierarchy (oparent?) - default version with specific overrides if required per project
need to discuss eelf consistency and MDC details for next meeting
LOG-85 discuss external mounts for /dockerdata-nfs shares and how to get it to work with emptyDir()
upfront arch discussion on implementation changes in the JIRA architecture page for epics in R2
small demo of functionality - proposals
f2f - discussions of logging exposure in the talks/hands-on/discussion-panels
vFirewall integration testing (daily) - always asking for logs in HEAT testing - we could address this in R2 either either through HEAT ELK or when OOM gets consumed in R2
2 logging related NPE bugs - we can fix these as well - we have 2 weeks
Actions
as above
Attendees -
...