/
LOG Meeting Minutes 2019-06-25

LOG Meeting Minutes 2019-06-25

Owned by Prudence Au
Last updated: Jun 25, 2019 by Luke Parker
2 min read

Agenda

  • Dublin release - made good process for the past couple of weeks but still waiting for the release notes update from 3 teams

  • After some effort for the past week or so, logging project (include POMBA) is good for Dublin sign-off

  • El Alto will be ā€œReduce Internal Debtā€Ā release, will be grooming the backlog within this week.Ā Ā 

  • POMBA Dublin Demos

  • use cases to showcase the capability of logging and logging specification (please use the comment section for your ideas)

Attendees

  1. @Artem NaluzhnyyĀ 

  2. @Dave WilliamsonĀ 

  3. @Lorraine A. WelchĀ 

  4. Sanjay Agraharam

  5. @Luke ParkerĀ 

Notes

  • @Prudence AuĀ not available today.

  • Reviewed Lorraine's analysis of POMBA logs:

    • An encouraging number of attributes populated.

    • InvocationID possibly unset.

    • Looked like the provider config still explicitly specified MDCs by name with %X{name}, so that's a pre-Amsterdam configuration.

  • Canonical examples:

    • No truly canonical examples. POMBA is probably among the best.

    • We've had past success identifying flows from SO/APPC/SDNC output:

      • What they execute is of interest in analytics (in a way that SDC, for example, is less interesting).Ā 

      • It took a lot of finessing, though, to allow for differences in naming, missing data, etc.

      • Action onĀ @Luke Parker to do a little analysis and summarize at the next meeting.

    • Regarding what canonical looks like:

      • The spec:

        • Is still daunting.Ā 

        • Understandably nobody is eager to reopen this issue.

        • But it probably couldn't hurt to separate out and emphasize the 1/3 (~10) or so that are critical.

      • The pipeline:

        • Filebeat was meant to be temporary, in response to:

          • EELF, and no authority to bypass logfiles.

          • Heterogeneous logging provider types and versions.

          • The impossibility of getting fancier transports into the classpath for every provider in every Java runtime in every container.Ā 

        • But if we can reconfigure the pipeline, then:

          • The output format won't necessarily matter, because we can ship logs in a structured format: syslog, logstash, JSON, etc.Ā 

          • Then Filebeat can be optional, and maybe can even cease to be enabled by default.

          • This has a much better ROI than fussing over formats.Ā 

          • (It's noteworthy that we've made it so far without even minimal extension points).

        • Spoke to @Borislav GlozmanĀ and @Mike ElliottandĀ @Avi ChapnickĀ fromĀ  OOM about this:Ā 

          • About making the Filebeat sidecar optional in deployment manifests.Ā 

          • About allowing provider configuration docs to be replaced (and thereby to enable other transports):

            • Individually.Ā 

            • En masse.Ā 

          • And their response was that they're working on generating manifests, which:

            • Could allow these kinds of changes to be made without nearly so much busy work.Ā 

            • But which also means coordination and delays.Ā 

          • Should try to get @Mike Elliott along next time.

  • Next meeting: July 9, 2019