LOG Meeting Minutes 2017-08-22

Owned by Luke Parker
Last updated: Aug 27, 2017
2 min read

Minutes:

  1. Filebeat deployment:

  2.  

    1. OOM deployment is proceeding with one Filebeat container per pod or (as a daemonset) per container. 

    2. Logs are initially being written to non-persistent volumes, making them vulnerable to loss. This is a provisional configuration only.

    3. Both were decisions by the architects for the implementation teams.

  3. Impact of Zipkin:

  4.  

    1. Confirmed to require in-container probes (→ ONAP-wide classpath changes), collector agents.

    2. That's seen as prohibitive. Will proceed with REST and MDC conventions for now.

  5. Tracing:

  6.  

    1. Continue to improve transactionID propagation.

    2. Limitations of X-FromAppID. 

    3. Intention is to identify and disambiguate invocations between components.

    4. Suggestion (as per diagram attached to last week's minutes):

    5.  

      1. Generate an invocationID per service call, logged as MDC. 

      2. Pass invocationID in REST invocations as per transactionID.

      3. Use Markers to:

      4.  

        1. Report relationship between caller invocationID and invoked invocationID. 

        2. Report return and/or exit.

        3. (Possibly other stuff like async execution).

      5. What this achieves:

      6.  

        1. Minimal impact; conventions, but no API changes.

        2. All of the above is captured in the Elastic Stack index, allowing an invocation graph to be construction.

    6. Question (Horace Ji):

      1. Does a suitable MDC already exist? 

      2. Usage is inconsistent, so I think we all agreed it's risky, and probably saves to invent something new. 

      3. (With MDCs logged as name-value pairs, there's no great cost to defining a new MDC).

    7. Suggestion (Dave Williamson):

    8.  

      1. Record relationship between caller and invoked elements at the caller end.

      2. Then:

        1. The invoking component issues an invocationID during invocation. (And records nothing new).

        2. The invoked component just sets the invocationID MDC wherever it presently sets the transactionID MDC.

    9. Suggestions (@Lee Breslau):

    10.  

      1. Use EELF audit log to record entry.

      2. Use EELF metric log to record exit.

  7. Logging wrappers/abstractions between the application and EELF. (Which we missed, sorry).

Actions:

  • Update on Filebeat deployment (@Luke Parker) 

  • M3 (API freeze) debrief (@Luke Parker):

    • No API changes! 

    • New documentation obligations. 

  • Logging abstractions (Dave Williamson).

  • Draft updated logging guide. (@Luke Parker).

  • Continue discussion of tracing options:

    • Suggestions above. 

    • Persisting invocationID and transactionID in cases like where execution may be deferred (e.g. BPMN). 

      • (I think this supports Dave's suggestion).