Logging Dublin Scope
- Michael O'Brien
- 1 High Level
- 1.1 Features
- 1.1.1 Logging Alignment before enhancement
- 1.1.2 Open Standards
- 1.1.3 Security
- 1.1.4 Infrastructure Improvements
- 1.1.5 HELM Hooks for Configuration
- 1.1.6 Monitoring
- 1.1.7 S3P
- 1.1.8 3rd party consumption alignment
- 1.1.9 Dublin Alignment
- 1.2 Deployment Model
- 1.3 Epics
- 1.4 Notes
- 1.1 Features
High Level
Features
see ONAP Application Logging Specification - Post Dublin
Follows deprecated - Logging Casablanca Scope
Key words are: out of the box, reuse components, iterative changes - not major spec changes.
There is one primary use case and one enabler use case
UC1: Provide for transaction tracing via search or dashboard for distributed transactions across ONAP components - enabled by the logs shipped to the ELK stack - in real time. (the ELK stack is there - we just need to adjust log content - amount and format) UC2: Provide for UC1 tracing via standardized logs - ideally via marker/mdc (label/key:value pair) markup |
Logging Alignment before enhancement
Plan is to take the portal/sdk library - align the rest of onap to this library and then add marker/mdc (labels and key/value pair) support as required in phase 2
Work will include 1) verify pipeline 2) verify logback format
Open Standards
Look at a standard that will allow us to use standard parsing libraries all the way to eventual AI/ML by adopting a standard like open tracing.
Security
REST and HTTPS security
LOG-494: Use Search Guard Community Edition for TLS REST encryptionClosed
Infrastructure Improvements
including filebeat sidecar anti-pattern/replacement, elk stack upgrade, deployment template, library language scope (python, swift...)
HELM Hooks for Configuration
https://github.com/helm/helm/blob/master/docs/charts_hooks.md
Monitoring
Investigate out of the box prometheus monitoring
S3P
backup/restore, security, log format monitoring, elk dashboards
3rd party consumption alignment
Lower in priority will be to align onap with requirements of Acumos and ECOMP for example - the focus will be on alignment of ONAP components only - to the primary goal of providing a transaction tracing system via the ELK stack.
Dublin Alignment
Release 4 (Dublin) Service Providers (EUAG) Requirements / Proposals
Deployment Model
In progress - was a one time casablanca static dashboard - will do an automated one with the Bell, Orange and Samsung guys
Epics
https://lf-onap.atlassian.net/browse/LOG-707
Commit or Stretch goal | Feature | Owner | Status | % complete | EPIC | Notes |
|---|---|---|---|---|---|---|
commit | bugs | 60% | SO-1110: Re-add Filebeat sidecars to SO after 2-9 container refactorClosed | |||
6.3 upgrade | 0% | Acumos and Clamp driven - need plugin/module support in 6.3 verify new filebeat version works verify indexes need upgrade strategy for future - index migration | ||||
commit | schema | 50% | ||||
commit | search guard | LOG-494: Use Search Guard Community Edition for TLS REST encryptionClosed | 20181113 added to scope in | |||
stretch | format standards for tracing | LOG-104: Investigate Jaeger / opentracing / zipkin distributed tracing agent/serverClosed | ||||
stretch | analytics | LOG-500: Machine Learning on ONAP Logs - streamed and bulk ML processingClosed | ||||
shipping infrastructure | @Michael O'Brien | starting | 20% |