ONAP Application Logging Specification v1.3 (Frankfurt)
- 1 About This Document
- 2 Introduction
- 3 Supported Languages
- 4 How to Log
- 5 Logging Specification Compliance
- 6 Logging Library Location and Use
- 6.1 What to Log
- 6.2 Context
- 6.2.1 MDCs
- 6.2.1.1 Example
- 6.2.1.2 Mapped Diagnostic Context Table
- 6.2.1.3 Logging
- 6.2.1.4 Serializing
- 6.2.1.5 MDC-RequestID
- 6.2.1.6 MDC - InvocationID
- 6.2.1.7 MDC - InstanceID
- 6.2.1.8 MDC - PartnerName
- 6.2.1.9 MDC - ServiceName
- 6.2.1.10 MDC - StatusCode
- 6.2.1.11 MDC - ResponseCode
- 6.2.1.12 MDC - Severity
- 6.2.1.13 MDC - ServerFQDN
- 6.2.1.14 MDC - ClientIPAddress
- 6.2.1.15 MDC - EntryTimestamp
- 6.2.1.16 MDC - InvokeTimestamp
- 6.2.1.17 MDC - TargetEntity
- 6.2.1.18 MDC - TargetServiceName
- 6.2.1.19 MDC - TargetElement
- 6.2.1.20 MDCs - the Rest
- 6.2.1.21 Deprecation
- 6.2.2 Examples
- 6.2.3 Output Format
- 6.2.1 MDCs
- 6.3 Output Location
- 6.4 Configuration
- 6.4.1 Locations
- 6.4.2 Reconfiguration
- 6.4.3 Overrides
- 6.4.4 Archetypes
- 6.5 Retention
- 7 Types of EELF Logs
- 8 New ONAP Component Checklist
- 9 What's New
- 10 Pending Specification Work
- 11 Developer Guide
About This Document
Official R1 documentation snapshot in https://onap.readthedocs.io/en/latest/submodules/logging-analytics.git/docs/
This document specifies logging conventions to be followed by ONAP component applications.
ONAP logging is intended to support operability, debugging and reporting on ONAP. These guidelines address:
Events that are written by ONAP components.
Propagation of transaction and invocation information between components.
MDCs, Markers and other information that should be attached to log messages.
MDC = Mapped Diagnostic Context
Human and machine-readable output format(s).
Files, locations and other conventions.
Original AT&T ONAP Logging guidelines (pre amsterdam release) - for historical reference only: https://wiki.onap.org/download/attachments/1015849/ONAP%20application%20logging%20guidelines.pdf?api=v2
The Acumos logging specification follows this document at https://wiki.acumos.org/display/OAM/Log+Standards
Logback reference: Logging Developer Guide#Logback.xml based on https://gerrit.onap.org/r/#/c/62405
Introduction
The purpose of ONAP logging is to capture information needed to operate, troubleshoot and report on the performance of the ONAP platform and its constituent components. Log records may be viewed and consumed directly by users and systems, indexed and loaded into a datastore, and used to compute metrics and generate reports.
The processing of a single client request will often involve multiple ONAP components and/or subcomponents (interchangeably referred to as ‘application’ in this document). The ability to track flows across components is critical to understanding ONAP’s behavior and performance. ONAP logging uses a universally unique RequestID value in log records to track the processing of every client request through all the ONAP components involved in its processing.
A reference configuration of Elastic Stack is being deployed using ONAP Operations Manager since the amsterdam release - see usage in Logging Analytics Dashboards (Kibana)
This document proposes conventions you can follow to generate conformant, indexable logging output from your component.
Supported Languages
How to Log
ONAP prescribes conventions. The use of certain APIs and providers is recommended, but they are not mandatory. Most components log via EELF or SLF4J to a provider like Logback or Log4j.
Logging Specification Compliance
Logging Library Location and Use
What to Log
Context
MDCs
A Mapped Diagnostic Context (MDC) allows an arbitrary string-valued attribute to be attached to a Java thread via a ThreadLocal variable. The MDC's value is then emitted with each message logged by that thread. The set of MDCs associated with a log message is serialized as unordered name-value pairs (see #Text Output).
A good discussion of MDCs can be found at https://logback.qos.ch/manual/mdc.html.
Example
From @Luke Parker's call graph work in https://git.onap.org/logging-analytics/tree/reference/logging-slf4j-demo
LogEntry(markers=ENTRY, logger=ComponentAlpha, requestID=eb3e0dc2-6c3c-4bb7-8ed6-e5cc4ec7aad2, invocationID=06c815ef-5969-45cc-b319-d0dbcde89329, timestamp=Tue May 08 04:23:27 AEST 2018)
Mapped Diagnostic Context Table
Must be set as early in invocation as possible.
Must be unset on exit.
keep in sync with https://wiki.acumos.org/display/OAM/Log+Standards
Legend |
|---|
Green = Required field |
Yellow = Optional field |
Pipe Order | Name | Type | Group | Description | Required? Y/N/C (C= context dependent) N = not required L=Library provided | Notes | Code References |
|---|---|---|---|---|---|---|---|
1 | LogTimestamp | log system | use %d field - see %d{"yyyy-MM-dd'T'HH:mm:ss.SSSXXX",UTC} | L | |||
MDC | if part of an ENTRY marker log | C | |||||
MDC | if part of an INVOKE marker log | C | |||||
RequestID | MDC | UUID to track the processing of each client request across all the ONAP components involved in its processing | Y | In general | |||
MDC | UUID correlates log entries relating to a single invocation of a single component In the case of an asynchronous request, the InvocationID should come from the original request | Y | See the comment section of the v1.2 spec spec on July 23, 2019 | ||||
MDC | An unique ID to differentiate between multiple instances of the same (named) log writing service/application. For example, either the Kubernetes pod ID or UUID can be used for this field. | Y | Was InstanceUUID | ||||
7 | ServiceInstanceID | MDC | An unique identifier of a newly instantiated network service instance. | C | |||
8 | thread | log system | use %thread field | L | |||
MDC | The service inside the partner doing the call - includes API name | Y | |||||
MDC | The identification of the entity that made the request being served. For a serving API that is authenticating the request, this should be the authenticated username or equivalent (e.g. an attuid or a mechid) unauthenticated = The part of the URI specifying the agent that the caller used to make the call to the component that is logging the message. authenticated = userid
| Y | See the comment section of the v1.2 spec spec on August 6, 2019 | ||||
MDC | This field indicates the high level status of the request - one of (COMPLETE, ERROR, INPROGRESS) | Y | 20180807: expand from 2 fields to add "INPROGRESS" addresses @Chris Lott question on https://wiki.acumos.org/display/OAM/Log+Standards | ||||
MDC | This field contains application-specific error codes. In the case where | Y | |||||
13 | ResponseDesc | This field contains a human readable description of the ResponseCode | Y | ||||
14 | level | %level | L | ||||
MDC | Logging level by default aligned with the reported log level - one of INFO/TRACE/DEBUG/WARN/ERROR/FATAL | Y | |||||
16 | ServerIPAddress | C | |||||
17 | ElapsedTime | C | |||||
MDC | The VM FQDN if the server is virtualized. Otherwise the host name of the logging component. | Y | |||||
MDC | This field contains the requesting remote client application’s IP address if known. Otherwise empty. | Y | |||||
20 | VirtualServerName | C | ServerFQDN supercedes VirtualServerName as mentioned in the comment section of the v1.2 spec on July 3, 2019. | ||||
21 | ContextName | C | The logging enhancement team could not find any definition for this field and it was agreed to leave out the description for this field. See comment section of the v1.2 spec on July 23, 2019. | ||||
MDC | The name of the ONAP component or sub-component, or external entity, at which the operation activities captured in this metrics log record is invoked. | C | |||||
MDC | The name of the API or operation activities invoked (name on the remote/target application) at the TargetEntity. | C | |||||
MDC | VNF/PNF context dependent - on CRUD operations of VNF/PNFs The IDs that need to be covered with the above Attributes are
| C | |||||
25 | User | MDC | User - used for %X{user} | C | |||
26 | p_logger | log system | The name of the class doing the logging (in my case the ApplicationController – close to the targetservicename but at the class granular level - this field is %logger | L | |||
27 | p_mdc | log system | allows forward compatability with ELK indexers that read all MDCs in a single field - while maintaining separate MDCs above. The key/value pairs all in one pipe field (will have some duplications currently with MDC’s that are in their own pipe – but allows us to expand the MDC list – replaces customvalue1-3 older fields - this field is %mdc | L | |||
28 | p_message | log system | Standard attribute - defined in logback.xml - Message - used for %msg% | L | |||
RootException | log system | %rootException - Dublin spec only | L | ||||
29 | p_marker | log system | The marker labels INVOKE, ENTRY, EXIT – and later will also include DEBUG, AUDIT, METRICS, ERROR when we go to 1 log file - this field is %marker | L |