ONAP Application Logging Specification v1.3 (Frankfurt)

ONAP Application Logging Specification v1.3 (Frankfurt)

About This Document

Official R1 documentation snapshot in  https://onap.readthedocs.io/en/latest/submodules/logging-analytics.git/docs/

This document specifies logging conventions to be followed by ONAP component applications.  

ONAP logging is intended to support operability, debugging and reporting on ONAP. These guidelines address:

  • Events that are written by ONAP components.

  • Propagation of transaction and invocation information between components.

  • MDCs, Markers and other information that should be attached to log messages.

    • MDC = Mapped Diagnostic Context

  • Human and machine-readable output format(s).

  • Files, locations and other conventions. 

Original AT&T ONAP Logging guidelines (pre amsterdam release) - for historical reference only: https://wiki.onap.org/download/attachments/1015849/ONAP%20application%20logging%20guidelines.pdf?api=v2

The Acumos logging specification follows this document at https://wiki.acumos.org/display/OAM/Log+Standards 

Logback reference: Logging Developer Guide#Logback.xml based on https://gerrit.onap.org/r/#/c/62405

Introduction

The purpose of ONAP logging is to capture information needed to operate, troubleshoot and report on the performance of the ONAP platform and its constituent components. Log records may be viewed and consumed directly by users and systems, indexed and loaded into a datastore, and used to compute metrics and generate reports. 

The processing of a single client request will often involve multiple ONAP components and/or subcomponents (interchangeably referred to as ‘application’ in this document). The ability to track flows across components is critical to understanding ONAP’s behavior and performance. ONAP logging uses a universally unique RequestID value in log records to track the processing of every client request through all the ONAP components involved in its processing.

A reference configuration of Elastic Stack is being deployed using ONAP Operations Manager since the amsterdam release - see usage in Logging Analytics Dashboards (Kibana)

This document proposes conventions you can follow to generate conformant, indexable logging output from your component.

Supported Languages

How to Log

ONAP prescribes conventions. The use of certain APIs and providers is recommended, but they are not mandatory. Most components log via EELF or SLF4J to a provider like Logback or Log4j.

Logging Specification Compliance

Logging Library Location and Use

What to Log

Context

MDCs

A Mapped Diagnostic Context (MDC) allows an arbitrary string-valued attribute to be attached to a Java thread via a ThreadLocal variable. The MDC's value is then emitted with each message logged by that thread. The set of MDCs associated with a log message is serialized as unordered name-value pairs (see #Text Output).

A good discussion of MDCs can be found at https://logback.qos.ch/manual/mdc.html

Example

From @Luke Parker's call graph work in https://git.onap.org/logging-analytics/tree/reference/logging-slf4j-demo

LogEntry(markers=ENTRY, logger=ComponentAlpha, requestID=eb3e0dc2-6c3c-4bb7-8ed6-e5cc4ec7aad2, invocationID=06c815ef-5969-45cc-b319-d0dbcde89329, timestamp=Tue May 08 04:23:27 AEST 2018)


Mapped Diagnostic Context Table

Legend

Legend

Green = Required field

Yellow = Optional field



Pipe Order

Name

Type

Group

Description

Required?

Y/N/C

(C= context dependent)

N = not required

L=Library provided



Notes

Code References

Pipe Order

Name

Type

Group

Description

Required?

Y/N/C

(C= context dependent)

N = not required

L=Library provided



Notes

Code References

1

LogTimestamp

log system



use %d field - see %d{"yyyy-MM-dd'T'HH:mm:ss.SSSXXX",UTC}

L





2

EntryTimestamp

MDC



if part of an ENTRY marker log

C





3

InvokeTimestamp

MDC



if part of an INVOKE marker log

C





4

RequestID
(pau)do we need to update this to TransactionID as per LOG-232 (discussed back in 20180426)?

MDC



UUID to track the processing of each client request across all the ONAP components involved in its processing

Y



In general

https://git.onap.org/logging-analytics/tree/reference/logging-slf4j-demo/src/test/java/org/onap/logging/ref/slf4j/demo/component/AbstractComponentTest.java

5

InvocationID

MDC



UUID correlates log entries relating to a single invocation of a single component

In the case of an asynchronous request, the InvocationID should come from the original request 

Y

See the comment section of the v1.2 spec spec on July 23, 2019



6

InstanceID

MDC



An unique ID to differentiate between multiple instances of the same (named) log writing service/application.  For example, either the Kubernetes pod ID or UUID can be used for this field.

Y

Was InstanceUUID



7

ServiceInstanceID

MDC



An unique identifier of a newly instantiated network service instance.

C





8

thread

log system



use %thread field

L





9

ServiceName

MDC



The service inside the partner doing the call - includes API name

Y





10

PartnerName

MDC



The identification of the entity that made the request being served.  For a serving API that is authenticating the request, this should be the authenticated username or equivalent (e.g. an attuid or a mechid)

unauthenticated = The part of the URI specifying the agent that the caller used to make the call to the component that is logging the message.

authenticated = userid

  1. If an authenticated API, then log the userid

  2. Otherwise, if the HTTP header "X-ONAP-PartnerName" was provided, then log that (note: this was a direction that we seemed to be going but never completed)

  3. Otherwise, if the HTTP header "X-FromAppId" was provided, then log that

  4. Otherwise, if the HTTP header "User-Agent" was provided, then log that

  5. Otherwise, log "UNKNOWN" (since the field is currently required, something must be in it)

Y

See the comment section of the v1.2 spec spec on August 6, 2019



11

StatusCode

MDC



This field indicates the high level status of the request - one of (COMPLETE, ERROR, INPROGRESS)



Y



20180807: expand from 2 fields to add "INPROGRESS"

addresses @Chris Lott question on https://wiki.acumos.org/display/OAM/Log+Standards

12

ResponseCode

MDC



This field contains application-specific error codes.

In the case where 

Y





13

ResponseDesc





This field contains a human readable description of the ResponseCode

Y





14

level





%level

L





15

Severity

MDC



Logging level by default aligned with the reported log level - one of INFO/TRACE/DEBUG/WARN/ERROR/FATAL

Y





16

ServerIPAddress







C





17

ElapsedTime







C





18

ServerFQDN

MDC



The VM FQDN if the server is virtualized. Otherwise the host name of the logging component.

Y





19

ClientIPAddress

MDC



This field contains the requesting remote client application’s IP address if known. Otherwise empty.

Y





20

VirtualServerName







C

ServerFQDN supercedes VirtualServerName as mentioned in the comment section of the v1.2 spec on July 3, 2019.
https://lf-onap.atlassian.net/browse/LOG-1163



21

ContextName







C

The logging enhancement team could not find any definition for this field and it was agreed to leave out the description for this field.  See comment section of the v1.2 spec on July 23, 2019.



22

TargetEntity

MDC



The name of the ONAP component or sub-component, or external entity, at which the operation activities captured in this metrics log record is invoked.

C





23

TargetServiceName

MDC



The name  of the API or operation activities invoked (name on the remote/target application) at the TargetEntity.  

C





24

TargetElement

MDC



VNF/PNF context dependent - on CRUD operations of VNF/PNFs

The IDs that need to be covered with the above Attributes are

  • VNF_ID OR VNFC_ID : (Unique identifier for a VNF asset that is being instantiated or that would generate an alarms)

  • VSERVER_ID OR VM_ID (or vmid): (Unique identified for a virtual server or virtual machine on which a Control Loop action is usually taken on, or that is installed  as part of instantiation flow)

  • PNF : (What is the Unique identifier used within ONAP?)

C





25

User

MDC



User - used for %X{user}  

C





26

p_logger

log system



The name of the class doing the logging (in my case the ApplicationController – close to the targetservicename but at the class granular level - this field is %logger

L





27

p_mdc

log system



allows forward compatability with ELK indexers that read all MDCs in a single field - while maintaining separate MDCs above.



The key/value pairs all in one pipe field (will have some duplications currently with MDC’s that are in their own pipe – but allows us to expand the MDC list – replaces customvalue1-3 older fields - this field is %mdc

L





28

p_message

log system



Standard attribute - defined in logback.xml - Message - used for %msg%

L







RootException

log system



%rootException - Dublin spec only

L





29

p_marker

log system



The marker labels INVOKE, ENTRY, EXIT – and later will also include DEBUG, AUDIT, METRICS, ERROR when we go to 1 log file - this field is %marker

L