Versions Compared

Old Version 126

changes.mady.by.user Toine Siebelink

Saved on

compared with

New Version Current

changes.mady.by.user Toine Siebelink

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

...

Original AT&T ONAP Logging guidelines (pre amsterdam release) - for historical reference only: https://wiki.onap.org/download/attachments/1015849/ONAP%20application%20logging%20guidelines.pdf?api=v2

The Acumos logging specification follows this document at https://wiki.acumos.org/display/OAM/Log+Standards 

...

Use a spring based Aspect library that emits Markers automatically around function calls and retrofit your code to log via Luke's SLF4J for internal log messages.

4853450616339021

Logging Library Location and Use

...

  1. By selection of a logging provider such as Logback or Log4j, typically via the classpath. 
  2. By way of a provider configuration document, typically logback.xml or log4j.xml. See 4853450616339021.

SLF4J

SLF4J is a logging facade, and a humble masterpiece. It combines what's common to all major, modern Java logging providers into a single interface. This decouples the caller from the provider, and encourages the use of what's universal, familiar and proven. 

...

Logback is the most commonly used provider. It is generally configured by an XML document named logback.xml. See 48534506 16339021.

See HELM template https://git.onap.org/logging-analytics/tree/reference/provider/helm/logback

...

Log4j 2.X is somewhat less common than Logback, but equivalent. It is generally configured by an XML document named log4j.xml. See 48534506 16339021.

Log4j 1.X

Strongly discouraged from Beijing onwards, since 1.X is EOL, and since it does not support escaping, so its output may not be machine-readable. See https://logging.apache.org/log4j/1.2/.

...

This is the most important item of information in most log messages. See 48534506 16339021 guidelines.

Internationalization

...

A Mapped Diagnostic Context (MDC) allows an arbitrary string-valued attribute to be attached to a Java thread via a ThreadLocal variable. The MDC's value is then emitted with each message logged by that thread. The set of MDCs associated with a log message is serialized as unordered name-value pairs (see 48534506 16339021).

A good discussion of MDCs can be found at https://logback.qos.ch/manual/mdc.html. 

...

Pipe OrderNameTypeGroupDescription

Applicable

(per log file)

Marker Associations

Moved

MDC

to


standard

attribute


Removed

(was in

older

spec)

Required?

Y/N/C

(C= context dependent)

N = not required

L=Library provided


DerivedHistoricalAcumos
ref		Use Cases

Code References

1LogTimestamplog system
use %d field - see %d{"yyyy-MM-dd'T'HH:mm:ss.SSSXXX",UTC}



L




2EntryTimestampMDC
if part of an ENTRY marker log



C




3InvokeTimestampMDC
if part of an INVOKE marker log



C




4

RequestID

MDC

UUID to track the processing of each client request across all the ONAP components involved in its processing





Y



In general

https://git.onap.org/logging-analytics/tree/reference/logging-slf4j-demo/src/test/java/org/onap/logging/ref/slf4j/demo/component/AbstractComponentTest.java

5InvocationIDMDC

UUID correlates log entries relating to a single invocation of a single component
In the case of an asynchronous request, the InvocationID should come from the original request 





Y




6InstanceIDMDC

UUID to differentiate between multiple instances of the same (named) log writing service/application





Y
was InstanceUUID


7ServiceInstanceIDMDC





C




8threadlog system
use %thread field



L




9ServiceName

The service inside the partner doing the call - includes API name





Y




10PartnerName

unauthenticated = The part of the URI specifying the agent that the caller used to make the call to the component that is logging the message.

authenticated = userid

  1. If an authenticated API, then log the userid
  2. Otherwise, if the HTTP header "X-ONAP-PartnerName" was provided, then log that (note: this was a direction that we seemed to be going but never completed)
  3. Otherwise, if the HTTP header "User-Agent" was provided, then log that
  4. Otherwise, log "UNKNOWN" (since the field is currently required, something must be in it)




Y

user

11StatusCode

This field indicates the high level status of the request - one of (COMPLETE, ERROR, INPROGRESS)






Y



20180807: expand from 2 fields to add "INPROGRESS"

addresses Chris Lott question on https://wiki.acumos.org/display/OAM/Log+Standards

12ResponseCode

This field contains application-specific error codes.



Y






13ResponseDesc

This field contains a human readable description of the ResponseCode



Y






14level

%level



L




15Severity

Logging level by default aligned with the reported log level - one of INFO/TRACE/DEBUG/WARN/ERROR/FATAL





Y

level (but numbers)

16ServerIPAddress






C




17ElapsedTime






C




18ServerFQDN

The VM FQDN if the server is virtualized. Otherwise the host name of the logging component.





Y






19ClientIPAddress

This field contains the requesting remote client application’s IP address if known. Otherwise empty.





Y






20VirtualServerName






C




21ContextName






C




22TargetEntity

The name of the ONAP component or sub-component, or external entity, at which the operation activities captured in this metrics log record is invoked.





C




23TargetServiceName

The name  of the API or operation activities invoked (name on the remote/target application) at the TargetEntity.  





C




24TargetElement

VNF/PNF context dependent - on CRUD operations of VNF/PNFs

The IDs that need to be covered with the above Attributes are

       -        VNF_ID OR VNFC_ID : (Unique identifier for a VNF asset that is being instantiated or that would generate an alarms)

       -        VSERVER_ID OR VM_ID (or vmid): (Unique identified for a virtual server or virtual machine on which a Control Loop action is usually taken on, or that is installed  as part of instantiation flow)

       -        PNF : (What is the Unique identifier used within ONAP)





C




25UserMDC
User - used for %X{user}  



C




26p_loggerlog system
The name of the class doing the logging (in my case the ApplicationController – close to the targetservicename but at the class granular level - this field is %logger



L




27p_mdclog system

allows forward compatability with ELK indexers that read all MDCs in a single field - while maintaining separate MDCs above.


The key/value pairs all in one pipe field (will have some duplications currently with MDC’s that are in their own pipe – but allows us to expand the MDC list – replaces customvalue1-3 older fields - this field is %mdc





L




28p_messagelog system
The marker labels INVOKE, ENTRY, EXIT – and later will also include DEBUG, AUDIT, METRICS, ERROR when we go to 1 log file - this field is %marker



L





RootExceptionlog system
%rootException - Dublin spec only



L




29p_markerlog system
The marker labels INVOKE, ENTRY, EXIT – and later will also include DEBUG, AUDIT, METRICS, ERROR when we go to 1 log file - this field is %marker



L




...

Used as valuable URI - to annnote invoke marker

Review in terms of 48534506 16339021 - possiblly add INVOKE-return - to filter reporting

...

Note there are 3 tabs (see p_mak in logback.xml) delimiting the MARKERS (ENTRY and EXIT) at the end of each line
<property name="p_mak" value="%replace(%replace(%marker){'\t', '\\\\t'}){'\n','\\\\n'}"/>


Code Block
languagejava
themeMidnight
2018-07-05T20:21:34.794Z	http-nio-8080-exec-2	INFO	org.onap.demo.logging.ApplicationService	InstanceID=ede7dd52-91e8-45ce-9406-fbafd17a7d4c, RequestID=f9d8bb0f-4b4b-4700-9853-d3b79d861c5b, ServiceName=/logging-demo/rest/health/health, InvocationID=8f4c1f1d-5b32-4981-b658-e5992f28e6c8, InvokeTimestamp=2018-07-05T20:21:26.617Z, PartnerName=, ClientIPAddress=0:0:0:0:0:0:0:1, ServerFQDN=localhost			ENTRY	
2018-07-05T20:22:09.268Z	http-nio-8080-exec-2	INFO	org.onap.demo.logging.ApplicationService	ResponseCode=, InstanceID=ede7dd52-91e8-45ce-9406-fbafd17a7d4c, RequestID=f9d8bb0f-4b4b-4700-9853-d3b79d861c5b, ServiceName=/logging-demo/rest/health/health, ResponseDescription=, InvocationID=8f4c1f1d-5b32-4981-b658-e5992f28e6c8, Severity=, InvokeTimestamp=2018-07-05T20:21:26.617Z, PartnerName=, ClientIPAddress=0:0:0:0:0:0:0:1, ServerFQDN=localhost, StatusCode=			EXIT

...

Excerpt

New ONAP Component Checklist

Add this procedure to the Project Proposal Template

By following a few simple rules:

  • Your component's output will be indexed automatically. 
  • Analytics will be able to trace invocation through your component.

Obligations fall into two categories:

  1. Conventions regarding configuration, line format and output. 
  2. Ensuring the propagation of contextual information. 

You must:

  1. Choose a Logging provider and/or EELF. Decisions, decisions.
  2. Create a configuration file based on an existing archetype. See 4853450616339021.
  3. Read your configuration file when your components initialize logging.
  4. Write logs to a standard location so that they can be shipped by Filebeat for indexing. See 4853450616339021.
  5. Report transaction state:
    1. Retrieve, default and propagate RequestID. See 48534506 16339021.
    2. At each invocation of one ONAP component by another:
      1. Initialize and propagate InvocationID. See 48534506 16339021.
      2. Report INVOKE and SYNCHRONOUS markers in caller. 
      3. Report ENTRY and EXIT markers in recipient. 
  6. Write useful logs!

 They are unordered. 

What's New

(Including what WILL be new in v1.2  / R2). 

  1. Field separator reverted to pipe. 
  2. Dual appenders in Logback and Log4j reference configurations:
    1. Indexable, for shipping and indexing. 
    2. EELF, for backward compatibility. 
    3. Minor changes to path conventions.
  3. XML output deprecated (required only for Log4j1.2, which is also expected to go).
  4. Improved documentation of semantics and usage (including initialization and propagation via ThreadLocaland HTTP headers) for existing MDCs and attributes. 
  5. Add MDCs/Markers + usage for invocation IDs, allowing call graphs to be built without reliance on heuristics.
  6. Revisiting persistence (a clear requirement) and rollover settings, based on feedback from operations. 
  7. More discussion of How to Log. (Where previously guidelines were largely concerned with architecture and mechanics).
  8. Locking in other changes proposed in R1, including MDC serialization, escaping, etc. These can be treated as accepted. (Note that they only affect indexable output).

In addition, we expect to provide (as a Beijing deliverable) a minimal, synthetic component as an example of best-practices, and this will provide all code examples for this guide. (Does that mean the example will log via EELF, or will we end up with two variants?)

Pending Specification Work

iddateitemdetailsstatus

20180614MDC ClientIPAddressAsk question of OPS to remove this field - 20180419todo

20180614MDC ResponseCode / ResponseDescriptionexpand/find note 1*todo


Developer Guide

see separate page (cross releases) in Logging Developer Guide