Logback Core Standardization for DCAE

Owned by Bert Sloan
Last updated: Mar 15, 2023
5 min read

Reference for the logging standardization effort: Jakarta Best Practice Proposal for Standardized Logging Fields - v2

This requirement outlines that we can use JSON, CSV, or any other delimited format as long as the field names are present.

JSON Approach

This has not been successfully tested on any other repo, but the CPS team implemented console JSON logging by adding the following to the XML files:

Add to the main logback xml file or break it up into pieces depending on how the xml files are structured on the project:

<configuration scan="true" scanPeriod="30 seconds" debug="false"> <include resource="org/springframework/boot/logging/logback/defaults.xml" /> <include resource="org/springframework/boot/logging/logback/console-appender.xml" /> <springProperty scope="context" name="springAppName" source="spring.application.name"/> <springProperty scope="context" name="username" source="security.auth.username"/> <springProperty scope="context" name="loggingFormat" source="logging.format"/> <property name="currentTimeStamp" value="%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX,UTC}"/> <appender name="jsonConsole" class="ch.qos.logback.core.ConsoleAppender"> <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder"> <providers> <pattern> <omitEmptyFields>true</omitEmptyFields> <pattern> { "logTimeStamp": "${currentTimeStamp:-}", "logTypeName": "", "logLevel": "%level", "traceId": "%X{traceId:-}", "statusCode": "", "principalId": "${username:-}", "serviceName": "${springAppName:-}", "message": "%message", "spanId": "%X{spanId:-}", "processId": "${PID:-}", "threadName": "%thread", "class": "%logger{40}", "exception": "%wEx" } </pattern> </pattern> </providers> </encoder> </appender> <appender name="asyncConsole" class="ch.qos.logback.classic.AsyncAppender"> <if condition='property("loggingFormat").contains("json")'> <then> <appender-ref ref="jsonConsole" /> </then> <else> <appender-ref ref="CONSOLE" /> </else> </if> </appender> <root level="INFO"> <appender-ref ref="asyncConsole" /> </root> </configuration>



Depending on how the logging files are setup, this could be added in the main XML or a separate file that has the property elements in it:

<property scope="context" name="springAppName" value="spring.application.name"/>
<property scope="context" name="username" value="security.auth.username"/>



The appender is the main piece here that will actually output the pattern into the console or a file:





String Output Approach

If the goal is to output strings to the console instead of JSON, these suggestions and resources might be helpful:

https://logback.qos.ch/manual/layouts.html

This documentation under pattern layouts shows all the fields that are available for outputting in a string format out of the box with logback core. We can also take advantage of the idea of adding <property> elements and referencing them in the log pattern as above.



General Notes

These notes are going to be here for extra help navigating the building and testing process to see if your logging changes are working:

  1. You may have to build the project from a subdirectory rather than the base project directory. This CI Management repo will help clarify where each of the projects are building from to better identify the subdirectory you should build from: https://github.com/onap/ci-management/tree/master/jjb/dcaegen2

  2. You will need the onap-settings.xml file to build each project. You can download it here: onap-settings.xml



Misc Resources:

Git Review: Setting Up Your Development Environment#git-review(optional)

ONAP Dev Env Overview: Development Procedures and Policies

Pushing to Gerrit: https://wiki.web.att.com/pages/viewpage.action?spaceKey=DPD&title=Pushing+Code+to+ONAP+Gerrit

Jira Ticket for DCAE logging Specs: DCAEGEN2-3257: Logging spec alignment for DCAE servicesIn Progress

CPS Logback POC: CPS-986 : Asynchronous Communication Logging

CPS Jira: CPS-986: Security Logging FieldsDelivered

CPS Gerrit: https://gerrit.onap.org/r/c/cps/+/128801/13/cps-service/src/main/resources/logback-spring.xml



List of Application Status with logback core:

Low level of effort*:

  • VES-mapper

  • HV-VES

Medium level of effort**:

  • DFC

  • DL-DES

  • DL-Feeder

  • PM-Mapper

  • PRH

  • SliceAnalysis MS

  • SON-Handler

  • TCAgen2

  • KPI-MS

*- can be completed with the string approach in a relatively short amount of time.

**- will take longer (and more troubleshooting) than basic changes to the XML files to complete the ask.