Jakarta Best Practice Proposal for Standardized Logging Fields - v2

Notation:

This has been approved as a best practice for Jakarta.

Meetings

  • 2/4/2022 - Recording not available

    • Attendees: Toine, Amy Z, Muddasar, Bob H, Byung

  • 2/11/2022

  • 2/18/2022

    • Meeting Canceled

  • 2/25/2022

  • 3/4/202

  • 4/8/2022

    • @Robert Heinemann  @Sourabh Sourabh @Toine Siebelink @Muddasar Ahmed Met to discuss progress on Security Logging Fields POC in CPS.  The meeting was not recorded.

    • Add ordering to fields.  Added Order Column to tables

  • 4/27/2022

    • @Robert Heinemann @Sourabh Sourabh @Toine Siebelink @Ajay Deep Singh Met and discussed log output type and integration with logging collection architecture.  The meeting was not recorded. Group agreed that a JSON format would be good to use.  This was easier to implement with the logging library being used in CPS.  The change was just adding an appender to logback,xml.  In addition, the fluentid software used in the logging collection architecture handles the JSON format.  Group decided that this was a good approach to continue with the POC.  @Ajay Deep Singh has committed code regarding the logging architecture POC.  A demo of the POC will be given in 1 - 2 weeks from today.



General feedback is that the first iteration of these fields were focused on transactional events.  However there are other logging situations that are not transactional such as debug / informational logging.  So, some of the tweaks below are to make the fields fit a more general case.



NOTE:  The fields below should be outputted in the order indicate if the output format is positional like CSV.

CPS Implementation: CPS-986: Security Logging FieldsDelivered

Order

Field Name

Property Name

Description

EELF Field

Log Spec Field

Reference

CPS Logging Field POC

Order

Field Name

Property Name

Description

EELF Field

Log Spec Field

Reference

CPS Logging Field POC

GREEN FIELDS: Best Practice for Jakarta: Existing Fields Recommended

1

Timestamp

logTimeStamp

The container and container application MUST log the field “date/time” in the security audit logs. 

The value should be represented in UTC and formatted per ISO 8601, such as “2015-06-03T13:21:58+00:00”. The time should be shown with the maximum resolution available to the logging component (e.g., milliseconds, microseconds) by including the appropriate number of decimal digits. For example, when millisecond precision is available, the date-time value would be presented as, as “2015-06-03T13:21:58.340+00:00”.

BeginTimestamp

Timestamp

LogTimeStamp

R-97445

v1.3 Spec

2

Log Type Name

logTypeName

The container and container application MAY log the field "Log type" in security audit logs.

This field will adhere to the following ENUM ::= "AUDIT" | "METRICS" | "ERROR" | "DEBUG" | ""

This is here for legacy purposes.  Older projects used to generate 4 separate log files.  However since projects now will output all logs to STDOUT and STDERR this field is here to give projects adhering to the old standard a way to specify those log file types.

NOTE: This field is optional but a placeholder is still required to be outputted.  That is why the "" is included in the ENUM.

N/A

p_marker

(4)

3

Log Level

logLevel

The container and container application MUST use an appropriately configured logging level that can be changed dynamically.

The intention of this field is to not cause performance degradation via excessive logging. 

This field will adhere to the following ENUM ::= "FATAL" | "ERROR" | "WARN" | "INFO" | "DEBUG" | "TRACE"

The verbosity of the logging increases from left to right.

Category Log Level

Level

R-28168

(4)

4

Trace ID

traceId

The container and container application MUST log Trace ID

A trace ID is a universally unique value that identifies a single transaction request or a series of related log events within the ONAP platform. Its value is conformant to RFC4122 UUID. This value is readily and easily obtained in most programming environments. 

Request ID

Transaction ID

v1.3 Spec

(4)

5

Status Code

statusCode

The container and container application MUST log a "status code" in the security audit logs. 

This field indicates the high level status for transactional, APIs calls, or sub operational events.  

This field will adhere to the following ENUM ::= "SUCCESS" | "INPROGRESS" |"FAIL_WARN" | "FAIL_ERROR" | "FAIL_FATAL"

  • SUCCESS when the operation is successful.  This represents a normal case.

  • INPROGRESS for states that are not COMPLETE or one of the FAIL_* enums.

  • The following ENUMs represents when an event is not successful or abnormal / failure cases.

    • FAIL_WARN: Indicates that something has not worked as it should.  Program operation may continue without issue but depends on the particular circumstances of the execution environment.

    • FAIL_ERROR: Indicates that something serious has gone wrong.  Program may be recoverable through error routines.

    • FAIL_FATAL: Also indicates that something serious has gone wrong but is not recoverable.

From an end user perspective these categories should not be considered strict due to the absence of contextual information of holistic operations. There may be some circumstances where FAIL_WARN may be more serious than FAIL_ERROR.  Regardless, from a developer view, FAIL_WARN, FAIL_ERROR, and FAIL_FATAL should be viewed as increasing importance and understand that the end user will need to provide additional context for their comprehension and execute and potential action from the particular failure.

Status Code

Status Code

R-15325

v1.3 Spec

(4)

6

Principal ID

principalId

The container and container application MUST log the Principal identity of a requestor in the security audit logs. 

This field should contain the identification of the entity (user agent, client id, user, user id, login ID, non-person entity (NPE), Token,  etc.)  that made the request of the service or API indicated in the Service/Program Name field. For a serving API that is authenticating the request, this should be the authenticated username or equivalent.

NOTE: The CPS project uses a framework that provides this field. 

N/A

User

R-89474

 R-89474

v1.3 Spec

7

Service / Program Name

serviceName

The container and container application MUST log the field “service or program used for access” in the security audit logs.

This intention is to capture the service name endpoint or an externally advertised API invoked, e.g., where are you connecting to. This is represented as a URI or URL. 



NOTE: The CPS project uses a framework that provides this field. 

ServiceName

ServiceName

R-06413

v1.3 Spec



(4)

8

Log Message

message

The free text payload of a log event. 

detailMessage

p_message





Order

Field Name

Description

EELF Field

Log Spec Field

Reference

Order

Field Name

Description

EELF Field

Log Spec Field

Reference

YELLOW FIELDS: This fields are not part of the best practice.  They are a work in progress and are intended to be added to generated logs via a logging architecture.



Container Image Name / Tag

The container and container application MUST log the Container Image Name/Tag.

The image name/tag is as returned by the docker images command.

NOTE:  Images are not required to have tags

N/A

N/A





Container Image Digest

The container and container application MUST log the container image digest.

The digest is a cryptographic digest as returned by the docker images --digests command.



N/A

N/A

T1036, T1525



Container ID

The container and container application MUST log the container ID.

The container ID is the same that is returned by the docker ps -q command.

NOTE: The container ID is unique for life time of the the container instance. Once the container is killed, this ID goes away.

N/A

N/A



THE BELOW FIELDS ARE OUT OF SCOPE FOR THE YELLOW FIELDS POC



Role / Attribute ID

The container and container application MUST log the Role or Attribute ID of the Principal identity of the entity accessing the requested service or API.

Note: The group ID is in reference to a Role or Attribute as part of a RBAC or ABAC scheme.

3/4/2022: Team decided this field is out of scope for yellow fields for now.  Potentially revisit for a later phase. 

  • This is a nice to have for forensics purposes.  Although this field could be looked up, in the course of a forensics investigation this information may no longer be available. 

  • Also, it is undecided that this field belongs in the yellow fields or green fields.

N/A

N/A





Protocol

The container and container application MUST log the field “protocol” in the security audit logs.

This refers to the communication mechanism for a request.  The value of this field should be representative of the OSI application layer  protocol. This is represented as a decimal formatted TCP/IP port number.

N/A

N/A

R-25547

Green Fields: Fields or data that is within the scope of a single ONAP sub-project

Yellow Fields: 

Fields or data that is broader in scope than a single project.  Maybe contain data that is only available at the container or orchestration level. Also may contain data that involves multiple ONAP sub-projects.

Results from comparison of existing logging