1 Purpose
2 Architecture
- 2.1 Microservices
  - 2.1.1 NBI - Northbound Interface (1/1)
    - 2.1.1.1 Logging Demo WAR
  - 2.1.2 PN - Processing Node (scaled N/N)
    - 2.1.2.1 Processing Node WAR
- 2.2 Project Structure
- 2.3 API
- 2.4 Deployment Diagram
3 Quickstart
4 Audits
5 Design Issues
6 Implementation
- 6.1 Deploying the RI
  - 6.1.1 Kubernetes deployment of the RI
- 6.2 Local Debugging via Eclipse
7 Testing
8 Links
- 8.1 EELF

WIP

This page is the design and reference site for the Logging RI. The following topics are being provided to the rest of the ONAP community.

How to use the EELF Logging Library
How to adhere to the logging guidelines
How to implement the filebeat sidecar container to push logs to the ELK stack pod
How to use the EELF AOP wrapping library
How to ensure requestID tracking during distributed transactions

Questions on onap-discuss from the teams
https://lists.onap.org/pipermail/onap-discuss/2018-January/007415.html

Latest Filebeat/ELK implementation by Clamp for reference - https://gerrit.onap.org/r/#/c/48961/ on CLAMP-170: Implement logging filebeat sidecar in CLAMPClosed

LOG-118: Add Logging Reference Impl example and AOP wrapper on Logging library Closed

Purpose

The goals of the logging project are the following

provide for standardized logs across all ONAP components

provide ELK framework for log capture, indexing and presentation/search

provide framework for requestID tracking of distributed transactions (correlation ID's in p.162 of Building Microservices 2015 by Sam Newman) via HTTP ETags

Architecture

Microservices

NBI - Northbound Interface (1/1)

Logging Demo WAR

PN - Processing Node (scaled N/N)

Processing Node WAR

Project Structure

Project	repo	Dependencies	Artifact	Docker

Project	repo	Dependencies	Artifact
logging-reference	logging-reference		pom
logging-demo	logging-demo		WAR
logging-mock-service	logging-mock-service	logging-library	WAR
Logging Library	logging-library		JAR
Kubernetes demo spec	kubernetes/log-mock-demo		yaml, config
Kubernetes service spec	kubernetes/log-mock-service		yaml, config
	logging-docker-root		pom
	logging-docker-demo	logging-demo	DockerFile
	logging-docker-service	logging-mock-service	DockerFile

API

Deployment Diagram

Quickstart

WIP (as of 20180121) - will notify when complete

Goto your IDE workspace

Clone the logging repo

https://gerrit.onap.org/r/#/admin/projects/logging-analytics

https://git.onap.org/logging-analytics/tree/

goto the reference folder and load the pom.xml project

obrienbiometrics:wse_sts392b michaelobrien$ git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics
Cloning into 'logging-analytics'...
Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts.
remote: Total 374 (delta 0), reused 374 (delta 0)
Receiving objects: 100% (374/374), 1001.91 KiB | 1.22 MiB/s, done.
Resolving deltas: 100% (120/120), done.
obrienbiometrics:wse_sts392b michaelobrien$ cd logging-analytics/reference/
obrienbiometrics:reference michaelobrien$ ls\
logging-demo		logging-library		logging-mock-service	pom.xml


# build the project to verify
obrienbiometrics:reference michaelobrien$ mvn clean install -U -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none
[INFO] logging-reference .................................. SUCCESS [  3.069 s]
[INFO] logging-library .................................... SUCCESS [  1.313 s]
[INFO] logging-mock-service ............................... SUCCESS [  0.395 s]
[INFO] logging-demo ....................................... SUCCESS [  2.329 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.232 s
[INFO] Finished at: 2018-01-21T09:29:15-05:00

Audits

Existing EELF implementations

Please use the latest 1.0.0 from https://github.com/att/EELF - nexus artifacts pending

app	project	repo	version

app	project	repo	version
aai	aai-common		1.0.0
aai	aai-core		1.0.0
aai	aai-service (deprecated)		0.0.1 (not in use)
aai	aai-traversal		1.0.0
aai	ajsc-aai		1.0.0
aai	eelf-logging	logging-service	0.0.1 (not in use)
appc	(pax.logging = wrapper on log4j, slf4j, commons-logging 1.2 and logback 1.1.1) needs this RI - provide a demo to the appc team		0.0.1 (upgrade?)
ccsdk	nbapi	ccsdk/platform/nbapi	1.0.0
dmaap	buscontroller	dmaap/buscontroller	0.0.1
ui	dcae_dmaap_webapp	dmaapbc	1.0.0
ui	dmaap-bc-app-common		1.0.0
ui	dmaap-bc-app-parent		1.0.0
policy	common-modules/	common	1.0.0
policy	drools-pdp		1.0.0
portal	ecompportal-be-common	portal	^{<eelf.version>1.0.0} ^{</eelf.version>}
policy	feature-eelf	drools-pdp	1.0.0
policy	PolicyEngineSuite/ONAP-logging	common	0.0.1 (not in use)
policy	ONAP-SDK-APP	engine	0.0.1 (not in use)
sdc			?
so	common	so	0.0.1 (upgrade ?)
vid	epsdk-app-onap	vid	1.0.0
vid	vid-app-common	vid	1.0.0
open-o
multicloud	python based		MULTICLOUD-151: Logging EnablementClosed

ONAP components with a Filebeat container

Projects with a filebeat container as of 20170114

ubuntu@ip-172-31-49-200:~$ kubectl get pods --all-namespaces | grep 2/2
onap-aai              aai-resources-1039856271-3ph1d                2/2       Running        0          37m
onap-aai              aai-traversal-143081090-f4cls                 2/2       Running        0          37m
onap-aai              model-loader-service-4144225433-0jr21         2/2       Running        0          37m
onap-aai              search-data-service-3842430948-53j1w          2/2       Running        0          37m
onap-aai              sparky-be-4222608366-966p4                    2/2       Running        0          37m
onap-appc             appc-1828810488-mhsf7                         2/2       Running        0          37m
onap-mso              mso-736294529-c2p8v                           2/2       Running        0          37m
onap-policy           drools-2600956298-dp7bq                       2/2       Running        0          37m
onap-policy           pap-466625067-n23bp                           2/2       Running        0          37m
onap-policy           pdp-2354817903-kmxv5                          2/2       Running        0          37m
onap-portal           portalapps-1783099045-79lcq                   2/2       Running        0          37m
onap-sdc              sdc-be-2336519847-p6p5l                       2/2       Running        0          37m
onap-sdc              sdc-fe-2862673798-xcpnk                       2/2       Running        0          37m
onap-sdnc             sdnc-0                                        2/2       Running        0          37m
onap-sdnc             sdnc-dbhost-0                                 2/2       Running        0          37m
onap-vid              vid-server-3026751708-rpmq4                   2/2       Running        0          37m

Current Logging Libraries

Current Framework usage impacting Logging

Logging in VNFs

Design Issues

DI 1: 20171231: Initial Maven Project Structure

https://gerrit.onap.org/r/#/admin/projects/logging-analytics

git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics

see

obrienbiometrics:logging-analytics michaelobrien$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)
	new file:   reference/logging-demo/.gitignore
	new file:   reference/logging-demo/pom.xml
	new file:   reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java
	new file:   reference/logging-library/.gitignore
	new file:   reference/logging-library/pom.xml
	new file:   reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java
	new file:   reference/pom.xml
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	reference/.project
	reference/.settings/
	reference/logging-demo/.checkstyle
	reference/logging-demo/.classpath
	reference/logging-demo/.project
	reference/logging-demo/.settings/
	reference/logging-library/.checkstyle
	reference/logging-library/.classpath
	reference/logging-library/.project
	reference/logging-library/.settings/
	reference/target/
obrienbiometrics:logging-analytics michaelobrien$ git commit -m "Initial logging lib/demo proj struct"
[master 9e42247] Initial logging lib/demo proj struct
 7 files changed, 306 insertions(+)
 create mode 100644 reference/logging-demo/.gitignore
 create mode 100644 reference/logging-demo/pom.xml
 create mode 100644 reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java
 create mode 100644 reference/logging-library/.gitignore
 create mode 100644 reference/logging-library/pom.xml
 create mode 100644 reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java
 create mode 100644 reference/pom.xml
obrienbiometrics:logging-analytics michaelobrien$ git review -s
obrienbiometrics:logging-analytics michaelobrien$ git commit -s --amend
[master c11cd50] Initial logging lib/demo proj struct
 Date: Sun Dec 31 13:30:04 2017 -0500
 7 files changed, 306 insertions(+)
 create mode 100644 reference/logging-demo/.gitignore
 create mode 100644 reference/logging-demo/pom.xml
 create mode 100644 reference/logging-demo/src/main/java/org/onap/logging/LoggingReference.java
 create mode 100644 reference/logging-library/.gitignore
 create mode 100644 reference/logging-library/pom.xml
 create mode 100644 reference/logging-library/src/main/java/org/onap/logging/LoggingReference.java
 create mode 100644 reference/pom.xml
obrienbiometrics:logging-analytics michaelobrien$ git review
Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts.
remote: Processing changes: new: 1, refs: 1, done            
remote: 
remote: New Changes:        
remote:   https://gerrit.onap.org/r/27195 Initial logging lib/demo proj struct        
remote: 
To ssh://gerrit.onap.org:29418/logging-analytics
 * [new branch]      HEAD -> refs/publish/master

For eclipse you will need to make sure the spring nature in the .classpath is set correctly - in order for maven build to also work in eclipse for war projects - this will show up in the project build path

DI 2: 20171231: Audit: EELF implementors

DI 3: 20171231: Audit: Mixed version of EELF dependency

LOG-110: Logging API Audit - existing codeClosed

Figuring out the reason for the different dependencies

aai-service, ajsc-aai, common-modules/common-logging

</dependency>

aai-common, eelf-logging, ONAP-logging, ONAP-SDK-APP

</dependency>

DI 4: 20170104: ETag propagation

Passing UUID based requestID's via rest calls

https://en.wikipedia.org/wiki/HTTP_ETag

DI 5: 20170120: Docker image design

PV for logback.xml,

container for mock-log-nbi

container for mock-log-proc (the scalable TX substrate)

DockerFile

build.sh

Building, tagging and pushing the docker image so we can consume it in Kubernetes

DI 6: 20170120: Docker wrapper of Logging Library

LOG-136: Logging RI: Code/build/tag microservice docker imageClosed

DI 7: 20180126: Config for Logging Demo Containers

OOM-628: Segregation of configuration for logging-demo component - LOG-118Closed

DI 8: 20180204: NodePort port reservation framework required

OOM-656: Framework for reserving nodeports 302nn - need a policyClosed

DI 9: 20180417: Restart a failed ELK stack container

Periodically one of the higher containers in a dependency tree will not get restarted in time to pick up running child containers - usually this is the kibana container

Fix this or "any" container by deleting the container in question and kubernetes will bring another one up.