Logging Reference Implementation
- Michael O'Brien
- 1 Purpose
- 2 Architecture
- 2.1 Microservices
- 2.1.1 NBI - Northbound Interface (1/1)
- 2.1.1.1 Logging Demo WAR
- 2.1.2 PN - Processing Node (scaled N/N)
- 2.1.2.1 Processing Node WAR
- 2.1.1 NBI - Northbound Interface (1/1)
- 2.2 Project Structure
- 2.3 API
- 2.4 Deployment Diagram
- 2.1 Microservices
- 3 Quickstart
- 4 Audits
- 5 Design Issues
- 5.1 DI 1: 20171231: Initial Maven Project Structure
- 5.2 DI 2: 20171231: Audit: EELF implementors
- 5.3 DI 3: 20171231: Audit: Mixed version of EELF dependency
- 5.4 DI 4: 20170104: ETag propagation
- 5.5 DI 5: 20170120: Docker image design
- 5.5.1 DockerFile
- 5.6 DI 6: 20170120: Docker wrapper of Logging Library
- 5.7 DI 7: 20180126: Config for Logging Demo Containers
- 5.8 DI 8: 20180204: NodePort port reservation framework required
- 5.9 DI 9: 20180417: Restart a failed ELK stack container
- 6 Implementation
- 7 Testing
- 8 Links
- 8.1 EELF
WIP
This page is the design and reference site for the Logging RI. The following topics are being provided to the rest of the ONAP community.
How to use the EELF Logging Library
How to adhere to the logging guidelines
How to implement the filebeat sidecar container to push logs to the ELK stack pod
How to use the EELF AOP wrapping library
How to ensure requestID tracking during distributed transactions
Questions on onap-discuss from the teams
https://lists.onap.org/pipermail/onap-discuss/2018-January/007415.html
Latest Filebeat/ELK implementation by Clamp for reference - https://gerrit.onap.org/r/#/c/48961/ on https://lf-onap.atlassian.net/browse/CLAMP-170
https://lf-onap.atlassian.net/browse/LOG-118
Purpose
The goals of the logging project are the following
provide for standardized logs across all ONAP components
provide ELK framework for log capture, indexing and presentation/search
provide framework for requestID tracking of distributed transactions (correlation ID's in p.162 of Building Microservices 2015 by Sam Newman) via HTTP ETags
Architecture
Microservices
NBI - Northbound Interface (1/1)
Logging Demo WAR
PN - Processing Node (scaled N/N)
Processing Node WAR
Project Structure
Project | repo | Dependencies | Artifact | Docker |
|---|---|---|---|---|
logging-reference | logging-reference | pom | ||
logging-demo | logging-demo | WAR | ||
logging-mock-service | logging-mock-service | logging-library | WAR | |
Logging Library | logging-library | JAR | ||
Kubernetes demo spec | kubernetes/log-mock-demo | yaml, config | ||
Kubernetes service spec | kubernetes/log-mock-service | yaml, config | ||
logging-docker-root | pom | |||
logging-docker-demo | logging-demo | DockerFile | ||
logging-docker-service | logging-mock-service | DockerFile |
API
Deployment Diagram
Quickstart
WIP (as of 20180121) - will notify when complete
Goto your IDE workspace
Clone the logging repo
https://gerrit.onap.org/r/#/admin/projects/logging-analytics
https://git.onap.org/logging-analytics/tree/
goto the reference folder and load the pom.xml project
obrienbiometrics:wse_sts392b michaelobrien$ git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics
Cloning into 'logging-analytics'...
Warning: Permanently added '[gerrit.onap.org]:29418,[198.145.29.92]:29418' (RSA) to the list of known hosts.
remote: Total 374 (delta 0), reused 374 (delta 0)
Receiving objects: 100% (374/374), 1001.91 KiB | 1.22 MiB/s, done.
Resolving deltas: 100% (120/120), done.
obrienbiometrics:wse_sts392b michaelobrien$ cd logging-analytics/reference/
obrienbiometrics:reference michaelobrien$ ls\
logging-demo logging-library logging-mock-service pom.xml
# build the project to verify
obrienbiometrics:reference michaelobrien$ mvn clean install -U -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dadditionalparam=-Xdoclint:none
[INFO] logging-reference .................................. SUCCESS [ 3.069 s]
[INFO] logging-library .................................... SUCCESS [ 1.313 s]
[INFO] logging-mock-service ............................... SUCCESS [ 0.395 s]
[INFO] logging-demo ....................................... SUCCESS [ 2.329 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.232 s
[INFO] Finished at: 2018-01-21T09:29:15-05:00
Audits
Existing EELF implementations
Please use the latest 1.0.0 from https://github.com/att/EELF - nexus artifacts pending
app | project | repo | version |
|---|---|---|---|
aai | aai-common | 1.0.0 | |
aai | aai-core | 1.0.0 | |
aai | aai-service (deprecated) | 0.0.1 (not in use) | |
aai | aai-traversal | 1.0.0 | |
aai | ajsc-aai | 1.0.0 | |
aai | eelf-logging | logging-service | 0.0.1 (not in use) |
appc | (pax.logging = wrapper on log4j, slf4j, commons-logging 1.2 and logback 1.1.1) needs this RI - provide a demo to the appc team | 0.0.1 (upgrade?) | |
ccsdk | nbapi | ccsdk/platform/nbapi | 1.0.0 |
dmaap | buscontroller | dmaap/buscontroller | 0.0.1 |
ui | dcae_dmaap_webapp | dmaapbc | 1.0.0 |
ui | dmaap-bc-app-common | 1.0.0 | |
ui | dmaap-bc-app-parent | 1.0.0 | |
policy | common-modules/ | common | 1.0.0 |
policy | drools-pdp | 1.0.0 | |
portal | ecompportal-be-common | portal | <eelf.version>1.0.0 |
policy | feature-eelf | drools-pdp | 1.0.0 |
policy | PolicyEngineSuite/ONAP-logging | common | 0.0.1 (not in use) |
policy | ONAP-SDK-APP | engine | 0.0.1 (not in use) |
sdc | ? | ||
so | common | so | 0.0.1 (upgrade ?) |
vid | epsdk-app-onap | vid | 1.0.0 |
vid | vid-app-common | vid | 1.0.0 |
open-o | |||
multicloud | python based |
ONAP components with a Filebeat container
Projects with a filebeat container as of 20170114
ubuntu@ip-172-31-49-200:~$ kubectl get pods --all-namespaces | grep 2/2
onap-aai aai-resources-1039856271-3ph1d 2/2 Running 0 37m
onap-aai aai-traversal-143081090-f4cls 2/2 Running 0 37m
onap-aai model-loader-service-4144225433-0jr21 2/2 Running 0 37m
onap-aai search-data-service-3842430948-53j1w 2/2 Running 0 37m
onap-aai sparky-be-4222608366-966p4 2/2 Running 0 37m