Table of Contents

...

Questions on onap-discuss from the teams
https://lists.onap.org/pipermail/onap-discuss/2018-January/007415.html

Latest Filebeat/ELK implementation by Clamp for reference - https://gerrit.onap.org/r/#/c/48961/ on

Jira Legacy

server	System Jira
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	CLAMP-170

Jira Legacy

server	System Jira
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	LOG-118

Purpose

The goals of the logging project are the following

...

Gliffy

name	Logging-RI-Deployment
pagePin	3

Quickstart

WIP (as of 20180121) - will notify when complete

...

app

project

repo

version

aai

aai-common

1.0.0

aai

aai-core

1.0.0

aai

aai-service (deprecated)

0.0.1 (not in use)

aai

aai-traversal

1.0.0

aai

ajsc-aai

1.0.0

aai

eelf-logging

logging-service

0.0.1 (not in use)

appc

(pax.logging = wrapper on log4j, slf4j, commons-logging 1.2 and logback 1.1.1)

needs this RI - provide a demo to the appc team

0.0.1 (upgrade?)

ccsdk

nbapi

ccsdk/platform/nbapi

1.0.0

dmaap

buscontroller

dmaap/buscontroller

0.0.1

ui

dcae_dmaap_webapp

dmaapbc

1.0.0

ui

dmaap-bc-app-common

1.0.0

ui

dmaap-bc-app-parent

1.0.0

policy

common-modules/

common

1.0.0

policy

drools-pdp

1.0.0

portal

ecompportal-be-common

portal

<eelf.version>1.0.0
</eelf.version>

policy

feature-eelf

drools-pdp

1.0.0

policy

PolicyEngineSuite/ONAP-logging

common

0.0.1 (not in use)

policy

ONAP-SDK-APP

engine

0.0.1 (not in use)

sdc

?

so

common

so

0.0.1 (upgrade ?)

vid

epsdk-app-onap

vid

1.0.0

vid

vid-app-common

vid

1.0.0

open-o

multicloud

python based

Jira Legacy

server	System Jira
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	MULTICLOUD-151

...

Code Block

obrienbiometrics:docker michaelobrien$ vi DockerFile
obrienbiometrics:docker michaelobrien$ docker build -t mock-logging-tomcat -f DockerFile .
Sending build context to Docker daemon  18.04MB
Step 1/2 : FROM tomcat:8.0.48-jre8
---> e072422ca96f
Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war
---> f0d234622af6
Successfully built f0d234622af6
Successfully tagged mock-logging-tomcat:latest
obrienbiometrics:docker michaelobrien$ docker images | grep mock-logging
mock-logging-tomcat                                           latest                            f0d234622af6        9 seconds ago       576MB
obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 mock-logging-tomcat:latest
ef5bdffb0e0aef9f185a51be2a915eae3d72fdb9a2125e1bb373516137c205b4
obrienbiometrics:docker michaelobrien$ docker ps
CONTAINER ID        IMAGE                        COMMAND             CREATED             STATUS              PORTS                    NAMES
ef5bdffb0e0a        mock-logging-tomcat:latest   "catalina.sh run"   5 seconds ago       Up 4 seconds        0.0.0.0:8888->8080/tcp   adoring_hawking
obrienbiometrics:docker michaelobrien$ docker logs -f adoring_hawking

20-Jan-2018 04:30:26.092 INFO [localhost-startStop-1] org.springframework.web.context.ContextLoader.initWebApplicationContext Root WebApplicationContext: initialization completed in 1352 ms
20-Jan-2018 04:30:26.592 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive /usr/local/tomcat/webapps/logging-demo.war has finished in 3,652 ms
20-Jan-2018 04:30:26.841 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3943 ms
^C
obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@433552a
obrienbiometrics:docker michaelobrien$ docker build -t obrienlabs/mock-logging-tomcat -f DockerFile .
Sending build context to Docker daemon  18.04MB
Step 1/2 : FROM tomcat:8.0.48-jre8
---> e072422ca96f
Step 2/2 : COPY logging-demo-1.2.0-SNAPSHOT.war /usr/local/tomcat/webapps/logging-demo.war
---> Using cache
---> f0d234622af6
Successfully built f0d234622af6
Successfully tagged obrienlabs/mock-logging-tomcat:latest
obrienbiometrics:docker michaelobrien$ docker tag obrienlabs/mock-logging-tomcat obrienlabs/mock-logging-tomcat:1
obrienbiometrics:docker michaelobrien$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username (michaelobrien): obrienlabs
Password:
Login Succeeded
obrienbiometrics:docker michaelobrien$ docker push obrienlabs/mock-logging-tomcat:1
The push refers to a repository [docker.io/obrienlabs/mock-logging-tomcat]
8bb560dc792d: Pushed
7a5faefa0b46: Pushed
d1a0f91a1ad8: Pushed
c4d99fd86e94: Pushed
04178c827c65: Pushed
c571db474d75: Pushed
93742a1ae069: Pushed
7ffe9646653b: Pushed
6ef532e39c1e: Pushed
a4d7b0ac0438: Pushed
06f4de5fefea: Pushed
851f3e348c69: Pushed
e27a10675c56: Pushed
1: digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b size: 3048

run

obrienbiometrics:docker michaelobrien$ docker run -d -it --rm -p 8888:8080 obrienlabs/mock-logging-tomcat:1
316023a8abed5911000a1c366b7c897a33abcd7e9e70f48a4e2c2611f4c98d93
obrienbiometrics:docker michaelobrien$ curl http://localhost:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@75f0964b
obrienbiometrics:docker michaelobrien$


test on separate machine

ubuntu@ip-172-31-52-161:~$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ip-172-31-52-161:~$ docker run -d -it  -p 8888:8080 obrienlabs/mock-logging-tomcat:1
Unable to find image 'obrienlabs/mock-logging-tomcat:1' locally
1: Pulling from obrienlabs/mock-logging-tomcat
723254a2c089: Pull complete
abe15a44e12f: Pull complete
409a28e3cc3d: Pull complete
a9511c68044a: Pull complete
9d1b16e30bc8: Pull complete
0fc5a09c9242: Pull complete
d34976006493: Pull complete
3b70003f0c10: Pull complete
bc7887582e2e: Pull complete
d2ab4f165865: Pull complete
7a0257d5eec9: Pull complete
a23d5c1d9825: Pull complete
42f73552ced4: Pull complete
Digest: sha256:a583f08001b21635e74e205f130ffbd23d933032686345be50f39a46b537e11b
Status: Downloaded newer image for obrienlabs/mock-logging-tomcat:1
4262ca54a570a177185b646ec92b440f37250f7ee2922e018aec5d226c5b4f2a
ubuntu@ip-172-31-52-161:~$ docker ps
CONTAINER ID        IMAGE                              COMMAND             CREATED             STATUS              PORTS                    NAMES
4262ca54a570        obrienlabs/mock-logging-tomcat:1   "catalina.sh run"   9 seconds ago       Up 8 seconds        0.0.0.0:8888->8080/tcp   kickass_hoover
ubuntu@ip-172-31-52-161:~$ curl http://dev.onap.info:8888/logging-demo/rest/read/test
testing: org.onap.logging.ApplicationService@86a2339

DI 7: 20180126: Config for Logging Demo Containers

...

Implementation

Deploying the RI

...

Jira Legacy
server System Jira
serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176
key
LOG
OOM-
137
Deploy a Kubernetes cluster as normal via (No ONAP yet, just the K8S cluster) - ONAP on Kubernetes#InstallDocker/Rancher/Helm/Kubectl
Run the config and log pods
...
628

DI 8: 20180204: NodePort port reservation framework required

Jira Legacy

server	System Jira
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	OOM-656

DI 9: 20180417: Restart a failed ELK stack container

Periodically one of the higher containers in a dependency tree will not get restarted in time to pick up running child containers - usually this is the kibana container

Fix this or "any" container by deleting the container in question and kubernetes will bring another one up.

Code Block

theme	Midnight

root@a-onap-auto-20180412-ref:~# kubectl get services --all-namespaces | grep log
onap          dev-vfc-catalog               ClusterIP   10.43.210.8     <none>        8806/TCP                                                                     5d
onap          log-es                        NodePort    10.43.77.87     <none>        9200:30254/TCP                                                               5d
onap          log-es-tcp                    ClusterIP   10.43.159.93    <none>        9300/TCP                                                                     5d
onap          log-kibana                    NodePort    10.43.41.102    <none>        5601:30253/TCP                                                               5d
onap          log-ls                        NodePort    10.43.180.165   <none>        5044:30255/TCP                                                               5d
onap          log-ls-http                   ClusterIP   10.43.13.180    <none>        9600/TCP                                                                     5d
root@a-onap-auto-20180412-ref:~# kubectl get pods --all-namespaces | grep log
onap          dev-log-elasticsearch-66cdc4f855-wmpkz         1/1       Running                 0          5d
onap          dev-log-kibana-5b6f86bcb4-drpzq                0/1       Running                 1076       5d
onap          dev-log-logstash-6d9fdccdb6-ngq2f              1/1       Running                 0          5d
onap          dev-vfc-catalog-7d89bc8b9d-vxk74               2/2       Running                 0          5d
root@a-onap-auto-20180412-ref:~# kubectl delete pod dev-log-kibana-5b6f86bcb4-drpzq -n onap
pod "dev-log-kibana-5b6f86bcb4-drpzq" deleted
root@a-onap-auto-20180412-ref:~# kubectl get pods --all-namespaces | grep log
onap          dev-log-elasticsearch-66cdc4f855-wmpkz         1/1       Running                 0          5d
onap          dev-log-kibana-5b6f86bcb4-drpzq                0/1       Terminating             1076       5d
onap          dev-log-kibana-5b6f86bcb4-gpn2m                0/1       Pending                 0          12s
onap          dev-log-logstash-6d9fdccdb6-ngq2f              1/1       Running                 0          5d
onap          dev-vfc-catalog-7d89bc8b9d-vxk74               2/2       Running                 0          5d

Implementation

Deploying the RI

WIP

Jira Legacy

server	System Jira
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	LOG-137

Deploy a Kubernetes cluster as normal via (No ONAP yet, just the K8S cluster) - ONAP on Kubernetes#InstallDocker/Rancher/Helm/Kubectl

Run the config and log pods

Code Block

# clone oom
git clone ssh://michaelobrien@gerrit.onap.org:29418/oom
# clone logging-analytics
git clone ssh://michaelobrien@gerrit.onap.org:29418/logging-analytics

# 
./createConfig.sh -n onap
./createAll.bash -n onap -a log
# temp create config dirs
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ sudo mkdir /dockerdata-nfs/onap/logdemo
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ sudo mkdir /dockerdata-nfs/onap/logdemo/eteshare
# copy the logdemo folder into the oom folder
root@obriensystemsu0:~/logging-analytics/reference/logging-kubernetes# cp -R logdemo ~/oom/kubernetes/
# make sure the 3 ELK log containers are up
root@obriensystemsu0:~/oom/kubernetes/oneclick# ./createAll.bash -n onap -a log
root@obriensystemsu0:~/oom/kubernetes/oneclick# kubectl get pods --all-namespaces -a
onap-log      elasticsearch-3138115406-mlwjf         1/1       Running     0          12m
onap-log      kibana-3372627750-260f2                1/1       Running     0          12m
onap-log      logstash-1708188010-bhlqf              1/1       Running     0          12m

# create the log demo
ubuntu@ip-172-31-52-161root@obriensystemsu0:~/oom/kubernetes/oneclick$oneclick# ./createAll.bash -n onap -a logdemo

********** Creating instance 1 of ONAP with port range 30200 and 30399

********** Creating ONAP: 


********** Creating deployments for logdemo ********** 

Creating namespace **********
namespace "onap-logdemo" created

Creating service account **********
clusterrolebinding "onap-logdemo-admin-binding" created

Creating registry secret **********
secret "onap-docker-registry-key" created

Creating deployments and services **********
NAME:   onap-logdemo
LAST DEPLOYED: Mon Jan 29 0111:0725:0144 2018
NAMESPACE: onap
STATUS: DEPLOYED

RESOURCES:
==> v1/Service
NAME     TYPE      CLUSTER-IP     EXTERNAL-IP  PORT(S)         AGE
logdemo  NodePort  10.43.19526.13445  <none>       8080:30299/TCP  0s

==> v1beta1/Deployment
NAME     DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
logdemo  1        1        1           0          0s

==> v1/Pod(related)
NAME                      READY  STATUS             RESTARTS  AGE
logdemo-18732489063133092390-7jgm9g4qx7  0/1    ContainerCreating  0         0s





#check
ubuntu@ip-172-31-52-161root@obriensystemsu0:~/oom/kubernetes/oneclick$oneclick#  kubectl get pods --all-namespaces | grep logdemo
onap-logdemo   logdemo-40919519303133092390-p0zclg4qx7               1/1       Running   0          43s
ubuntu@ip-172-31-52-161 5m
root@obriensystemsu0:~/oom/kubernetes/oneclick$oneclick# kubectl get services  --all-namespaces -o wide | grep logdemo
onap-logdemo   logdemo                NodePort       10.43.9826.10445     <none>           8080:30299/TCP   6m 57s       app=logdemo


root@obriensystemsu0:~/logging-analytics/reference# kubectl -n onap-logdemo logs -f logdemo-3133092390-g4qx7
29-Jan-2018 17:28:03.456 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 11584 ms

# test in-container and out-container
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ kubectl -n onap-logdemo exec -it logdemo-4091951930-zwd53 bash
root@logdemo-4091951930-zwd53:/usr/local/tomcat# ls webapps/
ROOT  docs  examples  host-manager  logging-demo  logging-demo.war  manager

root@logdemo-4091951930-zwd53:/usr/local/tomcat# curl http://127.0.0.1:8080/logging-demo/rest/read/test
testing: org.onap.demo.logging.ApplicationService@7a4fa9b

outside
ubuntu@ip-172-31-52-161:~/oom/kubernetes/oneclick$ curl http://dev.onap.info:30299/logging-demo/rest/read/test
testing: org.onap.demo.logging.ApplicationService@52772382

...

Versions Compared

Old Version 35

New Version Current

Key

Purpose

Quickstart

DI 7: 20180126: Config for Logging Demo Containers

Implementation

Deploying the RI

Jira Legacy
server System Jira
serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176
key
LOG
OOM-
137
Deploy a Kubernetes cluster as normal via (No ONAP yet, just the K8S cluster) - ONAP on Kubernetes#InstallDocker/Rancher/Helm/Kubectl
Run the config and log pods
...
628

DI 8: 20180204: NodePort port reservation framework required

DI 9: 20180417: Restart a failed ELK stack container

Implementation

Deploying the RI

Page Comparison

Versions Compared

Old Version 35

New Version Current

Key

Purpose

Quickstart

DI 7: 20180126: Config for Logging Demo Containers

Implementation

Deploying the RI

Jira LegacyserverSystem JiraserverId4733707d-2057-3a0f-ae5e-4fd8aff50176keyLOGOOM-137Deploy a Kubernetes cluster as normal via (No ONAP yet, just the K8S cluster) - ONAP on Kubernetes#InstallDocker/Rancher/Helm/KubectlRun the config and log pods...628

DI 8: 20180204: NodePort port reservation framework required

DI 9: 20180417: Restart a failed ELK stack container

Implementation

Deploying the RI

Jira Legacy
server System Jira
serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176
key
LOG
OOM-
137
Deploy a Kubernetes cluster as normal via (No ONAP yet, just the K8S cluster) - ONAP on Kubernetes#InstallDocker/Rancher/Helm/Kubectl
Run the config and log pods
...
628