Page Comparison

The aggregated logging framework (Logging Architecture), uses Filebeat to send logs from each pod to the ELK stack where they are processed and stored. This requires each pod to have an additional container which will run Filebeat, and for the necessary log files to be accessible between containers.

...

https://gerrit.onap.org/r/gitweb?p=logging-analytics.git;a=tree;f=reference/logging-kubernetes/logdemonode;h=8f84219f008e648d5ade5137dd3f6591863b1af2;hb=d882270162d56c55da339af8fb9384e1bdc0160d

logging directory (verified) example here

https://git.onap.org/logging-analytics/tree/reference/logging-kubernetes/logdemonode/charts/logdemonode/templates/deployment.yaml#n77

For the setup, we will add two resource files:

...

1. Add filebeat.yml to resources (e.g. oom/kubernetes/<component>/resources/config/log/filebeat/filebeat.yml)

   A sample filebeat.yml can be found in the log demon node project:
   https://gerrit.onap.org/r/gitweb?p=logging-analytics.git;a=tree;f=reference/logging-kubernetes/logdemonode/charts/logdemonode/resources/config/log/filebeat;h=abf4f0a84ae7a15ed99fc4776727e4333948b583;hb=d882270162d56c55da339af8fb9384e1bdc0160d

   
   

2. Add Filebeat container to deployment.yaml:
   
   

   Code Block

   # Filebeat sidecar container - name: {{ include "common.name" . }}-filebeat-onap image: "{{ .Values.global.loggingRepository }}/{{ .Values.global.loggingImage }}" imagePullPolicy: {{ .Values.global.pullPolicy | default .Values.pullPolicy }} volumeMounts: - name: {{ include "common.fullname" . }}-filebeat-conf mountPath: /usr/share/filebeat/filebeat.yml subPath: filebeat.yml - name: {{ include "common.fullname" . }}-data-filebeat mountPath: /usr/share/filebeat/data - name: {{ include "common.fullname" . }}-logs mountPath: /var/log/onap

   Here the image and pull policy and based on global values. 
   The {{ include "common.fullname" . }}-filebeat-conf volume mount is used to share the filebeat.yml config resource. 

   The {{ include "common.fullname" . }}-logs volume mount is used to share the application logs for other containers in the pod

   The {{ include "common.fullname" . }}-data-filebeat volume mount is used to share filebeat logs

3. Add configmap to component to reference the filebeat.yml file that will be deployed to container (e.g. kubernetes/<component>/templates/configmap.yaml:

   Code Block
   apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name }}-<component>-filebeat-configmap namespace: {{ include "common.namespace" . }} data: {{ tpl (.Files.Glob "resources/config/log/filebeat/filebeat.yml").AsConfig . | indent 2 }}

   Replace <component> with the component name 

4. Add filebeat repo and image specification to values.yaml and config of service name and port:
   In file, oom/kubernetes/<component>/values.yaml, add 

   
   

   Code Block
   global: loggingRepository: docker.elastic.co loggingImage: beats/filebeat:5.5.0 # application configuration config: logstashServiceName: log-ls logstashPort: 5044

   
   

5. Add logback.xml file to microservice resources:
   kubernetes/<component>/charts/<microservice>/resources/config/logback.xml

   A sample logback can be found here:
   https://gerrit.onap.org/r/gitweb?p=logging-analytics.git;a=tree;f=reference/logging-kubernetes/logdemonode/charts/logdemonode/resources/config;hb=d882270162d56c55da339af8fb9384e1bdc0160d

   
   

6. Create configmap for microservice to share logback.xml with application container:

   Create/modify file, oom/kubernetes/<component>/charts/<microservice>/templates/configmap.yaml

   Code Block
   apiVersion: v1 kind: ConfigMap metadata: name: {{ include "common.fullname" . }}-log namespace: {{ include "common.namespace" . }} data: {{ tpl (.Files.Glob "resources/config/logback.xml").AsConfig . | indent 2 }}

   
   

7. Add volume mounts to microservice deployment.yaml to share logs and reference logback.xml:

   Modify container section of file, oom/kubernetes/<component>/charts/<microservice>/templates/deployment.yaml

   Code Block
   volumeMounts: - name: {{ include "common.fullname" . }}-logs mountPath: /var/log/onap - mountPath: /opt/ajsc/etc/config/logback.xml name: {{ include "common.fullname" . }}-log-conf subPath: logback.xml

   Note volumeMounts section may already exist. If so, just add new content to the existing section

   
   

8. Add volumes specification to microservice deployment.yaml to 1) share logback.xml via configmap, 2) share filebeat.yml via configmap, 3) share filebeat data via emptyDir 4) share application log data via emptyDir
   Add/modify volumes section of file, oom/kubernetes/<component>/charts/<microservice>/templates/deployment.yaml

   Code Block
   volumes: - name: {{ include "common.fullname" . }}-log-conf configMap: name: {{ include "common.fullname" . }}-log - name: {{ include "common.fullname" . }}-filebeat-conf configMap: name: {{ .Release.Name }}-<component>-filebeat-configmap - name: {{ include "common.fullname" . }}-data-filebeat emptyDir: {} - name: {{ include "common.fullname" . }}-logs emptyDir: {}

   Replace <component> with the component name
   If volumes section already exists, add content to existing section