{"type":"doc","content":[{"type":"paragraph","content":[{"text":"(Ref: ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://lf-onap.atlassian.net/browse/CCSDK-4102"}},{"text":")","type":"text"}]},{"type":"paragraph","content":[{"text":"It is possible to leverage different kinds of format and functionality using logback configuration in the A1 Policy Management Service. This is useful for readability for the user but also for sharing logs with other services e.g. log aggregation services. This document will outline one way to use a ‘logback xml’ configuration together with the ELK stack (Elasticsearch, Logstash and Kibana) to format, display, collect, parse and visualise logs coming from A1PMS.","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Prerequisites","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Docker compose","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"maven","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"A1PMS code-base checked out from git.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Java 17","type":"text"}]}]}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"ELK Stack","type":"text"}]},{"type":"paragraph","content":[{"text":"Firstly, we will outline how we can use docker-compose to deploy the ELK stack in our local machines. The relevant docker-compose file is below","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"version: '3.8'\n\nservices:\n elasticsearch:\n image: docker.elastic.co/elasticsearch/elasticsearch:8.5.3\n container_name: elasticsearch\n environment:\n - discovery.type=single-node\n - \"ES_JAVA_OPTS=-Xms512m -Xmx512m\"\n - xpack.security.enabled=false\n ports:\n - \"9200:9200\"\n networks:\n - elk\n\n logstash:\n image: docker.elastic.co/logstash/logstash:8.5.3\n container_name: logstash\n volumes:\n - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf\n ports:\n - \"5044:5044\" # TCP input for Logstash\n - \"9600:9600\" # Logstash monitoring API\n depends_on:\n - elasticsearch\n networks:\n - elk\n\n kibana:\n image: docker.elastic.co/kibana/kibana:8.5.3\n container_name: kibana\n environment:\n - ELASTICSEARCH_HOSTS=http://elasticsearch:9200\n ports:\n - \"5601:5601\"\n depends_on:\n - elasticsearch\n networks:\n - elk\n\nnetworks:\n elk:\n driver: bridge","type":"text"}]},{"type":"paragraph","content":[{"text":"There are three services deployed in this docker compose","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Logstash – Log Collector & Processor","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Ingests logs from various sources, processes them (parsing, filtering, transforming), and forwards them to a destination like Elasticsearch. Supports multiple input sources (files, TCP, Kafka, Syslog, Beats, etc.).","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In this case we will use TCP","type":"text"}]}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Elasticsearch - Elasticsearch – Log Storage & Search Engine","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Stores logs in a ","type":"text"},{"text":"searchable","type":"text","marks":[{"type":"strong"}]},{"text":" and ","type":"text"},{"text":"structured","type":"text","marks":[{"type":"strong"}]},{"text":" format, allowing fast queries and analytics.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Stores log data as JSON documents.","type":"text"}]}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Kibana – Log Visualization & Dashboarding","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Provides a ","type":"text"},{"text":"UI","type":"text","marks":[{"type":"strong"}]},{"text":" to explore, analyze, and visualize log data stored in Elasticsearch.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Supports searching and filtering logs.","type":"text"}]}]}]}]}]},{"type":"paragraph","content":[{"text":"These tools perform many other functions that are described here but the descriptions here are what we are interested in.","type":"text"}]},{"type":"paragraph","content":[{"text":"Deployment of these services can be done by","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Copying and pasting the above into a file (docker-compose.yaml).","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Copying and pasting the below into a file named logstash.conf that will be in the same directory as the docker-compose.yaml file. This file configures logstash to send the streaming logs to elasticsearch.","type":"text"}]}]}]},{"type":"codeBlock","attrs":{"language":"json"},"content":[{"text":"input {\n tcp {\n port => 5044\n codec => json_lines\n }\n}\n\nfilter {\n mutate {\n add_field => { \"service\" => \"a1pms-app\" }\n }\n}\n\noutput {\n elasticsearch {\n hosts => [\"http://elasticsearch:9200\"]\n index => \"a1-pms-spring-boot-logs-%{+YYYY.MM.dd}\"\n }\n stdout { codec => rubydebug }\n}\n","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Then, we can run the below command from the same directory as the docker-compose.yaml file and the logstash.conf file.","type":"text"}]}]}]},{"type":"codeBlock","content":[{"text":"docker compose up -d","type":"text"}]},{"type":"paragraph","content":[{"text":"That should bring up all the services.","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"A1PMS Logback Configuration","type":"text"}]},{"type":"paragraph","content":[{"text":"The next step is to change the logback configuration of A1PMS to “feed into” logstash. Using the below logback-json.xml file will do a few things","type":"text"}]},{"type":"orderedList","attrs":{"order":1},"content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Create one appender that formats the logs as JSON and adds some fields","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Send the logs to logstash.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Creates one appender that will output to console as plain log format with some additional fields.","type":"text"}]}]}]},{"type":"codeBlock","attrs":{"language":"xml"},"content":[{"text":"\n \n localhost:5044\n \n \n \n version\n 1.2.0\n \n \n timestamp\n yyyy-MM-dd'T'HH:mm:ss.SSSZ\n \n \n true\n \n {\n \"service_id\": \"${SERVICE_ID:-a1pms}\",\n \"message\": \"%msg\",\n \"facility\": \"%X{facility}\",\n \"subject\": \"%X{subject}\",\n \"extra_data\": {\n \"logger\": \"%logger\",\n \"thread_info\": {\n \"thread_name\": \"%thread\"\n },\n \"dst\": {\n \"trace_id\": \"%mdc{traceId}\"\n },\n \"exception\": {\n \"stack_trace\": \"%xEx\"\n }\n },\n \"metadata\": {\n \"application_id\": \"${APP_ID:-a1pms}\"\n }\n }\n \n \n \n\n \n \n\n \n \n \n %d{yyyy-MM-dd'T'HH:mm:ss.SSSZ} [%thread] %-5level %logger - %msg [facility=%X{facility}, subject=%X{subject}, traceId=%mdc{traceId}] %n%xEx\n \n \n \n\n \n \n \n \n\n \n","type":"text"}]},{"type":"paragraph","content":[{"text":"Copy and paste this into a logback-custom.xml file in the src/main/resources directory of the a1-policy-management-service directory.","type":"text"}]},{"type":"paragraph","content":[{"text":"We will then need to point to that custom file in our application.yaml file - located in a1-policy-management-service/config. The below should be added to that file under “logging”.","type":"text"}]},{"type":"codeBlock","attrs":{"language":"yaml"},"content":[{"text":"logging:\n config: ${LOGBACK_CONFIG_FILE:classpath:logback-custom.xml}","type":"text"}]},{"type":"paragraph","content":[{"text":"Once that is done, we are ready to run the build.","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Running the build","type":"text"}]},{"type":"paragraph","content":[{"text":"In this section, we will finally run the A1PMS build, which will generate logs and output them to logstash. They will then be visible in Kibana.","type":"text"}]},{"type":"codeBlock","content":[{"text":"mvn clean install","type":"text"}]},{"type":"paragraph","content":[{"text":"(Note: It is possible to change logging settings & application configuration without re-compiling or re-assembling the application - but for this case it is easier to understand and experiment by rebuilding)","type":"text","marks":[{"type":"em"}]}]},{"type":"paragraph","content":[{"text":"We execute the newly-built adapted application, then invoke some operations to induce logging messages. ","type":"text"},{"type":"hardBreak"},{"text":"We notice then that logs in the console are formatted like below - in a plain format","type":"text"}]},{"type":"codeBlock","content":[{"text":"2025-03-19T15:59:22.108+0000 [reactor-http-epoll-6] INFO org.onap.ccsdk.oran.a1policymanagementservice.util.v3.ReactiveEntryExitFilter - For the request ID: 3e937e51 the Status code of the response: 201 CREATED [facility=log audit, subject=n/av, traceId=491a39122e0356fa4e10928fa8758aa2]","type":"text"}]},{"type":"paragraph","content":[{"text":"Now we can look at Kibana. We would expect that the json formatted logs will have made their way there for visualisation.","type":"text"}]},{"type":"paragraph","content":[{"text":"We create a data view - like below","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center","width":1800,"widthType":"pixel"},"content":[{"type":"media","attrs":{"width":2554,"alt":"image-20250319-165002.png","id":"c4758a82-a718-454f-9c84-2172011bc4ae","collection":"contentId-140640493","type":"file","height":1264}}]},{"type":"paragraph","content":[{"text":"Then we can browse the logs in Kibana","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"center","width":1800,"widthType":"pixel"},"content":[{"type":"media","attrs":{"width":2554,"alt":"image-20250319-165050.png","id":"c6c2599b-fab5-43f8-98e6-fdbb8a1b6398","collection":"contentId-140640493","type":"file","height":1264}}]},{"type":"paragraph","content":[{"text":"This guide is not intended to be a full guide to how to use EKS but just to enable this functionality for A1PMS.","type":"text"}]}],"version":1}

Browser not supported