...
Issue | Notes | Decision | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Agree on endurance performance KPI | The aim of the study is to early detection of bugs similar to
Thus, we should spot the specific function/s currently used in k6 to be improved. These function/s will be used in the Endurance test suit. Actions:
| CPS need to run all the test cases according to FS not according to what was tested in cps-2430 (Seeing they ran test that was outside of FS). Less focus on the KPI for this endurance run. Kolawole Adebisi-Adeolokun Halil Cakal | ||||||||
| 2 | Agree on the test environment | Blocked by
|
| ||||||||
| 3 | Visualization of memory trend | The most convenient way to represent the memory usage trend in a GUI is Grafana.
| Agreed to use Grafana majorly because it allows storage for longer periods sticking with our A/C Kolawole Adebisi-Adeolokun Halil Cakal The same decision was made by Daniel Hanrahan and Halil Cakal during the daily stand-up. Note: A PoC for the GNUPlot option has been made, please see the relevant Jira ticket details. Daniel Hanrahan Halil Cakal Toine Siebelink
| ||||||||
| 4 | Grafana access externally (If Grafana is selected at #3 then this issue should be discussed, otherwise ignore) |
Team Kraken support is needed either way. | Since Grafana is the preferred option, access issues shall be discussed with team Kraken Halil Cakal (consider Jenkins plug-in option as well)7
| ||||||||
| 5 | Permanent storage (DB) for Grafana (If Grafana is selected at #3 then this issue should be discussed, otherwise, ignore) | Investigate the solution strategy for the storage e.g. a permanent service 7/24 running … | Halil Cakal to investigate how to tune the volume |
...
Description | Jira | ||||||||
|---|---|---|---|---|---|---|---|---|---|
Add new test profile ‘Endurance’ |
| ||||||||
Add new Jenkins job to run endurance test |
| ||||||||
Add Grafana support to visualize memory usage pattern |
| ||||||||
Two docker-compose deployments simultaneously |
| ||||||||
Agree and Define new ‘Suite’ (js) |
|
Solution Proposal
Agree and Define new ‘Suite’ (js)
...
As mentioned in issues/decisions, there are two alternative ways of representing memory trends: Grafana and GNUPlotGnuPlot.
Grafana
...
Nordix has its own Prometheus and Grafana
...
(externally accessible, no need to install Globalprotect) and it can be configured to show cps-and-ncmp memory trends.
Link to Nordix Grafana: https://monitoring.nordix.org/login
Through prometheus.yml (of
...
Nordix), a new scrape_configs for the cps-and-ncmp microservice can be added
...
.
| Code Block | ||
|---|---|---|
| ||
scrape_configs:
- job_name: 'cps-and-ncmp'
metrics_path: '/actuator/prometheus'
scrape_interval: 5s
static_configs:
- targets:
- 'cps-and-ncmp:8080' // replace by <physical-server-ip:port> |
...
Also, the dashboard provider and dashboard config (jvm-micrometer-dashboard.json) can be added.
| Code Block | ||
|---|---|---|
| ||
providers:
- name: default
orgId: 1
type: file
options:
path: /var/lib/grafana/dashboards
foldersFromFilesStructure: true |
Then, the trend of G1 Old Gen space can be observed as seen below:
...
Permanent Storage Alternatives
Prometheus
Configuring the Prometheus with a persistent volume to retain data is possible.
This is an example service config for Prometheus:
| Code Block | ||
|---|---|---|
| ||
prometheus:
container_name: ${PROMETHEUS_CONTAINER_NAME:-prometheus}
image: prom/prometheus:latest
ports:
- ${PROMETHEUS_PORT:-9090}:9090
restart: always
volumes:
- ./config/prometheus.yml:/etc/prometheus/prometheus.yml
- prometheus_data:/prometheus
profiles:
- monitoring
volumes:
prometheus_data:
driver: local |
GNUPlot
GnuPlot can also draw a plot for only G1 Old Gen space
...
.