...
...
...
...
...
...
...
...
...
...
...
...
Table of Contents
...
This detailed workload setup provides a comprehensive overview of the Kafka consumer environment on Linux, detailing how cloud events are handled, system resource configurations, and message publishing mechanisms.
Test Flow Description
According to requirements the test flow included following
...
steps:
- Use docker compose (C:\CPS\master\cps\docker-compose\docker-compose.yml) to make all the required containers running
docker-compose --profile dmi-stub --profile monitoring up -d - Stop "cps-and-ncmp" container.
- Use K6 script (k6-tests\once-off-test\kafka\produce-avc-event.js) to publish
...
- cm avc cloud events to topic "dmi-cm-events"
- Verify number of messages published to topic "dmi-cm-events" using kafka UI "http://localhost:8089/"
- Start "cps-and-ncmp" container again
- Verify number of messages consumed by topic "cm-events" using kafka UI "http://localhost:8089/"
- Login grafana http://localhost:3000/ to capture CPU, Memory and Threads.
Load:
The flow was repeated in a loop multiple times (defined with
circlesparameter);
...
Code Block language py title K6 Scenarios collapse true produce_cm_avc_event: { executor: 'shared-iterations', exec: 'produce_cm_avc_event', vus: 1000, // You can adjust VUs to meet performance requirements iterations: $TOTAL_MESSAGES, // Total messages to publish (for example: 100 K , 200 K) maxDuration: '15m', // Adjust depending on the expected completion time }
Test Environment
# | Environment/Workload | Description | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Tested on Linux | Laptop : Dell Inc. XPS 15 9530 | |||||||||
| 2 | Tested on Windows | Laptop : Lenovo ThinkPad | |||||||||
| 2 | Number of CPS Instance | 1 | |||||||||
| NCMP resource config | YAML Configuration: Defines deployment resources for the NCMP service: Replicas: 1 CPU Reservations: 2 CPUs Memory Reservations: 2 GB CPU Limits: 3 CPUs Memory Limits: 3 GB | ||||||||||
| Kafka Topic configuration | CM Notification Topic Configuration: Enabled: false Group ID: cm_events Topic Name: dmi-cm-events | ||||||||||
| Publishing topic name | dmi-cm-events | ||||||||||
| Forwarded topic name | cm-events | ||||||||||
| 4 | Total number of Cm Avc cloud events | 100,000 Kafka messages sent through the Kafka topic. | |||||||||
| 5 | Cloud event headers | The headers for each Kafka message contain the following fields: ce_type: "org.onap.cps.ncmp.events.avc1_0_0.AvcEvent" ce_source: "DMI" ce_destination: "dmi-cm-events" ce_specversion: "1.0" ce_time: ISO-formatted timestamp ce_id: A unique ID generated using crypto.randomUUID() ce_dataschema: "urn:cps.onap.cps.ncmp.events.avc1_0_0.AvcEvent:1.0.0" ce_correlationid: Correlation ID generated using crypto.randomUUID() | |||||||||
| 6 | Kafka payload |
| |||||||||
| 8 | Number of DMI Plugin stub | 1 DMI Plugin Stub is used for testing purposes. | |||||||||
| 9 | Commit ID | 81eb7dfc2f100a72692d2cbd7ce16540ee0a0fd4 | |||||||||
| 10 | Commit ID link | Commit Link | |||||||||
| 11 | K6 script (to publish cloud events) | The test script used to publish cloud events is located at: ..\cps\k6-tests\once-off-test\kafka\produce-avc-event.js |
...
Performance Comparison Between Linux and Windows Kafka Consumers:
Test Environment | Linux | Windows | Linux | Windows | ||||
Total number of kafka messages | 100,000 | 200,000 | ||||||
CPU (%) | 11344.98 2 | 26882.01 | Memory (MB) | 522 | 4971 | 78.3 | Network data sent (MB) | 5972.6 |
120 | Network data received Memory (MB) | 113 | 168244 | 195 | 212 | 222 | ||
Total number of threads | 360-361 | 358-362321 | 320 | 320 | 319 | |||
1st kaka message forwarded (HH:MM:SS) | 1316:4937:2511 | 14:56:3317:30:51 | 16.52.54 | 17.42.56 | ||||
Last kaka message forwarded (HH:MM:SS) | 1316:4937:2814 | 1417:56:3931:03 | 16.52.59 | 17.43.10 | ||||
Time taken to consume and forward kafka messages (Seconds) | 3 | 12 | 5 | 14 | ||||
Consumption rate (Messages / second) | 33,33316 | ,6678,333 | 40,000 | 14,286 | ||||
Kafka Consumer Performance on Linux:
...