Versions Compared

Old Version 48

changes.mady.by.user Adam Wudziński

Saved on

compared with

New Version Current

changes.mady.by.user Gliffy Diagram for Confluence

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Gliffy
macroId3a5ae2ae-4423-46fe-a3da-7f3f788166e4
namePerf test env
pageid6400403416374735

DMaaP Simulator


DMaap Simulator is a simple Spring Boot application, which exposes two endpoints. First endpoint (@PostMapping("/events/unauthenticated.SEC_FAULT_OUTPUT")) which receives fault events topic and second (@GetMapping("/summary")) which display events count and average processing time in milliseconds(depending on "startEpochMicrosec").

JMeter is generating the current epoch (current time) and updates this field:

Image Added


Architecture

Gliffy
macroId263b2a3c-f8f5-4fd1-bba1-766ddc81bf56
nameDMaap Simulator architecture
pagePin3


DMaap Simulator image:

View file
nameves-dmaa-simulator-image.tar
height150


DMaap Simulator supports VES collector in the Frankfurt release.

What is measured

JMeter test results & metrics

...

Image Removed
Test scenarioDescriptionJMeter test results & metricsVes metricsK8s metrics

1_test_scenario_6_steps.jmx

  • 6.6 RPS → 6RPS → 2.5 min5min
  • 11.6 RPS → 6RPS → 2.5 min 5min 
  • 23.3 RPS → 3RPS → 2.5 min5min
  •  33.3 RPS → 3RPS → 2.5 min5min50 RPS →
  • 50RPS → 2.5 min5min
  • 66.6 RPS → 6RPS → 2.5 min5min




2_test_scenario_1k_rps.jmx

  • 1000 RPS → 1 s1000RPS → 1s

2_test_scenario_2k_rps.jmx

  • 2000 RPS → 1 s2000RPS → 1s

2_test_scenario_3k_rps.jmx

  • 3000 RPS 3000RPS → 1 s 1s 

2_test_scenario_4k_rps.jmx
  • 4000 RPS 4000RPS → 1 s 1s 

2_test_scenario_5k_rps.jmx

  • 5000 RPS 5000RPS → 1 s 1s 

custom
  • 11.1 RPS → 1RPS → 15min

custom
  • 22.2 RPS → 15 min2RPS → 15min

custom
  • 33.3 RPS → 15 min 3RPS → 15min 

custom
  • 44.4 RPS → 15 min

Image Removed

Image Removed

custom 
  • 44.4 RPS → 15 min4RPS → 15min


Environment 2

  • CPU - 24 cores
  • CPU clock speed - 2.4 GHz
  • Max Heap - 512 MB
  • Start Heap - 256 MB 


Test scenarioDescriptionJMeter test results & metricsVes metricsK8s metrics
custom
  • 11.
1 RPS → 1 h
  • 1RPS → 1h

Image Modified

Image Modified

Image Modified

custom
  • 22.
2 RPS→ 30 min
  • 2RPS→ 30min

Image Modified

Image Modified

Image Modified

custom
30 RPS → 10 min
  • 30RPS → 10min

Image Modified

Image Modified

Image Modified

custom
35 RPS → 10 min
  • 35RPS → 10min

Image Modified

Image Modified

Image Modified

custom
45 RPS → 10 min
  • 45RPS → 10min

Image Modified

Image Modified

Image Modified

custom
50 RPS → 10 min
  • 50RPS → 10min

Image Modified

Image Modified

Image Modified

custom
80 RPS → 10 min
  • 80RPS → 10min

Image Modified

Image Modified

Image Modified

custom
120 RPS → 5 min
  • 120RPS → 5min

Image Modified

Image Modified

Image Modified

custom
130 RPS → 5 min
  • 130RPS → 5min

Image Modified

Image Modified

Image Modified


Test scenarioDescriptionJMeter test results & metricsVes metricsVes additional metrics
custom
  • 11RPS → 2days

Image Added

Image Added

Image Added

Presentation


View file
nameDCAEGEN2 VES performance tests.odp
height250


Replacing Cambria with DMaaP Client

Presentation


View file
nameVesCollectorEventBatchAndNewClient.pptx
height250


Performance Tests with real DMaaP

Environment 

  • CPU - 24 cores
  • CPU clock speed - 2.4 GHz
  • Max Heap - 512 MB
  • Start Heap - 256 MB

Ves with Dmaap client

3_test_scenario_50_rps_time_600.jmx50RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_50_rps_time_600.jmx50RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_50_rps_time_600.jmx50RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_100_rps_time_600.jmx100RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_100_rps_time_600.jmx100RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_100_rps_time_600.jmx100RPS → 10min

Image Added

Image Added

Image Added


Ves with Cambria client

Ves version :  1.9.1


3_test_scenario_50_rps_time_600.jmx50RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_50_rps_time_600.jmx50RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_50_rps_time_600.jmx50RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_100_rps_time_600.jmx100RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_100_rps_time_600.jmx100RPS → 10min

Image Added

Image Added

Image Added

3_test_scenario_100_rps_time_600.jmx100RPS → 10min

Image Added

Image Added

Image Added


Summary test results:

Environment 

  • CPU - 24 cores
  • CPU clock speed - 2.4 GHz
  • Max Heap - 512 MB
  • Start Heap - 256 MB




Average and Max Sync Processing Time (Client → VES)Average VES Processing timeError Rate [%]Max CPU Usage [%]
VES with Dmaap50th percentile95th percentile99th percentile


50RPS → 10min122ms, 349ms181ms, 3.23s591ms, 4.43s91ms036
124ms, 515ms204ms, 4.27s609ms, 5.48s90ms042
121ms, 399ms177ms, 2.17s561ms, 5.2291ms030
100RPS → 10min274ms, 7.06570ms, 7.93s1.0s, 8.06s139ms0.05%72
574ms, 6,07s1,17s, 14.19s1.89s, 15.37s201ms091
291ms, 5.88s415.5ms, 6.45s922ms, 11.3s143ms078
VES with Cambria50th percentile95th percentile99th percentile


50RPS → 10min118ms, 520ms174ms, 2.10s571ms, 6.48s90ms048
122ms, 548ms230ms, 5.17s581ms, 5.54s88ms032
123ms, 557ms194ms, 2.50s676ms, 5.12s88ms042
100RPS → 10min301ms, 5.79s772ms, 16.97s1.16s, 17.08s153ms088
340ms, 7.13s636ms, 17.87s1.15s, 18.48s149ms076
307ms, 8.29s506ms, 9.44s855ms, 9.78s155ms088



Conclusion:

Results of performance tests for both VES collector client implementation (DMaap client and Cambria client) are very similar.

Max CPU usage, Error rate, Average VES processing time, average and max sync processing time(Client → Ves) are almost the same.


In Ves collector with DMaap client(100RPS for 10 min),  appeared an error with the connection pool. In that specific case, we had a connection poll set to 16 and we got an error that the connection poll limits have been reached (stack trace in attachment).

We have to handle that kind of error in code.