Versions Compared

Version Old Version 1 New Version Current
Changes made by

James Hahn

James Hahn

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Summary

Stability Performance test plan was triggered for 72 hours injecting vCPE, vFirewall, vDNS, VOLTE onset & abatement (where applicable) messages through REST interface of Drools PDP installed in windriver lab (PFPP).  This was running on a kubernetes host, using the m2.xlarge spec, having the following configuration:

  • 32GB RAM
  • 8 VCPU
  • 160GB Disk

The test environment was set-up using simulated components i.e. A&AI, VFC and SO using "features enable controlloop-utils".  However the performance test subscribes to the POLICY-CL-MGT topic, which is not simulated, to determine when the PDP has completed various steps.

The performance test runs 10 simultaneous threads injecting ONSET messages.  In each thread, each of the different types of ONSETs is injected and processed in a serial fashion, first vCPE is injected and its associated APPC requests are answered, then vFirewall is injected, and so on.  Each thread repeats this process 1000 times.  APPC response messages were injected in the test plan wherever needed.

Average processing time was calculated using the last 10000 records from audit.log for each use case, as well as the matching transaction time extracted from the network.log, using the below scripthttps://gerrit.onap.org/r/gitweb?p=policy/drools-applications.git;a=blob;f=testsuites/stability/src/main/resources/amsterdam/generate_performace_report.sh;h=846628543b5a127c49f57a1fa1f4a254dfff64da;hb=refs/heads/master:

generate-mt-performance-report


Result

Time taken by Drools PDP

...

...

CPU Utilization

Code Block
Average:Elapsed time for vCPE :
  matched CPU8870 samples, average 18 ms
%user  unmatched 1130 samples, %niceaverage 339 ms
%system
Elapsed time %iowaitfor    %steal     %idle
AveragevFirewall :
  matched 8871 samples, average 55 allms
  unmatched 1129 samples, 0.84average 177 ms

Elapsed time 0.00for vDNS :
  matched 0.238869 samples, average 9 ms
 0.02 unmatched 1131 samples, average 16 0.00ms

Elapsed time for 98.91VOLTE Average:
  matched 8868 samples, average 9 ms
 0 unmatched 1132 samples, average 15 1.16      0.00      0.30      0.00      0.00     98.54
Average:          1      1.00      0.00      0.25      0.00      0.00     98.75
Average:          2      1.21      0.00      0.33      0.00      0.00     98.46
Average:  ms


Note: the “unmatched samples” are requests for which no corresponding network time could be identified by the reporting tool.  Hence those numbers represent the total elapsed time, with nothing subtracted out.


CPU Utilization

Total CPU used by the PDP was measured before and after the test, using "ps -l".

Initial CPU timeFinal CPU timeTotal CPU used during testAverage CPU per ONSET
00:36:5500:43:28393 ms9.8 ms


Memory Utilization

Code Block
Number of young garbage collections used during the test: 1468
Avg. Young garbage collection time: ~5.8 ms per collection
Total number of Full garbage collection: 3
Avg. Full garbage collection time: ~112 ms per collection

S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC 3    MU  0.78  CCSC   CCSU 0.00   YGC   0.18  YGCT    0.00FGC     FGCT 0.00    GCT
99.03
Average:          4      0.77      0.00      0.18      0.00      0.00     99.04
Average:          5      0.80      0.00      0.28      0.12      0.00     98.80
Average:          6      0.54      0.00      0.13      0.00      0.00     99.33
Average:          72048.0 2048.0  0.0   704.0  68096.0  59110.3   432640.0   86116.6   73344.0 71849.4 8320.0 7801.2    321    2.033   3      0.49337    2.370
2560.0 0.00      0.14    1536.0  0.00      0.00     99.37

...

Code Block
Number of young garbage collection: 6711
Avg. Young garbage collection time: ~7.09 ms per collection
Number of Full garbage collection: 4
Avg. Full garbage collection time: ~197 ms per collection

 S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT  
0.00  90.62  45.71  21.36  97.74  93.42   6711   47.578     41088.0 93184.0  82890.4   432640.0   143366.1  73344.0 71882.9 8320.0 7804.8   1789   10.564   3      0.788337   4810.366901

Performance Metrics

Single with only a single policy executing

No.

Metric

Description

ResultComments
1
4Maximum Simultaneous ExecutionsMeasure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization10DMaaP connection limitations prevented the test from running more than 10 simultaneous threads/ONSETs
5Multi-Threaded Response TimeMeasure the execution time for onset and abatement in each use case when multiple threads are injecting ONSET events simultaneously

vCPE - 3 18 ms

vFirewall - 2 55 ms

vDNS - 8 9 ms

VOLTE - 7 9 ms

The average time taken is calculated based on last 10,000 samples in audit.log file.
2Single 6Multi Threaded CPU UsageCPU Usage for each use case when executing alone~ 1%3Single multiple threads are injecting ONSET events simultaneously9.8 ms
7Multi Threaded Memory UsageMemory Usage Memory Usage for each use case when executing alone~ 100 MB4Maximum Simultaneous ExecutionsMeasure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization2The current size of source/sink topics is 10. And 1 onset message results to 5 messages in sink topic. Hence we can only run 2 simaltaneous thread and assert the values coming in sink topics.multiple threads are injecting ONSET events simultaneously