Setting up Performance Tests in APEX

The apex-pdp has built in support for performance testing. A special performance testing REST server is available in the code base for performance testing. It is in the module performance-benchmark-test. To execute a benchmark test, you start the REST server, and then configure and run APEX against the server. There are example configurations for running tests in the resources of this module.

Here is an example of how to do this:

Clone and build the apex-pdp git repo

Go into the performance-benchmark-test module and run the REST server

cd testsuites/performance/performance-benchmark-test
mvn exec:java -Dexec.mainClass="org.onap.policy.apex.testsuites.performance.benchmark.eventgenerator.EventGenerator" -Dexec.args="-c  src/main/resources/examples/benchmark/EventGeneratorConfig.json"

Separately, create a local directory and unzip the APEX tarball

mkdir apex
cd apex
tar zxvf ~/git/onap/policy/apex-pdp/packages/apex-pdp-package-full/target/*gz

Run APEX with a configuration that runs against the benchmark REST server

bin/apexApps.sh engine -c ~/git/onap/policy/apex-pdp/testsuites/performance/performance-benchmark-test/src/main/resources/examples/benchmark/Javascript64.json

To get the test results, Issue the following command using CURL or from a browser
```
curl http://localhost:32801/EventGenerator/Stats
```

The results are similar to those below:

Example APEX performance metrics

 {
  "totalStats": {
    "batchNumber": -1,
    "batchSize": 1000,
    "apexClient": "TOTAL",
    "eventsNotSent": 0,
    "eventsSent": 1000,
    "eventsNotReceived": 0,
    "eventsReceived": 1000,
    "averageRoundTripNano": 159434725,
    "shortestRoundTripNano": 23059921,
    "longestRoundTripNano": 1106994552,
    "averageApexExecutionNano": 26780360,
    "shortestApexExecutionNano": 1066002,
    "longestApexExecutionNano": 323934929
  },
  "batchStatsList": [
    {
      "batchNumber": 0,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 995018520,
      "shortestRoundTripNano": 835971769,
      "longestRoundTripNano": 1106994552,
      "averageApexExecutionNano": 241706832,
      "shortestApexExecutionNano": 73723978,
      "longestApexExecutionNano": 323934929
    },
    {
      "batchNumber": 1,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 204429789,
      "shortestRoundTripNano": 56234319,
      "longestRoundTripNano": 305690768,
      "averageApexExecutionNano": 89544892,
      "shortestApexExecutionNano": 7785682,
      "longestApexExecutionNano": 177171848
    },
    {
      "batchNumber": 2,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 160524927,
      "shortestRoundTripNano": 41179914,
      "longestRoundTripNano": 266809149,
      "averageApexExecutionNano": 68178648,
      "shortestApexExecutionNano": 2784005,
      "longestApexExecutionNano": 160119899
    },
    {
      "batchNumber": 3,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 123442425,
      "shortestRoundTripNano": 32320688,
      "longestRoundTripNano": 204549694,
      "averageApexExecutionNano": 35810567,
      "shortestApexExecutionNano": 2325630,
      "longestApexExecutionNano": 102168076
    },
    {
      "batchNumber": 4,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 134903088,
      "shortestRoundTripNano": 61195662,
      "longestRoundTripNano": 205959446,
      "averageApexExecutionNano": 17987078,
      "shortestApexExecutionNano": 2243213,
      "longestApexExecutionNano": 39218327
    },
    {
      "batchNumber": 5,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 115759297,
      "shortestRoundTripNano": 46415800,
      "longestRoundTripNano": 180262438,
      "averageApexExecutionNano": 12719990,
      "shortestApexExecutionNano": 1868046,
      "longestApexExecutionNano": 32387313
    },
    {
      "batchNumber": 6,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 130974073,
      "shortestRoundTripNano": 60415786,
      "longestRoundTripNano": 191390626,
      "averageApexExecutionNano": 7577036,
      "shortestApexExecutionNano": 1888337,
      "longestApexExecutionNano": 21813501
    },
    {
      "batchNumber": 7,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 115724681,
      "shortestRoundTripNano": 23059921,
      "longestRoundTripNano": 170915878,
      "averageApexExecutionNano": 14406756,
      "shortestApexExecutionNano": 1402961,
      "longestApexExecutionNano": 57016237
    },
    {
      "batchNumber": 8,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 87268333,
      "shortestRoundTripNano": 29857934,
      "longestRoundTripNano": 142114113,
      "averageApexExecutionNano": 8295938,
      "shortestApexExecutionNano": 1292335,
      "longestApexExecutionNano": 22250460
    },
    {
      "batchNumber": 9,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 89515572,
      "shortestRoundTripNano": 27979472,
      "longestRoundTripNano": 151246340,
      "averageApexExecutionNano": 2988598,
      "shortestApexExecutionNano": 1251419,
      "longestApexExecutionNano": 10779438
    },
    {
      "batchNumber": 10,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 161559717,
      "shortestRoundTripNano": 54052523,
      "longestRoundTripNano": 220095891,
      "averageApexExecutionNano": 2439668,
      "shortestApexExecutionNano": 1413877,
      "longestApexExecutionNano": 7713140
    },
    {
      "batchNumber": 11,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 102126750,
      "shortestRoundTripNano": 43249960,
      "longestRoundTripNano": 147393457,
      "averageApexExecutionNano": 4201824,
      "shortestApexExecutionNano": 1343794,
      "longestApexExecutionNano": 20132831
    },
    {
      "batchNumber": 12,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 94621079,
      "shortestRoundTripNano": 44447504,
      "longestRoundTripNano": 136784978,
      "averageApexExecutionNano": 2053586,
      "shortestApexExecutionNano": 1314753,
      "longestApexExecutionNano": 4677176
    },
    {
      "batchNumber": 13,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 95569040,
      "shortestRoundTripNano": 45539798,
      "longestRoundTripNano": 131660842,
      "averageApexExecutionNano": 3325890,
      "shortestApexExecutionNano": 1711712,
      "longestApexExecutionNano": 14248153
    },
    {
      "batchNumber": 14,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 118841955,
      "shortestRoundTripNano": 41387040,
      "longestRoundTripNano": 164058281,
      "averageApexExecutionNano": 8456753,
      "shortestApexExecutionNano": 1066002,
      "longestApexExecutionNano": 40687580
    },
    {
      "batchNumber": 15,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 98047224,
      "shortestRoundTripNano": 53382064,
      "longestRoundTripNano": 137689355,
      "averageApexExecutionNano": 3103221,
      "shortestApexExecutionNano": 1642711,
      "longestApexExecutionNano": 13629027
    },
    {
      "batchNumber": 16,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 89551458,
      "shortestRoundTripNano": 43607586,
      "longestRoundTripNano": 125304371,
      "averageApexExecutionNano": 4844014,
      "shortestApexExecutionNano": 1753670,
      "longestApexExecutionNano": 18681287
    },
    {
      "batchNumber": 17,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 98811396,
      "shortestRoundTripNano": 41244373,
      "longestRoundTripNano": 141306528,
      "averageApexExecutionNano": 2908269,
      "shortestApexExecutionNano": 1398919,
      "longestApexExecutionNano": 10733647
    },
    {
      "batchNumber": 18,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38450",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 85947612,
      "shortestRoundTripNano": 40228871,
      "longestRoundTripNano": 124690954,
      "averageApexExecutionNano": 2324032,
      "shortestApexExecutionNano": 1478128,
      "longestApexExecutionNano": 5606719
    },
    {
      "batchNumber": 19,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 86057583,
      "shortestRoundTripNano": 45376757,
      "longestRoundTripNano": 119651361,
      "averageApexExecutionNano": 2733608,
      "shortestApexExecutionNano": 1704628,
      "longestApexExecutionNano": 12086274
    }
  ]
}

Policy APEX-PDP Stability and Performance Tests

Setting up Performance Tests in APEX