Table of Contents

Setting up Stability Tests in APEX

...

Code Block
mkdir jMeter cd jMeter wget http://mirrors.whoishostingthis.com/apache//jmeter/binaries/apache-jmeter-5.1.1.zip unzip apache-jmeter-5.1.1.zip

Run JMeter

Code Block
/home/ubuntu/jMeter/apache-jmeter-5.1.1/bin/jmeter

The above command will load the JMeter UI. Then navigate to File → Open → Browse and select the test plan jmx file to open. The jmx file is present in the apex-pdp git repository - jmx file path

Install apex-pdp in virtual machine

...

Install apex-pdp in virtual machine

We will be running apex-pdp as docker container. So we need to first install docker and then create the container hosting apex-pdp by pulling the image from ONAP repository.

...

Run the below command to create the container hosting apex-pdp by pulling the image from ONAP repository.

Code Block

language	bash

docker run -d --name apex -p 12561:12561 -p 23324:23324 -it nexus3.onap.org:10001/onap/policy-apex-pdp:2.1.0-SNAPSHOT-latest /bin/bash -c "/opt/app/policy/apex-pdp/bin/apexApps.sh jmx-test -c /opt/app/policy/apex-pdp/examples/config/SampleDomain/RESTServerJsonEvent.json"
docker ps

Note: If you observe that requests from JMeter client is failing due to timeout, then modify the "RESTServerJsonEvent.json" mentioned in the above command and increase the "synchronousTimeout" property as per needed.

...

Sample Screenshot of visualVM

Image RemovedImage Added

Test Plan

The 72 hours stability test will run the following steps in 20 threaded loop.

...

HTTP Header Manager - used to store headers which will be used for making HTTP requests.
HTTP Request Defaults - used to store HTTP request details like Server Name or IP, Port, Protocol etc.

User Defined Variables - used to store following user defined parameters.

Name	Description	Default Value
wait	Wait time after each request (in milliseconds)	500
threads	Number of threads to run test cases in parallel.	20
threadsTimeOutInMs	Synchronization timer for threads running in parallel (in milliseconds).	5000

Screenshot of Download and update the jmx file presented in the apex-pdp stability test planImage Removedgit repository - jmx file path.

HTTPSampler.domain - The ip address of VM which the apex container is running
HTTPSampler.port - The listening port, here is 23324
ThreadGroup.druation - Set the duration to 72 hours

Use the CLI mode to start the test

Code Block

language	bash

./jmeter.sh -n -t ~/apexPdpStabilityTestPlan.jmx -Jusers=1 -l ~/stability.log

Stability Test Result

Summary

Stability test plan was triggered for 72 hours injecting input events to apex-pdp from 20 client threads running in JMeter.

After the test stop, we can generate a HTML testreport via command

Code Block
~/jMeter/apache-jmeter-5.1.1/bin/jmeter -g stability.log -o ./result/

result.zip stability.zip onap.zip

Number of Client Threads running in JMeter	Number of Server Threads running in Apex engine	Total number of input events	Success Percentage	Error Percentage
20	4	6394602	99.999971%	0.0029%

Setting up Performance Tests in APEX

...

Code Block

language	js
title	Example APEX performance metrics
collapse	true

 {
  "totalStats": {
    "batchNumber": -1,
    "batchSize": 1000,
    "apexClient": "TOTAL",
    "eventsNotSent": 0,
    "eventsSent": 1000,
    "eventsNotReceived": 0,
    "eventsReceived": 1000,
    "averageRoundTripNano": 159434725,
    "shortestRoundTripNano": 23059921,
    "longestRoundTripNano": 1106994552,
    "averageApexExecutionNano": 26780360,
    "shortestApexExecutionNano": 1066002,
    "longestApexExecutionNano": 323934929
  },
  "batchStatsList": [
    {
      "batchNumber": 0,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 995018520,
      "shortestRoundTripNano": 835971769,
      "longestRoundTripNano": 1106994552,
      "averageApexExecutionNano": 241706832,
      "shortestApexExecutionNano": 73723978,
      "longestApexExecutionNano": 323934929
    },
    {
      "batchNumber": 1,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 204429789,
      "shortestRoundTripNano": 56234319,
      "longestRoundTripNano": 305690768,
      "averageApexExecutionNano": 89544892,
      "shortestApexExecutionNano": 7785682,
      "longestApexExecutionNano": 177171848
    },
    {
      "batchNumber": 2,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 160524927,
      "shortestRoundTripNano": 41179914,
      "longestRoundTripNano": 266809149,
      "averageApexExecutionNano": 68178648,
      "shortestApexExecutionNano": 2784005,
      "longestApexExecutionNano": 160119899
    },
    {
      "batchNumber": 3,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 123442425,
      "shortestRoundTripNano": 32320688,
      "longestRoundTripNano": 204549694,
      "averageApexExecutionNano": 35810567,
      "shortestApexExecutionNano": 2325630,
      "longestApexExecutionNano": 102168076
    },
    {
      "batchNumber": 4,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 134903088,
      "shortestRoundTripNano": 61195662,
      "longestRoundTripNano": 205959446,
      "averageApexExecutionNano": 17987078,
      "shortestApexExecutionNano": 2243213,
      "longestApexExecutionNano": 39218327
    },
    {
      "batchNumber": 5,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 115759297,
      "shortestRoundTripNano": 46415800,
      "longestRoundTripNano": 180262438,
      "averageApexExecutionNano": 12719990,
      "shortestApexExecutionNano": 1868046,
      "longestApexExecutionNano": 32387313
    },
    {
      "batchNumber": 6,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 130974073,
      "shortestRoundTripNano": 60415786,
      "longestRoundTripNano": 191390626,
      "averageApexExecutionNano": 7577036,
      "shortestApexExecutionNano": 1888337,
      "longestApexExecutionNano": 21813501
    },
    {
      "batchNumber": 7,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 115724681,
      "shortestRoundTripNano": 23059921,
      "longestRoundTripNano": 170915878,
      "averageApexExecutionNano": 14406756,
      "shortestApexExecutionNano": 1402961,
      "longestApexExecutionNano": 57016237
    },
    {
      "batchNumber": 8,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 87268333,
      "shortestRoundTripNano": 29857934,
      "longestRoundTripNano": 142114113,
      "averageApexExecutionNano": 8295938,
      "shortestApexExecutionNano": 1292335,
      "longestApexExecutionNano": 22250460
    },
    {
      "batchNumber": 9,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38430",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 89515572,
      "shortestRoundTripNano": 27979472,
      "longestRoundTripNano": 151246340,
      "averageApexExecutionNano": 2988598,
      "shortestApexExecutionNano": 1251419,
      "longestApexExecutionNano": 10779438
    },
    {
      "batchNumber": 10,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 161559717,
      "shortestRoundTripNano": 54052523,
      "longestRoundTripNano": 220095891,
      "averageApexExecutionNano": 2439668,
      "shortestApexExecutionNano": 1413877,
      "longestApexExecutionNano": 7713140
    },
    {
      "batchNumber": 11,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38428",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 102126750,
      "shortestRoundTripNano": 43249960,
      "longestRoundTripNano": 147393457,
      "averageApexExecutionNano": 4201824,
      "shortestApexExecutionNano": 1343794,
      "longestApexExecutionNano": 20132831
    },
    {
      "batchNumber": 12,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 94621079,
      "shortestRoundTripNano": 44447504,
      "longestRoundTripNano": 136784978,
      "averageApexExecutionNano": 2053586,
      "shortestApexExecutionNano": 1314753,
      "longestApexExecutionNano": 4677176
    },
    {
      "batchNumber": 13,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 95569040,
      "shortestRoundTripNano": 45539798,
      "longestRoundTripNano": 131660842,
      "averageApexExecutionNano": 3325890,
      "shortestApexExecutionNano": 1711712,
      "longestApexExecutionNano": 14248153
    },
    {
      "batchNumber": 14,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 118841955,
      "shortestRoundTripNano": 41387040,
      "longestRoundTripNano": 164058281,
      "averageApexExecutionNano": 8456753,
      "shortestApexExecutionNano": 1066002,
      "longestApexExecutionNano": 40687580
    },
    {
      "batchNumber": 15,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 98047224,
      "shortestRoundTripNano": 53382064,
      "longestRoundTripNano": 137689355,
      "averageApexExecutionNano": 3103221,
      "shortestApexExecutionNano": 1642711,
      "longestApexExecutionNano": 13629027
    },
    {
      "batchNumber": 16,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 89551458,
      "shortestRoundTripNano": 43607586,
      "longestRoundTripNano": 125304371,
      "averageApexExecutionNano": 4844014,
      "shortestApexExecutionNano": 1753670,
      "longestApexExecutionNano": 18681287
    },
    {
      "batchNumber": 17,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 98811396,
      "shortestRoundTripNano": 41244373,
      "longestRoundTripNano": 141306528,
      "averageApexExecutionNano": 2908269,
      "shortestApexExecutionNano": 1398919,
      "longestApexExecutionNano": 10733647
    },
    {
      "batchNumber": 18,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38450",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 85947612,
      "shortestRoundTripNano": 40228871,
      "longestRoundTripNano": 124690954,
      "averageApexExecutionNano": 2324032,
      "shortestApexExecutionNano": 1478128,
      "longestApexExecutionNano": 5606719
    },
    {
      "batchNumber": 19,
      "batchSize": 50,
      "apexClient": "localhost(127.0.0.1):38432",
      "eventsNotSent": 0,
      "eventsSent": 50,
      "eventsNotReceived": 0,
      "eventsReceived": 50,
      "averageRoundTripNano": 86057583,
      "shortestRoundTripNano": 45376757,
      "longestRoundTripNano": 119651361,
      "averageApexExecutionNano": 2733608,
      "shortestApexExecutionNano": 1704628,
      "longestApexExecutionNano": 12086274
    }
  ]
}

Performance Test Result

Summary

Performance test was triggered for 2 hours on a 4 core, 4GB RAM virtual machine.

...

Version	Old Version 14	New Version 15
Changes made by	Yehui Wang	Tim Huang
Saved on	May 21, 2019	May 28, 2019

Versions Compared

Key

Setting up Stability Tests in APEX

Install apex-pdp in virtual machine

Install apex-pdp in virtual machine

Test Plan

Stability Test Result

Summary

Setting up Performance Tests in APEX

Performance Test Result

Summary

Page Comparison

Versions Compared

Key

Setting up Stability Tests in APEX

Install apex-pdp in virtual machine

Install apex-pdp in virtual machine

Test Plan

Stability Test Result

Summary

Setting up Performance Tests in APEX

Performance Test Result

Summary