Versions Compared

Old Version 1

changes.mady.by.user Ruslan Kashapov

Saved on

compared with

New Version Current

changes.mady.by.user Toine Siebelink

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Full test application flow is shown on diagram belowImage Removed

Drawio
bordertrue
diagramNamecps-load-test-flow-diagram
simpleViewerfalse
width
linksauto
tbstyletop
lboxtrue
diagramWidth831
revision1

Test Data

Yang model (schema set) based on:

...

Full data JSON is (taken as is from from RFC-8944 Appendix-B):

Update data:

...

Remote services were deployed as docker containers. Docker was running on same machine.Image Removed

Drawio
bordertrue
diagramNamecps-load-test-deployment-diagram
simpleViewerfalse
width
linksauto
tbstyletop
lboxtrue
diagramWidth761
revision1


Hardware details

Laptop DELL Latitude 5500

  • Processors: 8 × Intel® Core™ i7-8665U CPU @ 1.90GHz
  • Memory: 15,5 GiB of RAM
  • lshw > lshw.txt

Software details

OS

  • Operating System: Kubuntu 20.04
  • KDE Plasma Version: 5.18.5
  • KDE Frameworks Version: 5.68.0
  • Qt Version: 5.12.8
  • Kernel Version: 5.4.0-70-generic
  • OS Type: 64-bit

...

  • OpenJDK version 13.0.4, 2020-07-14
  • OpenJDK Runtime Environment (build 13.0.4+8-Ubuntu-120.04)
  • OpenJDK 64-Bit Server VM (build 13.0.4+8-Ubuntu-120.04, mixed mode)
  • java -XX:+PrintFlagsFinal -version > java-print-flags-final.txt


Test Results

CPS revision

Tested CPS from a master branch at state on  
representing the Honolulu 1.0.1 + DELETE ANCHOR functionality targeted for Istanbul release

Operation execution time

The operation per seconds was calculated as 1000/(exec time in millis).

OperationEmbedded FlowRemote Flow

Execution time, millisecondsOperations per secondExecution time, millisecondsOperations per second

MinMaxAverageMinMaxAverageMinMaxAverageMinMaxAverage
Threads: 2
Circles: 500
CREATE ANCHOR0.99229.1112.07834.3511008.366481.18561.87152.43769.086.5616.16314.476
CREATE NODE13.171600.11523.8981.66675.92341.84474.617201.42185.4994.96513.40211.696
UPDATE NODE1.9227.5384.30636.314520.911232.24862.098249.31970.4964.01116.10414.185
DELETE ANCHOR2.55121.7134.35346.055392.038229.70462.77144.29971.0536.9315.93114.074

Threads: 10
Circles: 100

CREATE_ANCHOR1.64678.2076.78912.787607.464147.2966.84326.61140.8443.06214.9617.1
CREATE_NODE17.1771115.065105.9860.89758.2199.43582.275325.542170.053.07212.1545.881
UPDATE_NODE3.218115.99115.5168.621310.78864.44963.226256.484141.4893.89915.8167.068
DELETE_ANCHOR3.5653.77611.74218.596280.90485.16364.741411.13143.7972.43215.4466.954
Threads: 20
Circles: 50
CREATE_ANCHOR1.891177.36128.4125.638528.77935.19679.077568.126297.9161.7612.6463.357
CREATE_NODE20.4261781.881186.9420.56148.9585.34986.274978.945376.1451.02211.5912.659
UPDATE_NODE3.623401.14186.0792.493275.97811.61773.062615.295297.5391.62513.6873.361
DELETE_ANCHOR3.937114.20518.5548.756253.9953.89664.017588.028299.1711.70115.6213.343
Threads: 50
Circles: 20
CREATE_ANCHOR2.498805.03789.2721.242400.32811.202129.3122165.287788.8260.4627.7331.268
CREATE_NODE41.463317.529435.9170.30124.122.294115.4052403.888929.2740.4168.6651.076
UPDATE_NODE6.0391127.585323.5450.887165.5823.09186.8231729.146784.1390.57811.5181.275
DELETE_ANCHOR6.412364.36732.3142.744155.95630.94680.5362033.905785.3580.49212.4171.273


Resource usage

Below are resource usage diagrams (VisualVM monitor screenshots) for used Threads/Circles combinations.

Embedded then remote load tests were performed via single application execution (see test flow diagram above)

Threads: 2
Circles: 500		Threads: 10
Circles: 100
Image Removed

Image Added

Image Removed
Image Added


Threads: 20
Circles: 50		Threads: 50
Circles: 20
Image Removed

Image Added

Image Removed

Image Added

Following (calculated by default) heap settings were used:

...

Full dump of JVM flags is listed in attachment java-print-flags-final.txt


Test Application

Source

The source of test application is attached: cps-load-test-application.zip

Setting up

Unzip into cps folder.

Update root pom.xml file to include cps-load-test-application

Code Block
languagexml
titlecps/pom.xml
collapsetrue
...
    <modules>
        <module>cps-dependencies</module>
        <module>cps-bom</module>
        <module>cps-parent</module>
        <module>cps-service</module>
        <module>cps-rest</module>
        <module>cps-ncmp-service</module>
        <module>cps-ncmp-rest</module>
        <module>cps-ri</module>
        <module>checkstyle</module>
        <module>spotbugs</module>
        <module>cps-application</module>
        <!-- add following-line -->
        <module>cps-load-test-application</module>
    </modules>
...

Refresh the maven project. New module will appear as on screenshot below

Image Added

Create run configuration using same environment variables as for core CPS Application

Image Added

Tests related configuration is allocated within cps-load-test-application/src/main/resources/application.yml in load-test section 
like below

Code Block
languageyml
titleapplication.yml
collapsetrue
load-test:
    preset:
        dataspace: test-dataspace
        schema-set: test-schema-set
        anchor-prefix: test-anchor-
    embedded:
        enabled: true
    remote:
        enabled: true
        # resources zip file is corrupted on build, so referencing original from resources
        resources-zip: src/main/resources/yang/network-topology.zip
        base-url: http://localhost:8883/cps/api/
        auth:
            username: ***
            password: ***
    # number of threads
    threads: 50
    # number of times the flow repeated within a thread
    circles: 20
    # thread termination timeout
    termination-timeout-seconds: 1200

spring:
...