NCMP De-registration Performance test guide

Manual testing with Postman

Build cps-and-ncmp docker image for version to be tested. Run maven from cps repo directory:
```
mvn clean install -Pcps-ncmp-docker
```

Edit docker-compose/docker-compose.yml to filter notifications: set NOTIFICATION_DATASPACE_FILTER_PATTERNS to NON-EXISTING-DATASPACE

cps-and-ncmp:
  environment:
    notification.enabled: 'true'
    NOTIFICATION_DATASPACE_FILTER_PATTERNS: 'NON-EXISTING-DATASPACE'

Start docker-compose with DMI plugin stub and monitoring enabled:

docker-compose -f docker-compose/docker-compose.yml --profile dmi-stub --profile monitoring up

Wait for CPS to start. Console output will show:

cps-and-ncmp            | {"logTimeStamp":"2023-03-24T16:20:36.705Z","logLevel":"INFO","principalId":"cpsuser","serviceName":"cps-application","message":"Started Application in 47.336 seconds (JVM running for 48.444)","processId":"1","threadName":"main","class":"org.onap.cps.Application"}

Test using Postman
1. Start Postman, and import the NCMP de-register performance postman collection. (This step only needs to be done once.)
2. Click the new collection, and open the "Variables" tab
3. Change the current value of TOTAL_CMHANDLES to the desired value, e.g. 10000, and click the "Save" button. This is the number of CM handles that will be registered (and later de-registered).
4. Select "Register CM handles", and click "Send". (The Postman collection will use a script to automatically create the request body with 10000 CM handles.)
5. Wait until all CM handles are in READY state. This can take a while, especially with lot of handles.
  We can check how many handles are ready by running a query on the database. Here are instructions using the Postgres command line interface (psql), but you may alternatively use the Postgres GUI client.
  1. Open a new terminal window, and install postgres-client if needed:
    $ sudo apt install postgresql-client Reading package lists... Done Building dependency tree... Done Reading state information... Done postgresql-client is already the newest version (14+238). 0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
  2. Run psql command to connect to the database (default password is cps):
    $ psql -h localhost -p 5432 cpsdb cps Password for user cps: psql (14.6 (Ubuntu 14.6-0ubuntu0.22.04.1), server 14.1) Type "help" for help. cpsdb=#
  3. Run the following query to count the number of handles in the READY state. You will need to repeat this command, until all are ready (e.g. 10000).
    cpsdb=# SELECT count(*) FROM public.fragment where attributes @> '{"cm-handle-state": "READY"}'; count ------- 3540 (1 row) cpsdb=# SELECT count(*) FROM public.fragment where attributes @> '{"cm-handle-state": "READY"}'; count ------- 10000 (1 row)
  4. Now that all CM handles are ready, we can proceed to test de-registration. Go back to Postman.
6. In Postman, select "De-Register CM handles", and click "Send".
7. Record the time taken (54.91 s in this case).

Stop docker-compose:

docker-compose -f docker-compose/docker-compose.yml down

Clean up docker containers and volumes:

docker container prune -f && docker volume prune -f

Automated testing

There is a script in the cps repo for automatic testing of NCMP handle de-registration. The script will test registering and de-registering 100, 500, 1000, 5000, 10,000 and 20,000 by default. To use it, simply run the script.

Before testing, you may need to edit docker-compose/docker-compose.yml to filter notifications: set NOTIFICATION_DATASPACE_FILTER_PATTERNS to NON-EXISTING-DATASPACE

cps-and-ncmp:
  environment:
    notification.enabled: 'true'
    NOTIFICATION_DATASPACE_FILTER_PATTERNS: 'NON-EXISTING-DATASPACE'

Run the script:

$ ./test-tools/test-deregistration.sh
~/Work/ONAP/cps/cps/test-tools ~/Work/ONAP/cps/cps
Testing deregistration of 100 out of 100 CM handles
Restarting docker
Removing grafana-container    ... done
Removing kafka                ... done
Removing cps-and-ncmp         ... done
Removing zookeeper            ... done
Removing dbpostgresql         ... done
Removing ncmp-dmi-plugin-stub ... done
Removing prometheus-container ... done
Removing network docker-compose_default
Creating network "docker-compose_default" with the default driver
Creating volume "docker-compose_grafana" with default driver
Creating dbpostgresql         ... done
Creating ncmp-dmi-plugin-stub ... done
Creating zookeeper            ... done
Creating prometheus-container ... done
Creating kafka                ... done
Creating cps-and-ncmp         ... done
Creating grafana-container    ... done
Waiting for CPS to start
Creating request bodies
[2023-03-24T17:24:19+00:00] Creating CM handles
Waiting for CM handles to be in READY state
There are 0 CM handles in READY state.
There are 100 CM handles in READY state.
Grabbing metrics before deregistration
[2023-03-24T17:25:23+00:00] Removing CM handles
There are 0 CM handles still in READY state.
Grabbing metrics after deregistration
Generating metrics report

Testing deregistration of 500 out of 500 CM handles
Restarting docker
...

While the script is running, you may monitor the progress in another terminal. A summary report will be generated at test-tools/metrics-reports/deregister-summary-{CURRENT-DATETIME}.tsv:

$ tail -f test-tools/metrics-reports/deregister-summary-2023-03-24T17\:23\:28+00\:00.tsv 
Removed	Total	Time
 100	 100	1.099639
 500	 500	3.452250
1000	1000	6.132099
5000	5000	25.372256

The script will also produce Prometheus metrics reports for each individual test, stored in test-tools/metrics-reports. For example, this report is generated for de-registering 1000 out of 1000 CM handles: deregister-2023-03-24T17:29:20+00:00-1000-1000.tsv