NCMP De-registration Performance test guide

Manual testing with Postman

1. Build cps-and-ncmp docker image for version to be tested. Run maven from cps repo directory:

   mvn clean install -Pcps-ncmp-docker

    

2. Edit docker-compose/docker-compose.yml to filter notifications: set NOTIFICATION_DATASPACE_FILTER_PATTERNS to NON-EXISTING-DATASPACE

   cps-and-ncmp: environment:     notification.enabled: 'true'     NOTIFICATION_DATASPACE_FILTER_PATTERNS: 'NON-EXISTING-DATASPACE'

    

3. Start docker-compose with DMI plugin stub and monitoring enabled:

   docker-compose -f docker-compose/docker-compose.yml --profile dmi-stub --profile monitoring up

    

4. Wait for CPS to start. Console output will show:

    

5. Test using Postman

   1. Start Postman, and import the NCMP de-register performance postman collection. (This step only needs to be done once.)
      
      

      Open

      

      
      
      
      

   2. Click the new collection, and open the "Variables" tab
      
      

      Open

      

      
      
      
      

   3. Change the current value of TOTAL_CMHANDLES to the desired value, e.g. 10000, and click the "Save" button. This is the number of CM handles that will be registered (and later de-registered).
      
      

      
      
      
      

   4. Select "Register CM handles", and click "Send". (The Postman collection will use a script to automatically create the request body with 10000 CM handles.)
      
      

      
      
      
      

   5. Wait until all CM handles are in READY state. This can take a while, especially with lot of handles.
      We can check how many handles are ready by running a query on the database. Here are instructions using the Postgres command line interface (psql), but you may alternatively use the Postgres GUI client.

      1. Open a new terminal window, and install postgres-client if needed:

          

      2. Run psql command to connect to the database (default password is cps):

          

      3. Run the following query to count the number of handles in the READY state. You will need to repeat this command, until all are ready (e.g. 10000).

          

      4. Now that all CM handles are ready, we can proceed to test de-registration. Go back to Postman.
         
         
         

   6. In Postman, select "De-Register CM handles", and click "Send".
      
      

   7. Record the time taken (54.91 s in this case).
      
      

6. Stop docker-compose:

    

7. Clean up docker containers and volumes:

    

Automated testing

There is a script in the cps repo for automatic testing of NCMP handle de-registration. The script will test registering and de-registering 100, 500, 1000, 5000, 10,000 and 20,000 by default. To use it, simply run the script.

Before testing, you may need to edit docker-compose/docker-compose.yml to filter notifications: set NOTIFICATION_DATASPACE_FILTER_PATTERNS to NON-EXISTING-DATASPACE

Run the script:

While the script is running, you may monitor the progress in another terminal. A summary report will be generated at test-tools/metrics-reports/deregister-summary-{CURRENT-DATETIME}.tsv:

The script will also produce Prometheus metrics reports for each individual test, stored in test-tools/metrics-reports. For example, this report is generated for de-registering 1000 out of 1000 CM handles: deregister-2023-03-24T17:29:20+00:00-1000-1000.tsv