CPS-2637 Realistic Public & Private Properties in K6 tests

References

https://lf-onap.atlassian.net/browse/CPS-2637

Overview

This document captures the performance test results for CM-handle operations and NCMP overhead measurements. The goal of these tests is to evaluate the efficiency of various CM-handle operations, including registration, search, de-registration, and synchronous pass-through operations, across different filtering criteria.

The test results compare the master version with the CPS-2637 Commit version, highlighting performance variations.

Test Description

The following tests were conducted:

CM-handle Registration & De-registration: Measures the rate (handles/second) at which CM-handles are registered and removed.
CM-handle ID & Property-Based Searches: Evaluates search performance across different filter types such as No filter, Module filter, Property filter, Cps Path filter, and Trust Level filter.
NCMP Overhead: Measures the latency of synchronous CM-handle pass-through read and write operations.
Legacy Batch Read: Measures the efficiency of retrieving batch data in terms of events processed per second.

Each test was executed using predefined CM-handle properties to maintain consistency across test runs.

Performance Testing Summary

CM-Handle Properties Used for Testing

The following properties were assigned to CM-handles during testing:

Public Properties: These are standard attributes assigned to every CM-handle, including systemId (unique numerical identifier) and systemName (used for no-filter queries).
Private Properties: These are CM-specific attributes such as segmentId, networkSegment, deviceIdentifier, hardwareVersion, softwareVersion, syncStatus, and nodeCategory.

The detailed property table is included below for reference.

Cm Handle Public Properties

Proposed Property Name	Data Type	Data Size	Property Value	Description

Proposed Property Name	Data Type	Data Size	Property Value	Description
systemId	Integer	4 bytes	2	Unique numerical ID (auto-incremented) EMS identifier
systemName	String	UTF-8, 10-20 bytes	NCMP	For no filter scenario that returns all cm handles

Cm Handle Private Properties

Proposed Property Name	Data Type	Data Size	Property Value	Description

Proposed Property Name	Data Type	Data Size	Property Value	Description
segmentId	Integer	4 bytes	2	Unique per EMS. Subsystem range: 1-10.
networkSegment	String	50-100 bytes	Region=NorthAmerica	thousands of different. Unique per CM handle
deviceIdentifier	String	UTF-8, 32 bytes	Element=RadioBaseStation_5G_999	Unique per CM handle. Represents a specific network element.
hardwareVersion	String	UTF-8, 10-20 bytes	HW-AlphaX9	Sharing uniqines with module set tag.
softwareVersion	String	UTF-8, 10-20 bytes	Firmware_77B	Sharing uniqines with module set tag.
syncStatus	Enum (String / Integer)	5 or 8 bytes	ACTIVE
nodeCategory	String	10 bytes	VirtualNode

Storage Size Breakdown for Private and Public Properties

Property	Private Property	UTF-8 String Size (bytes)	Serialized Object Size (bytes)	In Database (bytes)

Property	Private Property	UTF-8 String Size (bytes)	Serialized Object Size (bytes)	In Database (bytes)
Pre-change	`{"neType":"RadioNode"}`	22	29	172
Post-change	`{"segmentId":10000,"networkSegment":"Region=NorthAmerica,Segment=10","deviceIdentifier":"Element=RadioBaseStation_5G_11000","hardwareVersion":"HW-tagA","softwareVersion":"Firmware_tagA","syncStatus":"ACTIVE","nodeCategory":"VirtualNode"}`	237	244	1,344
	% change	+974	+741	+361
Property	Public Property	UTF-8 String Size (bytes)	Serialized Object Size (bytes)	In Database (bytes)
Pre-change	`{"Color":"yellow","Size":"small","Shape":"cube"}`	48	55	484
Post-change	`{"systemId":10000,"systemName":"ncmp"}`	38	45	339
	% change	-21	-18	-30

The given SQL query calculates the approximate size (in bytes) of each row in the fragment table in PostgreSQL.

SELECT
    COALESCE(pg_column_size(id), 0) +
    COALESCE(pg_column_size(xpath), 0) +
    COALESCE(pg_column_size(attributes), 0) +
    COALESCE(pg_column_size(anchor_id), 0) +
    COALESCE(pg_column_size(parent_id), 0) +
    COALESCE(pg_column_size(schema_node_id), 0) AS row_size
FROM fragment;

This Java program measures the size of a string using two different approaches:

UTF-8 Encoding Size (stringSize)
- Uses jsonData.getBytes("UTF-8").length to compute the number of bytes required to store the string in UTF-8 encoding.
Serialized Object Size (objectSize)
- Uses Java serialization (ObjectOutputStream) to write the object into a byte stream and then measures its byte size.

import java.io.*;
public class SerializedSizeCalculator {
    public static int getObjectSize(Object obj) throws IOException {
        ByteArrayOutputStream byteOut = new ByteArrayOutputStream();
        ObjectOutputStream out = new ObjectOutputStream(byteOut);
        out.writeObject(obj);
        out.close();
        return byteOut.toByteArray().length;
    }

    public static void main(String[] args) throws IOException {
        String jsonData = "{\"systemId\":10000,\"systemName\":\"ncmp\"}";

        int stringSize = jsonData.getBytes("UTF-8").length;
        int objectSize = getObjectSize(jsonData);

        System.out.println("UTF-8 String Size: " + stringSize + " bytes");
        System.out.println("Serialized Object Size: " + objectSize + " bytes");
    }
}

Performance Analysis of FS Requirements vs. Master branch

#	Test Name	Unit	KPI (FS Requirement)	Master	% Gain / Loss	Status	Properties Needed

#	Test Name	Unit	KPI (FS Requirement)	Master	% Gain / Loss	Status	Properties Needed
1	Registration of CM-handles	CM-handles/second	22	93	323		Yes
2	De-registration of CM-handles	CM-handles/second	22	152	591		Yes
3a	CM-handle ID search with No filter	milliseconds	2,000	1,192	40		No
3b	CM-handle ID search with Module filter	milliseconds	2,000	297	85		No
3c	CM-handle ID search with Property filter	milliseconds	2,000	2,545	27	📉 Slower	No
3d	CM-handle ID search with Cps Path filter	milliseconds	2,000	2,708	35	📉 Slower	No
3e	CM-handle ID search with Trust Level filter	milliseconds	2,000	2,834	42	📉 Slower	No
4a	CM-handle search with No filter	milliseconds	15,000	15,900	6	📉 Slower	Yes
4b	CM-handle search with Module filter	milliseconds	15,000	17,289	15	📉 Slower	Yes
4c	CM-handle search with Property filter	milliseconds	15,000	20,302	35	📉 Slower	Yes
4d	CM-handle search with Cps Path filter	milliseconds	15,000	22,022	47	📉 Much Slower	Yes
4e	CM-handle search with Trust Level filter	milliseconds	15,000	21,207	41	📉 Much Slower	Yes
5a	NCMP overhead for Synchronous single CM-handle pass-through read	milliseconds	40	67	68	📉 Increased Overhead	Not all?!
5b	NCMP overhead for Synchronous single CM-handle pass-through read with alternate id	milliseconds	40	108	170	📉 Increased Overhead	Not all?!
6a	NCMP overhead for Synchronous single CM-handle pass-through write	milliseconds	40	81	103	📉 Increased Overhead	Not all?!
6b	NCMP overhead for Synchronous single CM-handle pass-through write with alternate id	milliseconds	40	111	178	📉 Increased Overhead	Not all?!
7	Legacy batch read operation	events/second	150	99	34	📉 Slower	Not all?!

Key Observations:

Observation:

The registration and de-registration of CM-handles significantly exceed expectations.
The CM-handle ID search with No filter and Module filter is much faster than expected..
Higher response time indicates a bottleneck in filtering by Property, Cps Path, and Trust Level.
Search operations are slower across multiple filters.
NCMP operations are much slower than expected.
Batch read operation processed fewer events per second than required.

Overall Summary

✅ CM-handle registration & de-registration are highly optimized (323% and 591% improvement).
✅ CM-handle ID search without filters is very fast (up to 85% improvement).
❌ CM-handle ID searches with complex filters are slower than expected (27% to 42% slower).
❌ CM-handle searches are significantly slower across all filters (6% to 47% slower).
❌ NCMP overhead increased drastically (67% to 178% increase).
❌ Legacy batch read throughput dropped by 34%.

Action Plan:

Possible improvment before capturing final figures:

140398: Optimize registration parameters | https://gerrit.onap.org/r/c/cps/+/140398 Merged
140391: Added query limits to NCMP service methods | https://gerrit.onap.org/r/c/cps/+/140391 Open

Issues & Decisions

	Issue	Notes	Decision

	Issue	Notes	Decision
1	High Availability	Not beign able to run load with single instance
2	Is it for worst case scenario?	All these properties being used for client testing ?