https://lf-onap.atlassian.net/browse/CPS-391

Guiding Principles

NCMP REST Interface will follow/be inspired by RESTConf interface for easy acceptance of and transition to this interface
Will follow ONAP's RESTful API Design Specification
The interface will include the concept of data-stores inspired by Network Management Datastore Architecture (NMDA) and as used in RESTConf
The application should be able to easily switch between 'pass-through' and other datastores (also identical rest endpoint and responses)

References

Follow principles/patterns of RESTCONF RFC-8040 https://datatracker.ietf.org/doc/html/rfc8040
Follow principles/patterns of yang-patch RFC-8072 https://datatracker.ietf.org/doc/html/rfc8040
Follow principles/patterns of RESTCONF NMDA RFC-8527 https://datatracker.ietf.org/doc/html/rfc8527

Issues & Decisions

	Issues	Notes	Decisions

	Issues	Notes	Decisions
1	1	KPI for De-registration of 100 CM-handles	This was mentioned. Was this ever agreed, is this a valid use case that needs to be covered together with the Registration ?	Not priority for now, but acceptable if we match the registration req. #2 for de-reg May 21, 2024 @kieran mccarthy @Kolawole Adebisi-Adeolokun
2	2	DMI delay	Could we get some feedback on DMI-delays for other use cases as not mentioned in FS document	Awaiting for ETH feedback May 21, 2024 AP On @Kolawole Adebisi-Adeolokun and @Csaba Kocsis Jul 2, 2024 Provided
3	3	Number of instances In some cases, ETH have used 2 instances, can we verify the number of instances for each use case. Some of the req were defined per instance and resources used : Identify which of these ?	Agreed to; CPS use 1 instance currently, but should focus on aligning performance with 2 instances for all use case	Jul 9, 2024 @Kolawole Adebisi-Adeolokun @Csaba Kocsis
4	4	Input Load Distribution the CM-handle search and ID search	Currently has 5 parallel request between them distributed at 2.5 each. This fractional distribution isn't feasible for parallel processing; the load should be allocated as whole numbers. Load needs to be distributed at. Would it be acceptable to adjust this distribution to either 2 or 3 parallel requests each (and vice versa ) without any negative repercussions? Agreed to do 6 parallel request combined total and divide the load to 3 parallel request each	Jul 9, 2024 @Kolawole Adebisi-Adeolokun @Csaba Kocsis
5	Regarding CM-handle search and ID search	FS only identified Module performance, are there any testing done towards a combined search of properties and modules in a single query Confirmed no other testing was previously done on this..... CPS have the capabilities to do mixed testing. ETH tbc on if they want to consider this ( @Csaba Kocsis )

Requirements

Please note this section was added long after the implementation and focuses on characteristic and enhancements after this study only.

Characteristics

	Operation	Concurrent requests/parallel	DMI Delay	Response size	Performance Requirement (Blue Stone tablet KPI)	Notes	Sign-Off

	Operation	Concurrent requests/parallel	DMI Delay	Response size	Performance Requirement (Blue Stone tablet KPI)	Notes	Sign-Off
1	Registration of 50,000 CM-handles (in batches of 2,000)	1 (requests are sequential)	100 ms to get module references 1,000 ms to get module resources	N/A	11 CM-Handles/second as per Stone Tablet E2E Which include module conversion warm-up NCMP Budget: 22 Cm Handles/second	Batch Size: 2,000 (per request) Not using Module Set Tags Time measured start of first rest-call until all cm handle states READY 1,000 unique module references. Five different types of Nodes. So 5 requests for Module Resources. Avg 200 modules each. The average size of yang resources in bytes 31.437 (32KiB)	May 21, 2024 @Csaba Kocsis @Toine Siebelink
2	De-registration of 2,000 CM-handles	1 (requests are sequential)	No Module delays	N/A	11 NEs/second NCMP should target 22 NEs/Second	De-registration is currently not mentioned in Stone Tablet KPI or FS, however we have agreed to match the performance of registration for now as de-reg is also not a priority at this point in time	May 21, 2024 @Csaba Kocsis @Toine Siebelink @Kolawole Adebisi-Adeolokun
3	CM-handle ID search with Module filter	5 parallel request	N/A	50,000 CM Handles i.e. 100*50.000 = 5MB	2 seconds/operation	FS stated 5 parallel request for each of ID search and search, meaning a combined total of 10 parallel search requests. These test now include returning AlternateIds	Jul 9, 2024 @Kolawole Adebisi-Adeolokun @Csaba Kocsis
4	CM-handle search with Module filter	5 parallel request	N/A	50,000 CM Handles i.e. 500*50.000 = 25MB	15 seconds/operation	FS stated 5 parallel request for each of ID search and search, meaning a combined total of 10 parallel search requests. These test now include returning AlternateIds	Jul 9, 2024 @Kolawole Adebisi-Adeolokun @Csaba Kocsis
5	Synchronous single CM-handle pass-through read	4 (Parallel operations)	300 ms	5 KB	25 request/second	Read are done in parallel with Write and searches. Note: CPS will test pass-through read using alternateId.
6	Synchronous single CM-handle pass-through write (CUD)	4 (Parallel operations)	670 ms	5 KB	12.5 request/second	No response is expected Note: CPS will test pass-through write using alternateId.
7	Batch/Bulk Read	60 read request with 200 cmHandles each at 1 req/second.			150 cmhandles/second	60 request in 60 seconds overall budget for last response is 80seconds for 60 request to be processed The batch data operation is asynchronous. NCMP returns the response on the ncmp-async-m2m kafka topic.
8	8	CM change Notification Event				1,640 avc event per/sec - Base Rate 2,660 avc event per/sec - Peak Rate 12,640 avc event per/sec - Storm rate	Peak Rate happens 96 times a day for 5 minutes each time. Storm Rate happens 6 times a day for 6 minutes each time.

Notes

This is for mixed TCs
Single KPIs will be monitored in NCMP owned pipeline with our performance every day(2 hrs interval) - Performance
Test cases 3 through 7 are to run in parallel.

Synchronous single cm-handle pass-through (read) requests

Parameter	Expectation	Notes	Sign-Off

Parameter	Expectation	Notes	Sign-Off
Average Response Size	5KB	Shall not exceed 5KB	Dec 6, 2023 @Kolawole Adebisi-Adeolokun
Concurrent request	12 clients requests toward 1 NCMP simultaneously DMI also support 12 simultaneous requests	40ms of overhead on top of DMI latency for each requests, at most for NCMP request. This shall remain within 40ms for 12 parallel requests. Given the DMI delay below; this means up to 240 request per second	Dec 6, 2023 @Kolawole Adebisi-Adeolokun
DMI Delay	10ms	This is not in control of CPS. So for performance testing our stub should simulate a 10ms delay Assume DMI is 1.25 seconds average DMI response time for high latency, low latency =10 ms, this should also work for DMI Plugin. I.e 40ms ontop of the DMI. 1.25seconds+40ms= 1.29seconds	Dec 6, 2023@Kolawole Adebisi-Adeolokun
Test Environment	CPS and NCMP `requests:` `cpu: 2000m` `memory: 2Gi` `limits:` `memory: 3Gi` `cpu: 3000m` 2. Postgres `requests:` `cpu: 4000m` `memory: 1Gi` `limits:` `memory: 3Gi` `cpu: 6000m`		Dec 6, 2023 @Kolawole Adebisi-Adeolokun
Security	Disable Basic Authentication in Springboot	If configurable from application yaml, then it’s acceptable.	Dec 6, 2023 @Kolawole Adebisi-Adeolokun

Open Issues & Decisions

	Description	Notes	Decision

	Description	Notes	Decision
1	Priority of async calls		In Istanbul, async calls are required only in pass-through cases. NCMP does not have to handle these requests
2	Will we use the data node wrapper on GET rest operations?	Currently, we wrap the response of GET operations using the data node wrapper.	we should mainly support yang-data/json controlled by "accept-header"
3	In the URI will we distinguish between data and operations (RFC calls) as part of the path?	e.g. http://localhost:8080/ncmp/v1/data http://localhost:8080/ncmp/v1/operation	This only applies to pass-through yes, we will distinguish between data and operation
4	Which query parameters will NCMP support?		Parent data resource identifier can handle any path using the same query parameter cpsPath RESTConf Path (pass-through) Proprietary Path (pass-through)
5	Yml should include return types and examples of the payload		Legacy and new API documentation needs to include output examples. Task created, see https://lf-onap.atlassian.net/browse/CPS-401
6	camel case or dash in URI		We will use a dash for param names e.g. cmHandle (although it has since been agreed we use 'ch' in this particular case) See no.3 https://restfulapi.net/resource-naming/
7	Insert /resourcePath in front of the resource path to prevent ambiguous paths	<OP>/ncmp/<v{vNumber}>/ch/<cmHandle>/<data\|operations\|{ncmp-operation}>/ds/{datastore}/[rp:]{resourcePath}?{query}	Optionally insert the resource path ('rp:') if it clashes with the current
8	Granularity of update scenarios (and priority)	Add child and its descendants (supported in cps core) Add all list elements (supported in cps core) Replace child and its descendants (supported in cps core) Replace all list elements (pending in cps core) Update single leaf (new) Add list entry (new)	Priority is pass-through only so it depends on the RETSConf protocol that is supported. In Jakarta or if required by other projects more fine-grained 'operation' datastore update options can be implemented
9	Fallback option for datastore in release I		No, explicit datastore options will be used in Istanbul
10	fields is a rest conf option, investigate is it fully supported by onap		Supported in pass-through for ONAP DMI plugin but depending on the support by the actual target. The intention is to increase support 'fields' in future requirements following RFC-8040 for operational datastore etc.
11	Agree on URI syntax	Proposed syntax by CPS team <OP>/ncmp/<v{vNumber}>/ch/<cmHandle>/<data\|operations\|{ncmp-operation}>/ds/{datastore}/[rp:]{resourcePath}?{query}	review completed and proposed URI agreed
12	Will we combine query capabilities with update capabilities?		We have decided not to combine query capabilities with update capabilities
13	Description of header limitations		HTTP Header Limitations Some servers put size limitations on HTTP headers, making them unsuitable for storing cmHandle information. LIMITATION NOTE: server implementations put size limits on the headers meaning header contents should be designed carefully : Apache - 8K Nginx - 4K-8K IIS - 8K-16K Tomcat - 8K – 48K
14	Will NCMP support paths for pass-through:running		The plugin will not do transformation or validation of paths in the case of pass-through:running
15	Specification of path per cm handle		DMI Plugin can take cps paths or restconf paths and it needs to specify that per cm handle when cm handle is created
16	What is the default path for NCMP		In NCMP default will always be cps path and depending on the adapter we can change it as needed per cm handle
17	Fields parameter for ncmp/operational?		The fields parameter is ignored in ncmp/operational (in Istanbul release)
18	Is specifying the datastore mandatory?		Datastore is mandatory in Istanbul release
19	Register a DMI plugin with NCMP		DMI plugin is a part of cm handle registration. The rest endpoint on NCMP can be multiple calls
20	Retrieve list of modules (names) for a cmHandle		Retrieve a list of module names for cm handle - this will be used by ncmp to get the models. - assuming ncmp model discovery is complete and it is stored in cps core, this will come from cached information
21	Where will sync be implemented?		Implement sync in the dmi plugin and then have ncmp be able to pass on the request. This is not a bulk operation
22

CPS-391Spike: Define and Agree NCMP REST Interface