CM Data Notification Subscription LCM (incl. merge)

References

CPS-1616 - Getting issue details... STATUS
CPS-1615 - Getting issue details... STATUS
CPS-1812 - Getting issue details... STATUS
CPS-1865 - Getting issue details... STATUS
Impl. Proposal for Merging of Subscriptions ( Positive Scenarios )

Requirements

Functional

	Interface	Requirement	Additional Information	Signoff
1	CPS-E-08e	NCMP will forward 'CM Notifications Subscription' LCM Create events to the appropriate DMI Plugins responsible for the CM Handles (ids) defined in the LCM Events. The message sent to DMI should include the private properties for each CM-Handle		24 Jul 2023
2	CPS-E-08e	NCMP will forward 'CM Notifications Subscription' LCM Delete events to the appropriate DMI Plugins responsible for the CM Handles is defined in the LCM Events. The message sent to DMI should include the private properties for each CM-Handle. A delete should only be forwarded if there is no other subscription for cmhandle datastore path.		13 Sep 2023
3	CPS-E-08e	NCMP responds using Kafka message(s) with the result of the LCM operation These messages will use agreed status fields: StatusCode (to be included in documentation) StatusMessage resulterrorInfo (NEW to be fleshed out)	Action: Define error details later Now in separate Epic, see CPS-1865 - Getting issue details... STATUS
4	CPS-E-08e	All events should use Cloud Events format	https://cloudevents.io/	24 Jul 2023
5	CPS-E-08e	Messages should be processed in the order they are created (received)	It will depend on DMIs responding within 30s. Subscription-id should be used as Kafka 'key' to ensure this.	05 Sep 2023
6	CPS-E-08.e	NCMP is to merge CM Notification Subscriptions create request and forward those to DMI plugin. (Interface to be added).	Prevent unnecessary subscription updates to nodes already involved in a subscription to the same path and datastore. For possible combinations, see table below	30 Aug 2023
7	CPS-E-08.e	Last lights out: upon subscription Delete request only when there is no more subscription for a cm-handle & xpath & datastore combination a subscription-delete request will be sent to the relevant DMI(s).		30 Aug 2023
8	CPS-E-08.e	A single client subscription request should result into a maximum of one request per DMI. Of course there can be several messages if more than 1 DMI is involved.		30 Aug 2023
9	CPS-E-08.e	Amalgamate response should include rejected/accepted/pending DMI responses received within 30 seconds. A client shall be notified of available DMI subscription information after 30 seconds. Subsequent DMI subscription updates shall be notified to clients as they become available.	Same schema for all notifications. Subsequent notifications contains the state of all cmhandles involved in the subscription.	30 Aug 2023
10	CPS-NCMP-I-01	CM Handle deletion should NOT update subscription details.	Do NOT delete dmi-subscription entry until owning subscription is deleted, see issue #4 below	30 Aug 2023
11	CPS-E-08.e	~~Backward compatible with 'basic' created/delete operations..~~		30 Aug 2023

Error Handling

	Error Scenario	Expected behavior	Sign-off
1	Non responding DMIs (configurable timeout)	An error message listing the 'pending' CM Handle IDs. statusMessage/statusCode to be agreed.	05 Sep 2023
2	Non Existing or non-valid CM Handle Ids	An error message listing the 'rejected' CM Handle IDs.	05 Sep 2023
3	DMI Plugin without CM Event subscription support	An error message listing the 'rejected' CM Handle IDs	05 Sep 2023
4	Late Response (after timeout defined in #1) from DMI	A list of outcomes (complete status on how many cmhandles are pending/rejected/accepted) from client subscription is returned in response. statusMessage/statusCode to be agreed.	13 Sep 2023
5	Create for existing subscription id	The whole subscription will be rejected. statusMessage/statusCode to be agreed.	13 Sep 2023
6	Delete for non-existing subscription id	The whole delete subscription will be rejected. statusMessage/statusCode to be agreed.	13 Sep 2023
7	Error upon error	An error scenario on a second subscription for the same cm-handle/xpath as a previous subscription which did not complete successfully (yet) to be discussed, see Error Upon Error Combinations

Characteristics

	Parameter	Expectation	Notes	Sign-off
1	Frequency of LCM Subscription Events	No high volume of such events is expected For testing purpose 1 LCM event per minute would suffice.	kieran mccarthy confirmed	13 Sep 2023
2	Number of Subscriptions	Max: 200	kieran mccarthy confirmed	05 Sep 2023
3	CM Handles per Subscription	Avg: 200 Max: 500 (document only)	kieran mccarthy to confirm Note: consider limit and future 'simple' ALL (wildcard) option when above that limit	05 Sep 2023
4	Xpaths per Cm Handle	Avg: 10	kieran mccarthy	05 Sep 2023
5	Status update processing time	(Order of 30) seconds (NCMP side)	kieran mccarthy	05 Sep 2023

Out-of-scope

Subscription Update scenarios for an ongoing subscription.
CM Notification Forwarding Check: When forwarding CM Notification NCMP will not check the content to see if the is a valid active subscription. It is assumed that the DMI Plugin as acted on the 'delete subscription' request (that request is NCMPs responsibility). And of course there wil be timing issues it also possible a CM Notification was send just after the subscription-delete was send (from client) but before the whole change had acted upon that.
Retry: NCMP will only report when actions are pending or rejected. NCMP will not implement a retry mechanism
Wildcards: Wildcards or similar functionality where one string represent 0 or more xpaths is not covered as part of this requirement but it should be kept in mind as a future possibility
Dynamic Topic: Topic for CM Data Notifications back to client will be hardcode for now.
Study should consider compatibility with an 'ALL' cm handles options
The proposed Schema Change & HTTP Rest (reate DME subscription interfaces as a new subscriotion epic) has led to new epic which will be implemented in → -[CPS-2233] CM Subscription with DME interface - ONAP Jira

Assumptions

	Assumption	Notes
1	Targets is a collection of xpaths	can be implemented as a json array of strings
2	for the DMI plugin cm-handle-id, datastore and xpath are the key to a unique subscription	the client-id and subscription name are not relevant for the DMI (should not be included in the request) as this would break the concept of merging!

Issues & Decisions

	Issue	Notes	Decision
1	Is it intended that CM Notification subscription request cover (all) descendants of the given xpath too?!	e.g.. if a child is removed and there is a subscription for the parent node, will a notification be send (grandchild, child leaf updates etc.) I hope NOT! Consider: Risk of client effectively subscribing to ALL data in a cm handle by specifying top level datanode(s) Complexity (i.e. cost of) of merge operation. It might even required NCMP to check relevant dat model Future use of wildcards, could be a viable alternative for including descendants	24 Aug 2023 Descendent don't need to be taken into account
2	Could xpath point to an element that does not exist (yet)	if not how, how can I client be informed about a create event?	24 Aug 2023 xpath can point to an element that does not exist (yet) CPS/NCMP Does not need to check Xpath
3	Should NCMP support re-homing, moving of a CM Handle from one DMI to another?	assume only trough delete & create	24 Aug 2023 only trough delete & create
4	CM Handle Delete: Should DMI or Clients be sent a subscription update	do NOT delete dmi-subscription entry until owning subscription is deleted (just ignore upon future delete if cm handle is gone altogether) Note. LCM is already broadcast (today)	24 Aug 2023 No, subscription update Need to upgrade to CloudEevent format Add subscription id (name+client id) to header (so client can filter)
5	Validation of xpath	options order of implement and also performance cost! none ~~xpath-parser~~ ~~model check~~ ~~instance check~~	24 Aug 2023 None
6	can DMI plugin 'reject' a subscription create (for a given cm-handle-xpath combination)	As NCMP might not validate as per issue#5 the DMI=plugin or component further down might have to reject an invalid xpath...	Priyank Maheshwari yes currently DMI can use response to say which cm handles are not accepted i.e. rejected' (but not 'pending') 24 Aug 2023
7	implementation question: should 'rejected' DMI-subscriptions be stored	not needed as whole subscription should be rejected	24 Aug 2023 not needed as whole subscription should be rejected
8	Dimensioning of DB depends on #cm handles, #subscriptions and #xpaths per subscription, this could be too big for fast processing of updates!	Need to agree maximum and possibly realistic average/total number of entries based on the characteristics above The team is blocked until this becomes clear as it will affect the way the data needs to be modelled exactly.	13 Sep 2023 - kieran mccarthy- Dimensioning has been agreed. See Characteristics
9	Maximum (error) message size	theoretically all cm handles and all xpaths combinations could be rejected or pending leading to a very large error message!	24 Aug 2023 ?? what was agreed ? Kolawole Adebisi-Adeolokun to update what was agree
10	can each CM-Handle have different set of xpath(s) per subscription	the current 'basic' solution only supports a common set of datastore/xpaths (filter)	24 Aug 2023 Yes, schema modification allows this
11	can the same cm handle/xpath have different subscriptions with different datastores, does that affect the cm data notifications send (which datastore applies)	can register on any (passthrough) datastore running operational compulsory, validation needed part of KEY (unique entry) so cmHandle+datastore+xpath is the key	28 Aug 2023 Although this wiki-page will not be updated where cmHandle+xpath is mentioned as unique entry etc. this should now include a third field: datastore as well.
12	Will migration from 'basic' be supported	Preferred to ask customers to create new subscriptions	24 Aug 2023 No, basic sub. will not be delivered as a separate feature
13	list v list instances filtering	/p/c1 /p/c1[@key=23] Are these different paths or not (from ncmp point of view)	24 Aug 2023 List entries are unique. *List instances might not always be supported, depending DMI Plugin impl.
14	confirm subscription id (currently subscription name + client id)	previously was clientid and subcrioptionid and we were relying on concatenation of both to be unique	05 Sep 2023 Now only the event would contain subcrioptionid
15	what subscription details should be send when there is a change (in the union)	just delta ~~just union~~ ~~union and delta (delta flagged)~~	24 Aug 2023 Just Delta
16	one DMI rejects whole (see decision #6) subscription (affected cmhandles) but other DMI accepts the same subscription, is this possible how to handle		28 Aug 2023 Yes, this has been analysed
17	What all datastores are supported ?	ncmp-datastore:passthrough-running or ncmp-datastore:passthrough-operational ?? or both?	13 Sep 2023 - kieran mccarthy Both will be supported
18	When NCMP sends subscription request to DMI and then DMI responds back to NCMP , do we need to have a correlationId to track/map the request and response?	Value: subscriptionID:DMIPluginName (Check if subscriptionId or DMIPlugin name doesn't contain a colon: ) - The seperator is to be chosen by the development team, will try with colon first.	18 Sep 2023 subscriptionId to be used.
19	Which status codes to be used when the DMI Plugins respond back to NCMP ? We already have codes defined in NCMP. Do we reuse the same status codes when DMI Plugin responds back to NCMP with the subscription status ?	Yes, reuse.	18 Sep 2023 kieran mccarthy - Reuse the same codes defined in NCMP.
20	Impl. Proposal for Merging of Subscription		02 Oct 2023 Schemas Agreed for the below events ( Source to Destination ) for the positive scenarios. DME to NCMP NCMP to DMI-Plugin DMI-Plugin to NCMP NCMP to DME
21	Create DME subscription interfaces Schema Change	Schema Change, Events driven change now HTTP Rest. Study status is at 2/3 of review. CPS need to study new changes to subscription once study is approved.	New Subscription changes forecasted for 25Q2. Deliver current event based subscription discuss with wider stakeholders on 29 May 2025 Create DME subscription interfaces as a new subscription epic -[CPS-2233] CM Subscription with DME interface - ONAP Jira 21 May 2024 Kolawole Adebisi-Adeolokun to create backlog Jira

Solution Proposal

State handling for initial (basic)subscription create/delete use cases (not using concept of 'merging')

Click here to expand...

Scenario 1
1) Create Sub from Client: CH-1, CH-2
2) All handled by DM1
3) NCMP to DMI request Create sub: CH-1, CH-2
DMI responds within 30 seconds: Status OK

NCMP Send message to client: Statsu OK (no list with Rejected/Pending)

Scenario 2
1) Create Sub from Client: CH-1, CH-2
2) All handled by DM1
3) NCMP to DMI request Create sub: CH-1, CH-2
DMI does NOT respond with 30 seconds

NCMP Send message to client: Pending [CH-1, CH-2]

Scenario 3
1) Create Sub from Client: CH-1, CH-2
2) All handled by DM1
3) NCMP to DMI request Create sub: CH-1, CH-2
DMI responds within 30 seconds: Rejected [CH-2]

NCMP Send message to client: Rejected [CH-2]

Scenario 4:
4 different DMIs each handling 2 Cm Handles, create sub for all CHs

Scenario 4-part a
DMI1 CH-1, CH-2 All accepted within 30 secs
DMI2 CH-3, CH-4 Mixed result within 30 seconds e.g. Rejected [CH-4]
DMI3 CH-5, CH-6 No result after 30 seconds
DMI4 CH-7, CH-8 No result after 30 seconds

NCMP -> Client : Rejected [CH-4], Pending [ CH-5, CH-6, CH-7, CH-8 ]

4-part b
DMI3 PLugin responds after 40 seconds; status OK
NCMP -> Client : Rejected [CH-4], Pending [ CH-7, CH-8 ]

4-part c
DMI4 PLugin responds after 50 seconds; Rejected [CH-7]
NCMP -> Client : Rejected [CH-4, CH-7]

Note. The above algorithm depends ons storing (in DB using yang modelled data) a cm-handle status for each cm-handle for each subscription!

Merge → Split

NCMP treats a Client Subscription as one or more (small) DMI Subscriptions, each of which wil have there own state.
Each DMI Subscription is related to 1 or more client subscriptions.
If there is no more related client subscription the DMI Subscription can be deleted (once accepted by DMI Plugin)!

Data Model (not a great diagram...)

Create Combinations

Below examples demonstrate what should happen when two separate subscriptions operations are performed:

an operation on subscription 'A' for client id '10
an operation on subscription 'B' for client id '52'

#

Existing Subscription A-10

Client Create Subscription B-52 Request

DMI Sub Create Request

Data Model Before (2nd) Create

Data Model After Create

1

CH-1, [ /p/c1, /p/c2 ]

CH-1, [ /p/c1 ]

None

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-1	/p/c2	[ A-10 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10, B-52 ]
B	52	CH-1	/p/c2	[ A-10 ]

2

CH-1, [ /p/c1, p/c2 ]

CH-1, [ /p/c2, /p/c3 ]

CH-1, [ /p/c2, /p/c3 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-1	/p/c2	[ A-10 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-1	/p/c2	[ A-10, B-52 ]
		CH-1	/p/c3	[ B-52 ]

3

CH-1, [ /p/c1]
CH-2, [ /p/c1]

CH-2, [ /p/c1]
CH-3, [ /p/c1]

CH-3, [ /p/c1]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-2	/p/c1	[ A-10 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-2	/p/c1	[ A-10, B-52 ]
		CH-3	/p/c1	[ B-52 ]

4

CH-1, [ /p/c1]

CH-1, [ /p/c1/gc1 ]

*as per decision 1 the xpath to a parent does NOT include any children...

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-1	/p/c1/gc1	[ B-52 ]

Create 'Merge' Diagram

Below diagram shows and example for two subscriptions with party overlapping CM Handles and XPaths.

* Note 1: given the possible combinations the message to DMI needs to be able to specify different xpaths per cm-handle. So a more complex structure is needed for this even ie. an array of CM Handles objects each having their own list of (target) xpaths!
Note 2: as per decision #11'datastore' should be included in all messages as well
Note 3: the correlation-id for message to and from to DMI will be composed from the subscription-id and the dmi-name

Delete Combinations

#

Existing Subscription A-10

Existing Subscription B-52

Client Delete A-10 Request

DMI Sub update/delete request

Data Model^* Before Before

Data Model^* After Delete

1

CH-1, [ /p/c1, p/c2 ]

CH-1, [ /p/c1 ]

CH-1, [ p/c2 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10, B-52 ]
B	52	CH-1	/p/c2	[ A-10 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-1	/p/c2	[ A-10 ]

2

CH-1, [ /p/c1]
CH-2, [ /p/c1]

CH-2, [ /p/c1]
CH-3, [ /p/c1]

CH-1, [ /p/c1]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-2	/p/c1	[ A-10, B-52]
		CH-3	/p/c1	[ B-52 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
B	52	CH-2	/p/c1	[ B-52 ]
		CH-3	/p/c1	[ B-52 ]

3

CH-1, [ /p/c1]

CH-1, [ /p/c1]
CH-2, [ /p/c1]

None

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-1	/p/c2	[ A-10, B-52 ]

Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
B	52	CH-2	/p/c1	[ B-52 ]

* Note: as per decision #11'datastore' should be included in the 'DMI susbcription' data model too now.

Error-Upon-Error Combinations

( To be captured as part of CPS-1865 - Getting issue details... STATUS )

	Previous Interaction	Current Interaction	Expectation	Notes	Sign-Off
1	any operation on non-existing cm-handle	operation for same non-existing cm-handle	listed in 'rejected' immediately	behavior as normal	18 Sep 2023
2	create operation rejected by DMI	create for same cm-handle/xpath	Submit create request again.		18 Sep 2023
3	create pending	create for same cm-handle/xpath	Set it to pending without submitting a new request. When we get response for previous interaction it is applied for the current interaction as well.		18 Sep 2023
4	create pending	delete for same cm-handle/xpath	Set an error 'Conflict/Busy'	up to client to retry the operation.	18 Sep 2023
5	delete pending	delete for same cm-handle/xpath	Set it to pending without submitting a new request. When we get response for previous interaction it is applied for the current interaction as well.		18 Sep 2023
6	delete pending	create for same cm-handle/xpath	Set an error 'Conflict/Busy'	up to client to retry the operation.	18 Sep 2023

Client-Schema Update

Based on Issue#10 , the incoming schema from DME to NCMP should

A single subscription contains multiple predicates
Predicates should be an array of Predicates ( currently a single instance )
- Each predicate contains an array of targets ( cmhandles) - as is.
- Each predicate contains a single datastore (mandatory)
- datastore-xpath-filter should become an array of xpath-filters (instead of pipe separated single string)

Performance considerations

Below options (partly) explored dring CPS teem meeting 02 Aug 2023

DMI record could reach in millions of instances
In memory solution (ie hazelcast) can handle 10s - 100s of Megabytes (add some sample calc)
- still need persistence if we don't have enough instances
use many small messages reduce need for amalgamation
amalgamated string-based solution....

Kafka Topics


CM_SUBSCRIPTION_NCMP_IN_TOPIC
CM_SUBSCRIPTION_DMI_IN_TOPIC
CM_SUBSCRIPTION_DMI_OUT_TOPIC
CM_SUBSCRIPTION_NCMP_OUT_TOPIC
DMI_CM_EVENTS_TOPIC
NCMP_CM_EVENTS_TOPIC

Operation Specific Impl. Proposals

Create and Merge

Impl. Proposal for Merging of Subscriptions ( Positive Scenarios )

Delete

Impl. Proposal CM Event Subscription LCM: Delete ( Positive Scenarios ) // to be deprecated

#	Topic Name	Environment Variable	Source	Destination	Notes
1	subscription	CM_SUBSCRIPTION_NCMP_IN_TOPIC	DME	DME/Client-Apps
2	ncmp-dmi-cm-avc-subscription	CM_SUBSCRIPTION_DMI_IN_TOPIC	NCMP	DMI Plugin
3	dmi-ncmp-cm-avc-subscription	CM_SUBSCRIPTION_DMI_OUT_TOPIC	DMI Plugin	NCMP
4	subscription-response	CM_SUBSCRIPTION_NCMP_OUT_TOPIC	NCMP	DME/Client-Apps
5	dmi-cm-events	DMI_CM_EVENTS_TOPIC	DMI Plugin	NCMP	actual data notifications as per the subscriptions
6	cm-events	NCMP_CM_EVENTS_TOPIC	NCMP	DME/Client-Apps	NCMP to forward the notifications to the client-apps.