CM Data Notification Subscription LCM (incl. merge)

CPS-E-09e

NCMP will forward 'CM Notifications Subscription' LCM Create events to the appropriate DMI Plugins responsible for the CM Handles (ids) defined in the LCM Events. 
The message sent to DMI should include the private properties for each CM-Handle

 Jul 24, 2023

CPS-E-09e

NCMP will forward 'CM Notifications Subscription' LCM Delete events to the appropriate DMI Plugins responsible for the CM Handles is defined in the LCM Events. 
The message sent to DMI should include the private properties for each CM-Handle.

A delete should only be forwarded if there is no other subscription for cmhandle datastore path.

Sep 13, 2023 

CPS-E-09e

NCMP responds using Kafka message(s) with the result of the LCM operation
These messages will use agreed status fields:

• StatusCode  (to be included in documentation)

• StatusMessage

• resulterrorInfo  (NEW to be fleshed out)

Action: Define error details later
Now in separate Epic, see https://lf-onap.atlassian.net/browse/CPS-1865

CPS-E-09e

All events should use Cloud Events format

https://cloudevents.io/

Jul 24, 2023

CPS-E-09e

Messages should be processed in the order they are created (received)

It will depend on DMIs responding within 30s.
Subscription-id should be used as Kafka 'key' to ensure this.

Sep 5, 2023

CPS-E-09.e

NCMP is to merge CM Notification Subscriptions create request and forward those to DMI plugin. (Interface to be added).

• Prevent unnecessary subscription updates to nodes already involved in a subscription to the same path and datastore.

• For possible combinations, see table below

Aug 30, 2023

CPS-E-09.e

Last lights out: upon subscription Delete request only when there is no more subscription for a cm-handle & xpath & datastore combination a subscription-delete request will be sent to the relevant DMI(s).

Aug 30, 2023

CPS-E-09.e

A single client subscription request should result into a maximum of one request per DMI. Of course there can be several messages if more than 1 DMI is involved.

Aug 30, 2023

CPS-E-09.e

Amalgamate response should include rejected/accepted/pending DMI responses received within 30 seconds. A client shall be notified of available DMI subscription information after 30 seconds.  Subsequent DMI subscription updates shall be notified to clients as they become available.

• Same schema for all notifications.

• Subsequent notifications contains the state of all cmhandles involved in the subscription. 

Aug 30, 2023

CPS-NCMP-I-01

CM Handle deletion should NOT update subscription details. 

Do NOT delete dmi-subscription entry until owning subscription is deleted, see issue #4 below

Aug 30, 2023

CPS-E-09.e

Sep 2, 2025 Currently ALL CM Data Notifications are forwarded unconditionally. 
event forwarded should become transactional ie. when the incoming event is rolled back (instead of committed) our outgoing event should also be rolled back.
This should implemented by standard Kafa Configuration and Spring Support

• NCMP shall support transactional forwarding of cm data notification events

• NCMP shall ensure only one message is sent

• All agreed requirement can be found - 

Sep 16, 2025 

Non responding DMIs (configurable timeout)

An error message listing the 'pending' CM Handle IDs. statusMessage/statusCode to be agreed. 

Sep 5, 2023

Non Existing or non-valid CM Handle Ids

An error message listing the 'rejected' CM Handle IDs.

Sep 5, 2023

DMI Plugin without CM Event subscription support

An error message listing the 'rejected' CM Handle IDs 

Sep 5, 2023

Late Response (after timeout defined in #1) from DMI

A list of outcomes (complete status on how many cmhandles are pending/rejected/accepted) from client subscription is returned in response. 

statusMessage/statusCode to be agreed. 

Sep 13, 2023 

Create for existing subscription id

The whole subscription will be rejected. statusMessage/statusCode to be agreed. 

Sep 13, 2023 

Delete for non-existing subscription id

The whole delete subscription will be rejected. statusMessage/statusCode to be agreed. 

Sep 13, 2023 

Error upon error

An error scenario on a second subscription for the same cm-handle/xpath as a previous subscription which did not complete successfully (yet)

to be discussed, see Error Upon Error Combinations

Frequency of LCM Subscription Events

No high volume of such events is expected
For testing purpose 1 LCM event per minute would suffice.

@kieran mccarthy confirmed

Sep 13, 2023 

Number of Subscriptions

Max: 200

@kieran mccarthy confirmed

Sep 5, 2023

CM Handles per Subscription

Avg: 200

Max: 500 (document only)

@kieran mccarthy to confirm
Note: consider limit and future 'simple' ALL (wildcard) option when above that limit

Sep 5, 2023

Xpaths per Cm Handle

Avg: 10

@kieran mccarthy 

Sep 5, 2023

Status update processing time

(Order of 30) seconds (NCMP side)

@kieran mccarthy 

Sep 5, 2023

Targets is a collection of xpaths

can be implemented as a json array of strings

for the DMI plugin cm-handle-id, datastore and xpath are the key to a unique subscription

the client-id and subscription name are not relevant for the DMI (should not be included in the request) as this would break the concept of merging!

Is it intended that CM Notification subscription request  cover (all) descendants of the given xpath too?!

e.g.. if a child is removed and there is a subscription for the parent node, will a notification be send (grandchild, child leaf updates etc.) I hope NOT!
Consider:

• Risk of client effectively subscribing to ALL data in a cm handle by specifying top level datanode(s)

• Complexity (i.e. cost of) of merge operation. It might even required NCMP to check relevant dat model

• Future use of wildcards, could be a viable alternative for including descendants

Aug 24, 2023

Descendent don't need to be taken into account

Could xpath point to an element that does not exist (yet)

if not how, how can I client be informed about a create event? 

Aug 24, 2023

xpath can point to an element that does not exist (yet)

CPS/NCMP Does not need to check Xpath

Should NCMP support re-homing, moving of a CM Handle from one DMI to another?

assume only trough delete & create 

Aug 24, 2023

only trough delete & create

CM Handle Delete: Should DMI or Clients be sent a subscription update 

do NOT delete dmi-subscription entry until owning subscription is deleted
(just ignore upon future delete if cm handle is gone altogether)

Note. LCM is already broadcast (today) 

• Aug 24, 2023

No, subscription update

• Need to upgrade to CloudEevent format

• Add subscription id (name+client id) to header (so client can filter)

Validation of xpath

options order of implement and also performance cost!

1. none

2. xpath-parser

3. model check

4. instance check

Aug 24, 2023

1. None

can DMI plugin 'reject' a subscription create (for a given cm-handle-xpath combination)

As NCMP might not validate as per issue#5 the DMI=plugin or component further down might have to reject an invalid xpath...

@Priyank Maheshwari 

yes currently DMI can use response to say which cm handles are not accepted i.e. rejected' (but not 'pending') 

Aug 24, 2023

implementation question: should 'rejected' DMI-subscriptions be stored

not needed as whole subscription should be rejected

Aug 24, 2023

not needed as whole subscription should be rejected

Dimensioning of DB depends on #cm handles, #subscriptions and #xpaths per subscription, this could be too big for fast processing of updates! 

Need to agree maximum and possibly realistic average/total number of entries based on the characteristics  above
The team is blocked until this becomes clear as it will affect the way the data needs to be modelled exactly.

Sep 13, 2023 - @kieran mccarthy- Dimensioning has been agreed.  See Characteristics

Maximum (error) message size

theoretically all cm handles and all xpaths combinations could be rejected or pending leading to a very large error message!

Aug 24, 2023

?? what was agreed ? @Kolawole Adebisi-Adeolokun to update what was agree

can each CM-Handle have different set of xpath(s) per subscription

the  current 'basic' solution only supports a common set of datastore/xpaths (filter)

Aug 24, 2023

Yes, schema modification allows this

can the same cm handle/xpath have different subscriptions with different datastores, does that affect the cm data notifications send (which datastore applies)

• can register on any (passthrough) datastore

  • running

  • operational
    
    

• compulsory, validation needed

• part of KEY (unique entry) so cmHandle+datastore+xpath is the key 

Aug 28, 2023

Although this wiki-page will not be updated where cmHandle+xpath is mentioned as  unique entry etc. this should now include a third field: datastore as well.

Will migration from 'basic' be supported

Preferred to ask customers to create new subscriptions

Aug 24, 2023

No, basic sub. will not be delivered as a separate feature 

list v list instances filtering

/p/c1
/p/c1[@key=23]
Are these different paths or not (from ncmp point of view)

Aug 24, 2023 List entries are unique.
*List instances might not always be supported, depending DMI Plugin impl.

confirm subscription id (currently subscription name + client id)

previously was clientid and subcrioptionid and we were relying on concatenation of both to be unique

Sep 5, 2023

Now only the event would contain subcrioptionid 

what subscription details should be send when there is a change (in the union)

1. just delta

2. just union

3. union and delta (delta flagged) 

Aug 24, 2023 Just Delta

one DMI rejects whole (see decision #6) subscription (affected cmhandles) but other DMI accepts the same subscription, is this possible how to handle

Aug 28, 2023

Yes, this has been analysed

What all datastores are supported ?

ncmp-datastore:passthrough-running or ncmp-datastore:passthrough-operational ??

or both?

Sep 13, 2023  - @kieran mccarthy  Both will be supported

When NCMP sends subscription request to DMI and then DMI responds back to NCMP , do we need to have a correlationId to track/map the request and response? 

Value: subscriptionID:DMIPluginName (Check if subscriptionId or DMIPlugin name doesn't contain a colon: ) - The seperator is to be chosen by the development team, will try with colon first. 

Sep 18, 2023

subscriptionId to be used.

Which status codes to be used when the DMI Plugins respond back to NCMP ? 

We already have codes defined in NCMP. Do we reuse the same status codes when DMI Plugin responds back to NCMP with the subscription status ?

Yes, reuse. 

Sep 18, 2023 @kieran mccarthy - Reuse the same codes defined in NCMP.  

Impl. Proposal for Merging of Subscription

Oct 2, 2023

Schemas Agreed for the below events ( Source to Destination ) for the positive scenarios.

1. DME to NCMP

2. NCMP to DMI-Plugin

3. DMI-Plugin to NCMP 

4. NCMP to DME

Create DME subscription interfaces Schema Change

Schema Change, Events driven change now HTTP Rest. Study status is at 2/3 of review. CPS need to study new changes to subscription once study is approved. 

New Subscription changes forecasted for 25Q2.
Deliver current event based subscription discuss with wider stakeholders on May 29, 2025 

Create DME subscription interfaces as a new subscription epic -[CPS-2233] CM Subscription with DME interface - ONAP Jira

May 21, 2024 

@Kolawole Adebisi-Adeolokun  to create backlog Jira