Page Comparison

...

Characteristics

Out-of-scope

...

Out-of-scope

1. Subscription Update scenarios for an ongoing subscription.
2. CM Notification Forwarding Check: When forwarding CM Notification NCMP will not check the content to see if the is a valid active subscription. It is assumed that the DMI Plugin as acted on the 'delete subscription' request (that request is NCMPs responsibility). And of course there wil be timing issues it also possible a CM Notification was send just after the subscription-delete was send (from client) but before the whole change had acted upon that. 
3. Retry: NCMP will only report when actions are pending or rejected. NCMP will not implement a retry mechanism
4. Wildcards: Wildcards or similar functionality where one string represent 0 or more xpaths is not covered as part of this requirement but it should be kept in mind as a future possibility
5. Dynamic Topic: Topic for CM Data Notifications back to client will be hardcode for now. 
   Study should consider compatibility with an 'ALL' cm handles options

...

• DMI record could reach in millions of instances
• In memory solution (ie hazelcast) can handle 10s - 100s of Megabytes (add some sample calc)
  • still need persistence if we don't have enough instances
• use many small messages reduce need for amalgamation
• amalgamated string-based solution.... 

...

Storage & Merging

Storage

• We have the below configuration managed by CPS-NCMP
  
  

  #	cmhandle	dmi-plugin
  1	ch-1	dmi-1
  2	ch-2	dmi-1
  3	ch-3	dmi-2
  4	ch-4	dmi-2
  5	ch-5	non-responding-dmi

  
  

1. We get a Subscription Create Request with unique subscription-id and a list of predicates(list of targets , datastore and list of filters)

   Code Block
   title Subscription Create Request

2. We persist the incoming request as is in our persistent store using a relevant subscription model ( this model also needs to be revised )
We also maintain a distributed datastructure in Hazelcast to keep track of the request and response to the client.
SubscriptionIddmi-pluginaffectedCmHandlesstatusA-10dmi-1["ch-1","ch-2"]PENDINGA-10dmi-2["ch-3"]PENDING
3. Read the requests from the distributed data structure and form the request for the DMI-Plugin and publish it to the internal kafka channel.
If the DMI-Plugin responds back within the configured time , so the request will either be ACCEPTED or REJECTED.
4. if whole subscription request is ACCEPTED , update the distributed data structure and then store the subscription to the ActiveSubscriptions in cps-cache.
5. if whole subscription request is REJECTED , update the distributed data structure and send back the response to the client.
if the DMI Plugin fails to respond then the status would remain PENDING only as there is no change.
SubscriptionIddmi-pluginaffectedCmHandlesstatusA-10dmi-1["ch-1","ch-2"]ACCEPTEDA-10dmi-2["ch-3"]REJECTEDWe store the "ActiveSubscriptions" using a new model which will look like below.
SubscriptionIdcmHandlefilterdatastoreA-10ch-1p1/c1ncmp-datastore:passthrough-operationalA-10ch-1p2/c2ncmp-datastore:passthrough-operationalA-10ch-2p1/c1ncmp-datastore:passthrough-operationalA-10ch-2p2/c2ncmp-datastore:passthrough-operationalNow based on whatever we have in the distributed map , we will send the response to the clients and may be clear the in-memory structure if required.

Merging ( in-progress )

1. We get a new subscription create request.
   code

   Message collapse true

   { "subscriptionId" : "A-10", "predicates" : [   {     "targets": [ch-1,ch-2],     "datastore" : "ncmp-datastore:passthrough-operational",     "datastore-xpath-filter" : ["p1/c1","p2/c2"]     },   { "targets": [ch-3],     "datastore" : "ncmp-datastore:passthrough-operational",     "datastore-xpath-filter" : ["p3/c3"]     } ]   }

   
   

2. We persist the incoming request as is in our persistent store using a relevant subscription model ( this model also needs to be revised )

3. We also maintain a distributed datastructure in Hazelcast to keep track of the request and response to the client.
   
   

   SubscriptionId	dmi-plugin	affectedCmHandles	status
   A-10	dmi-1	["ch-1","ch-2"]	PENDING
   A-10	dmi-2	["ch-3"]	PENDING

   
   

   
   

4. Read the requests from the distributed data structure and form the request for the DMI-Plugin and publish it to the internal kafka channel.
5. If the DMI-Plugin responds back within the configured time , so the request will either be ACCEPTED or REJECTED.
   1. if whole subscription request is ACCEPTED , update the distributed data structure and then store the subscription to the ActiveSubscriptions in cps-cache.
   2. if whole subscription request is REJECTED , update the distributed data structure and send back the response to the client.

   3. if the DMI Plugin fails to respond then the status would remain PENDING only as there is no change.
      
      
      

      SubscriptionId	dmi-plugin	affectedCmHandles	status
      A-10	dmi-1	["ch-1","ch-2"]	ACCEPTED
      A-10	dmi-2	["ch-3"]	REJECTED

      
      
      

6. We store the "ActiveSubscriptions" using a new model which will look like below.
   
   

   SubscriptionId	cmHandle	filter	datastore
   A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
   A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational

   
   
   

7. Now based on whatever we have in the distributed map , we will send the response to the clients and may be clear the in-memory structure if required.
   
   

Merging ( in-progress )

1. We get a new subscription create request.

   Code Block
   language xml title 2nd Subscription Create Message collapse true
   { "subscriptionId" : "B-52", "predicates" : [ { "targets": [ch-2], "datastore" : "ncmp-datastore:passthrough-operational", "datastore-xpath-filter" : ["p2/c2" , "p3/c3"] } ] }

   
   

2. We now look at the "ActiveSubscriptions"  if we already have an active subscription either for a combination of "cmhandle and xpath filter and the datastore".
   
   Current view of ActiveSubscriptions
   
   

   SubscriptionId	cmHandle	filter	datastore
   A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
   A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational

   
   

   Check if we have ch-2 , p2/c2 and ncmp-datastore:passthrough-operational then directly add to the ActiveSubscription.Similarly check for ch-3 , p3/c3 and ncmp-datastore:passthrough-operational ( which in this case will not be directly to the active subscription)
   
   

3. Now the state of ActiveSubscription is :
   
   

   SubscriptionId	cmHandle	filter	datastore
   A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
   A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational
   B-52	ch-2	p2/c2	ncmp-datastore:passthrough-operational

   
   

   and state of the in-memory datastructure data structure is as follows.
   
   

   SubscriptionId	dmi-plugin	affectedCmHandles	status
   B-52	dmi-1	["ch-2"]	PENDING

   
   
   

4. Now we send the request to DMI plugin (dmi-1) with just the private properties of the ch-2 with the delta xpath.
   
   1. DMI Plugin could either apply the whole subscription request or reject the whole subscription. i.e respond with accepted/rejected

5. Based on the response from DMI plugin the in-memory structure is updated and response is sent back to the clients.
   
   if DMI says it is accepted then : 

   SubscriptionId	dmi-plugin	affectedCmHandles	status
   B-52	dmi-1	["ch-2"]	ACCEPTED

   
   if DMI says it is rejected then : 

   SubscriptionId	dmi-plugin	affectedCmHandles	status
   B-52	dmi-1	["ch-2"]	REJECTED

   
   if we get no response within the configured time then :

   
   

   SubscriptionId	dmi-plugin	affectedCmHandles	status
   B-52	dmi-1	["ch-2"]	PENDING

   
   But we still send some response within the pre-configured time.

6. if the above request was accepted then we should add it to the list of "ActiveSubscriptions" as well.
   
   

   SubscriptionId	cmHandle	filter	datastore
   A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
   A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
   A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational
   B-52	ch-2	p2/c2	ncmp-datastore:passthrough-operational
   B-52	ch-2	p3/c3	ncmp-datastore:passthrough-operational

   
   

7. If the status was accepted/rejected then after sending the response to the client we can clear the in-memory structure for subscriptionId B-52.

...