CPS-1065 Spike: CM data notifications from NCMP to applications including subscriptions

References

CPS-1065 - Getting issue details... STATUS

Open Issues & Decisions

#IssueNotesDecision
1List scenariosNeed to clarify scenariosRefer to Internal study
2Topicswhat topics are used in what scenario (topic for each Datastore ?)

See Topics section below.


All NCMP notifications to be published on the public cm-events topic by default regardless of data type, notification type or datastore the change came from.  This topics acts as the TBAC_ALL topic.

cm-events topic name should be configurable in deployment settings.


Future : In future NCMP shall publish the notification based on TBAC where the notification is published on the appropriate target group topic cm-events[-<target-group-name>] matching the target group(s) the data type(s) is associated with.  Not in scope of this story.

3Handle unknown event schemasWrap event?Unsupported for now.  Thrown exception and log error.
4Event Flow Overview (Flow 1 and 2 )

We will need for ONAP specific solution for that and we need to check with E2E Slicing Team on how the DMI would receive the Network Changle Cm Notification ( via message bus or something else ) , how will it look.. etc etc.


We have add/remove and attribute-value changes(AVCs) . A&AI supports add/remove part.

Need to check for AVCs.


5CM Avc Subscriptions Event StructureCm Subscription Event schema is been dictated by ORAN-DME (onap component which dictates event schema) so NCMP shall adhere to the schema.We have to make up something similar( since this schema may/ may not be ready yet )  and check with Kieran on details here.
6E2E Slicing needs to persist the dataWe need to check if we need to persist the data in flow-point #5 (Also depends on question #4)
7Scope of the notifications ? Need to check if these are only AVCs  ( as the schema name suggests) Or CRUD or both. 

Event Flow Overview


CM Event Specification Outline

CM Event Specification Outline
{

  "eventId"               : "9999",                                         # some generic event uuid generated by DMI Plugin
  “eventCorrelationId”    : “cmhandleId-001”,                               # cmhandleId used for event correlation
  "eventTime"             : "2021-11-16T16:42:25-04:00",
  "eventSource"           : "org.onap.ncmp",                                                  
  "eventType"             : "org.onap.ncmp.cm-network-avc-event",           # event type for async request response events 
  ”eventSchema”           : “org.onap.ncmp:cm-network-avc-event.rfc8641",   # event schema for async request response events 
  ”eventSchemaVersion”    : “1.0",                                          # event schema version

  "event":  {
              <RFC 8641-yang-datastore-notification-payload>                                   # See following sections for example
   }

}


RFC 8641 : yang-datastore notification definition

Yang-Datastore Notification Definition
        notifications:
          +---n push-update
          |  +--ro id?                   sn:subscription-id
          |  +--ro datastore-contents?   <anydata>
          |  +--ro incomplete-update?    empty
          +---n push-change-update {on-change}?
             +--ro id?                   sn:subscription-id
             +--ro datastore-changes
             |  +--ro yang-patch
             |     +--ro patch-id    string
             |     +--ro comment?    string
             |     +--ro edit* [edit-id]
             |        +--ro edit-id      string
             |        +--ro operation    enumeration
             |        +--ro target       target-resource-offset
             |        +--ro point?       target-resource-offset
             |        +--ro where?       enumeration
             |        +--ro value?       <anydata>
             +--ro incomplete-update?    empty

'operation' Type Definitions

OperationDescription
create

create a new data resource if it does not already exist. If it already exists, return an error

delete

delete a data resource if it already exists. If it does not exists, return an error

insertInsert a new user-ordered data resource
mergeMerge the edit value with the target data resource; create if it does not already exist
move
Reorder the target data resource
replace

Replace the target data resource with the edit value

removeRemove a data resource if it already exists

Event Payload Example

Proposed CM Event (paylaod example)
{
  "eventId"                : "9999",                                         # some generic event uuid generated by DMI Plugin
  “eventCorrelationId”     : “cmhandleId-001”,                               # cmhandleId used for event correlation
  "eventTime"              : "2021-11-16T16:42:25-04:00",
  "eventSource"            : "ncmp-datastore:passthrough-operational",       # eventSource always specifies the datastore where the event is emanating from                                      
  "eventType"              : "org.onap.ncmp.cm-network-avc-event",           # event type 
  ”eventSchema”            : “org.onap.ncmp:cm-network-avc-event.rfc8641",   # event schema   
  ”eventSchemaVersion”     : “1.0",                                          # event schema version

  "event": {

"push-change-update" : {
    "datastore-changes" : {
      "ietf-yang-patch:yang-patch" : {
          "patch-id" : "34534ffd98",  # Some unreadable patch id generated by the machine 
          "edit" : [
               {
                  "edit-id" : "ded43434-1",
                  "operation" : "create",
                   "target" : "/_3gpp-common-managed-element:ManagedElement=Kista-001/_3gpp-nr-nrm-gnbdufunction:GNBDUFunction=1/_3gpp-nr-nrm-nrcelldu:NRCellDU=1",
                   "value" : {
                          "_3gpp-nr-nrm-nrcelldu:NRCellDU" :  [
                               {
                                 "id" : 1,
                              }
                           ]
                      }
              },

               {
                  "edit-id" : "ded43434-1-1",
                  "operation" : "create",
                   "target" : "/_3gpp-common-managed-element:ManagedElement=Kista-001/_3gpp-nr-nrm-gnbdufunction:GNBDUFunction=1/_3gpp-nr-nrm-nrcelldu:NRCellDU=1/attributes",
                   "value" : {
                          "attributes" :  {
                                  "cId" : 511,
                                  "userLabel" : "some-cell-label",
                                  ...
                              }
                      }
              },


              {
                  "edit-id" : "ded43434-2",
                  "operation" : "create",
                   "target" : "/_3gpp-common-managed-element:ManagedElement=Kista-001/_3gpp-nr-nrm-gnbdufunction:GNBDUFunction=1/_3gpp-nr-nrm-nrcelldu:NRCellDU=1/attributes/pLMNIdList=35301",
                   "value" : {
                         "_3gpp-nr-nrm-nrcelldu:pLMNIdList" : [
                               {
                                      "mcc" : 353,
                                      "mnc" : "01",
                                      ...

                                 }

                              }
                         ]
                      }
              },
              {
                  "edit-id"   : "ded43434-3",
                  "operation" : "merge",
                   "target" : "/_3gpp-common-managed-element:ManagedElement=Kista-001/_3gpp-nr-nrm-gnbdufunction:GNBDUFunction=1/_3gpp-nr-nrm-nrcelldu:NRCellDU=3/attributes",
                   "value" : {
                         "attributes" : 
                               {               
                                  "cId" : 412,
                                  "userLabel" : "yet-another-cell-label",
                                   ...
                              }
                    }
              },
              {
                  "edit-id" : "ded43434-4",
                  "operation" : "delete",
                   "target" : "/_3gpp-common-managed-element:ManagedElement=Kista-001/_3gpp-nr-nrm-gnbdufunction:GNBDUFunction=1/_3gpp-nr-nrm-nrcelldu:NRCellDU=4"
            }
        ]
     }
   }
 }

    }
}

CM AVC Event Subscriptions 

A client app may subscribe on change notification on different datastores.  From a client app perspective each client subscription will be treated independently BUT internally the subscriptions will be merged and published on one or more collective CM topics based on target groups.  The CM ALL target group is the default and ALL CM notifications are directed to the associated cm-events topic.  

Subscription Steps

  1. NCMP is configured at startup with the cm subscription topic information (cm topic name, kafka addressing info).
  2. Some app sends a 'CreateSubscription' event to the public cm subscription topic (cm-event-subscription).

    UsecaseParticipantsSchemaExample
    Create Subscriptionclient app → ncmp

    Protocol : Kafka Event
    Topic : cm-avc-subscription

    Event Schema
    {
      "version": "<event type version>",
      "eventType": "subscriptionCreated",
      "event": {
        "subscription": {
          "clientID": "<unique identifier for the client >",
          "name": "<unique subscription name per client>",
          "isTagged": "<yes|no>, optional parameter, default is no"
        },
        "dataType": {
          "dataspace": "<data space>",
          "dataCategory": "<data category type>",
          "dataProvider": "<data provider type>"
          "schemaName": "<schema name>"
          "schemaVersion": "<schema version>"
        },
        "predicates": {
          "<parameter>": "<value>",
          "param2": [
            "value21",
            "value22"
          ]
    
        }
      }
    }
    Create Example
    {
      "version": "1.0",
      "eventType": "subscriptionCreated",
      "event": {
        "subscription": {
          "clientID": "SCO-9989752",
          "name": "cm-subscription-001"
        },
        "dataType": {
          "dataspace": "ALL",
          "dataCategory": "CM",
          "dataProvider": "CM-SERVICE"
          "schemaName": "org.onap.ncmp:cm-network-avc-event.rfc8641"
          "schemaVersion": "1.0"
        },
        "predicates": {
            "datastore": “passthrough-operational",
            "datastore-xpath-filter": "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/ 
                 _3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
                //_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
                //_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”  
    
        }
      }
    }

                   Table 1 : Create Subscription request from App

  3. NCMP receives consumes the create cm subscription event and processes it as follows : 
    1. Persist the subscription and cm-subscription-filter information to db
    2. Read all CM handles that match the cm-filter-subscription (model matching, cmhandleId matching)
    3. If a subscription is already ongoing for the same cmhandle datastore then separate them from the list for later processing (after the existing ongoing cmhandle subscription has completed - See table 3 below, administrativeState : running->active).
      Use the administrativeState of cmhandle 'subscriptions' as shown in table 3 below.  This can be used to know if a cmhandle subscription update is ongoing.
    4. Merge the existing cmhandle filter (if one exists from a previous subscription) with the new cm-filter-subscription
    5. Record the cmhandles whose existing cm subscription filter has been modified after filter merging in step c)
  4. For the cmhandle(s) with a new or modified subscription filter, group them according to their controlling dmi-plugin and send one or more bulk subscription request(s) to the appropriate dmi-plugin(s).
             - Change the subscription administrativeState to 'updating' for the associated datastore.  Do not modify the other subscription attributes (e.g. datastore-xpath-filter) until after successful response received from dmi.
             -  send the a bulk subscription REST request to each dmi-plugin containing either a 'create' (where no previous subscription exists on the cmhandle) or an 'update' subscription info for each cmhandle as per table 2 below
             - the REST request to the dmi-plugin is asynchronous.  The dmi-plugin shall process the each of the subscription requests per cmhandle and send an event response on the dmi-cm-avc-subscription-events topic for each cmhandle. 
UsecaseParticipants Request SchemaRequest Example
Register Subscriptionsncmp → dmi

Protocol : REST
Method : POST
URI : /ncmpInventory/v1/subscriptions
Async : true

Request Body
{
  "subscriptions" : [
      {  
          "cmhandleId" : "<cmhandle-id>",
          "subscriptionType": "subscriptionCreated | subscriptionUpdated | subscriptionDeleted",
          "existingSubscriptionId" : "<existing subscription-id on remote device>,
                                                      optional, required for subscriptionUpdated | subscriptionDeleted",
          "schemaName": "<schema name>, default is org.onap.ncmp.cm-notification-event",
          "schemaVersion": "<schema version>, default is latest",
          "isTagged": "<yes|no>, optional parameter, default is no",
          "predicates": {
               "<parameter>": "<value>",
               "param2": [
                      "value21",
                      "value22"
                ]
          },
          "cmhandleProperties"  : [ 
                  <all-cmhandle-properties-as-per-normal-crud-request>
          ]
      }
  ]
}

Response : 202 Accepted  { }

Protocol : REST
Method :
POST
URI :
/ncmpInventory/v1/subscriptions
Async : true

Request Body
{
  "subscriptions" : [
      {  
          "cmhandleId" : "e34553",              (M)
          "subscriptionType": "create",        (M)
          "schemaName": "org.onap.ncmp:cm-network-avc-event.rfc8641",    (O)
          "schemaVersion": "1.0"    (O)
          "isTagged": "yes",             (O)
          "predicates"  :   {              (M)
                "datastore": “passthrough-operational",  (O)
                "datastore-xpath-filter": "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/ 
                       _3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
                      //_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
                     //_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”  
          },
          "cmhandleProperties"  : [
                  <all the cmhandle properties>
           ]
      },
      {  
          "cmhandleId" : "gg8769",             (M)
          "subscriptionType": "update",        (M)
          "existingSubscriptionId" : "6738462g3494hw9",   (O)
          "schemaName": "org.onap.ncmp:cm-network-avc-event.rfc8641",    (O)
          "schemaVersion": "1.0"    (O)
          "isTagged": "yes",             (O)
          "predicates"  :   {              (M)
                "datastore": “passthrough-operational",  (O)
                "datastore-xpath-filter" : "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/ 
                       _3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
                      //_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
                     //_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”  
          },
          "cmhandleProperties"  : [
                  <all the cmhandle properties>
           ]
      }
  ]
}

Response : 202 Accepted { }

                Table 2 : Register Subscriptions with DMI plugin

5. dmi-plugin loops through the cmhandle subscriptions and creates a new subscription or modifies an existing subscription for the remote 'device' associated with the cmhandle.
6. dmi-plugin sends a subscription response event (org.onap.ncmp:cm-avc-subscription-event.response) on the  dmi-cm-avc-subscription-events topic notifying NCMP of the subscription request status for each cmhandle
7. NCMP receives the subscription response event for the cmhandle and updates the cmhandle subscription data to reflect the response status (success/fail)
          - Update the cmhandle subscription data for the appropriate datastore
          - If subscription 'success' response, set the subscriptionId and the datastore-xpath-filter to that in the response event.  Also set administrativeState to 'active' in the same update transaction
          - If subscription 'fail' response, leave the subscriptionId and the datastore-xpath-filter as they were and modify the administrativeState from 'updating' to 'active'.   If there is no subscriptionId in the subscription response then it means there was no active subscriptions toward the remote device.  In this case the   "passthrough-operational" node should either be removed from the NCMP DB completely OR the administrativeState shall be set from 'updating' to 'inactive'.
           - This completes the subscription cycle.

UsecaseparticipantsRequest Schema / ExampleNCMP Updates 
Notify of create subscription request success
on passthrough-operational datastore -
success case
dmi → ncmp

Protocol : Kafka Event
Topic : dmi-cm-avc-subscription

dmi-cm-avc-subscription schema
{

  "eventId"               : "00001",   (M)
  "eventCorrelationId     : "cmhandle-001" (M)    
  "eventTime"             : "2021-11-16T16:42:25-04:00",                          (M)
  "eventSource"           : "ncmp-datastore:passthrough-operational",             (M) 
  "eventType"             : "org.onap.ncmp:cm-avc-subscription-event.response",   (M)
  ”eventSchema”           : “org.onap.ncmp:cm-avc-subscription-event",            (M)
  "eventSchemaVersion"    : "1.0"            (M)
 
  "event": {
         “cmHandleId”                     : “cmhandle-001”,(M)
         "subscriptionResponse": {                         (M)
             "subscriptionType"           : "created",     (M)
             “subscriptionRequestStatus”  : “success”,     (O)
             “subscriptionId”             : “cc77765sddf”, (O - Mandatory if subscriptionRequestStatus=success)
             "predicates" : {
                 "datastore" : "passthrough-operational",
                 "datastore-xpath-filter" : ""
             }
         }  
    }

}



NCMP register the new cmhandle subscription data.

Store the datastore subscription data in the cmhandle.  Also store the administrativeState that may be used to prevent race conditions between parallel subscriptions being applied.

cmhandle data in NCMP DB
{
     "cmhandleId" : <cmhandle-id>

     "subscriptions" :  {
          "operational" : {    # 'local' ncmp subscription is required with the cmhandle - store separately?
                 "administrativeState" : "active",
                 "datastore-xpath-filter" : <xpath-filter>
            },

          "running" : {   # 'local' ncmp subscription is required with the cmhandle - store subId separately?
                 "administrativeState" : "active",
                 "datastore-xpath-filter" : <xpath-filter>
            },

          "passthrough-operational" : {
                 "administrativeState" : "updating",   # if 'updating' then block/queue new sub's until 'active'
                 "subscriptionId" : "cc77765sddf",
                 "datastore-xpath-filter" : <xpath-filter>
            },

          "passthrough-running" : {
                 "administrativeState" : "active",
                 "subscriptionId" : <sub-id-of-device">,
                 "datastore-xpath-filter" : <xpath-filter>
            }

    }
}

Once NCMP receives the subscription response if shall update the cmhandle subscription state attributes as described in step 7.

Notify of modify subscription request success
on passthrough-operational datastore -
success case
dmi → ncmp

Protocol : Kafka Event
Topic : dmi-cm-avc-subscription-events

dmi-cm-avc-subscription-events schema
{

  "eventId"               : "00001",   (M)
  "eventCorrelationId     : "cmhandle-001" (M)    
  "eventTime"             : "2021-11-16T16:42:25-04:00",                          (M)
  "eventSource"           : "ncmp-datastore:passthrough-operational",             (M)      
  "eventType"             : "org.onap.ncmp:cm-avc-subscription-event.response",   (M)
  ”eventSchema”           : “org.onap.ncmp:cm-avc-subscription-event",            (M)
  "eventSchemaVersion"    : "1.0"            (M)
 
  "event": {
         “cmHandleId”                     : “cmhandle-001”,(M)
         "subscriptionResponse" : {                         (M)
             "subscriptionType"           : "modify",     (M)
             “subscriptionRequestStatus”  : “success”,     (O)
             “subscriptionId”             : “cc77765sddf”, (O - Mandatory if subscriptionRequestStatus=success)
             "predicates" : {
                 "datastore" : "passthrough-operational",
                 "datastore-xpath-filter" : "....."
             }
         }  
    }

}

                           Table 3.   DMI plugin sends Subscription Response to NCMP

Create Subscription

Detailed flow Create Subscription

 


Queued Subscriptions

Any cmhandle subscription requests that were queued during an ongoing subscription request on the same datastore shall be processed once the previous subscription has completed (cmhandle.subscriptions.<datastore>.administrativeState goes to active).
The cmhandle subscription filters will need to be re-merged with the newly registered cmhandle subscription filters before being applied to the appropriate datastore.

Update Subscription

Update is quite similar to the create case above.  The only difference is the eventType = subscriptionUpdated

UsecaseParticipantsSubscription SchemaExample
Modify Subscriptionclient app → ncmp

Protocol : Kafka Event
Topic : cm-avc-subscription

cm-avc-subscription schema
{
  "version": "<event type version>",
  "eventType": "subscriptionUpdated",
  "event": {
    "subscription": {
      "clientID": "<unique identifier for the client >",
      "name": "<unique subscription name per client>",
      "isTagged": "<yes|no>, optional parameter, default is no"
    },
    "dataType": {
      "dataspace": "<data space>",
      "dataCategory": "<data category type>",
      "dataProvider": "<data provider type>"
      "schemaName": "<schema name>"
      "schemaVersion": "<schema version>"
    },
    "predicates": {
      "<parameter>": "<value>",
      "param2": [
        "value21",
        "value22"
      ]

    }
  }
}


Subscription Updated Example
{
  "version": "1.0",
  "eventType": "subscriptionUpdated",
  "event": {
    "subscription": {
      "clientID": "SCO-9989752",
      "name": "cm-subscription-001"
    },
    "dataType": {
      "dataspace": "ALL",
      "dataCategory": "CM",
      "dataProvider": "CM-SERVICE"
      "schemaName": "org.onap.ncmp.cm-notification-event"
      "schemaVersion": "1.0"
    },
    "predicates": {
        "datastore": “passthrough-operational",
        "datastore-xpath-filter": "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/ 
             _3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
            //_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
            //_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”  

    }
  }
}

                   Table 4   Update Subscription request from client app


The only real difference is that an update subscription from the client app might result in some filters getting removed for a cmhandle.  

This means that a subscription request for a cmhandle may  subscriptionType=delete in the request to the dmi plugin in order to remove the subscription which may no longer be applicable for the cmhandle.

See below Request example showing the update-delete example (in green bold)

UsecaseParticipants Request SchemaRequest Example
Register Subscriptionsncmp → dmi

Protocol : REST
Method : POST
URI : /ncmpInventory/v1/subscriptions
Async : true

Request Body
{
  "subscriptions" : [
      {  
          "cmhandleId" : "<cmhandle-id>",
          "subscriptionType": "subscriptionCreated | subscriptionUpdated | subscriptionDeleted",
          "existingSubscriptionId" : "<existing subscription-id on remote device>,
                                                      optional, required for subscriptionUpdated | subscriptionDeleted",
          "schemaName": "<schema name>, default is org.onap.ncmp.cm-notification-event",
          "schemaVersion": "<schema version>, default is latest",
          "isTagged": "<yes|no>, optional parameter, default is no",
          "predicates": {
               "<parameter>": "<value>",
               "param2": [
                      "value21",
                      "value22"
                ]
          },
          "cmhandleProperties"  : [ 
                  <all-cmhandle-properties-as-per-normal-crud-request>
          ]
      }
  ]
}


Response :

202 Accepted 

{ }

Protocol : REST
Method :
POST
URI :
/ncmpInventory/v1/subscriptions
Async : true

Request Body
{
  "subscriptions" : [
      {  
          "cmhandleId" : "e34553",              (M)
          "subscriptionType": "create",        (M)
          "schemaName": "org.onap.ncmp:cm-network-avc-event.rfc8641",    (O)
          "schemaVersion": "1.0"    (O)
          "isTagged": "yes",             (O)
          "predicates"  :   {              (O)
                "datastore": “passthrough-operational",  (O)
                "datastore-xpath-filter": "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/ 
                       _3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
                      //_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
                     //_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”  
          },
          "cmhandleProperties"  : [          (M)
                  <all the cmhandle properties>
           ]
      },
      {  
          "cmhandleId" : "gg8769",             (M)
          "subscriptionType": "update",        (M)
          "existingSubscriptionId" : "6738462g3494hw9",   (O)
          "schemaName": "org.onap.ncmp:cm-network-avc-event.rfc8641",    (O)
          "schemaVersion": "1.0"    (O)
          "isTagged": "yes",             (O)
          "predicates"  :   {              (O)
                "datastore": “passthrough-operational",  (O)
                "datastore-xpath-filter" : "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/ 
                       _3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
                      //_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
                     //_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”  
          },
          "cmhandleProperties"  : [          (M)
                  <all the cmhandle properties>
           ]
      },
      {  
          "cmhandleId" : "ff56743",             (M)
          "subscriptionType": "delete",    (M)
          "subscriptionId" : "6738462g3494hw9",   (O but must be set for update or delete subscriptionType)
          "cmhandleProperties"  : [            (M)
                  <all the cmhandle properties>
           ]
      }
  ]
}

Response : 202 Accepted  { }

CM AVC Subscription Response - delete dmi subscription case 

UsecaseparticipantsRequest Schema / ExampleNCMP Updates 
Notify of delete subscription request success
on passthrough-operational datastore -
success case
dmi → ncmp

Protocol : Kafka Event
Topic : dmi-cm-avc-subscription

Schema
{

  "eventId"               : "00001",   (M)
  "eventCorrelationId     : "ff56743" (M)    
  "eventTime"             : "2021-11-16T16:42:25-04:00",                          (M)
  "eventSource"           : "ncmp-datastore:passthrough-operational",             (M) 
  "eventType"             : "org.onap.ncmp:cm-avc-subscription-event.response",   (M)
  ”eventSchema”           : “org.onap.ncmp:cm-avc-subscription-event",            (M)
  "eventSchemaVersion"    : "1.0"            (M)
 
  "event": {
         “cmHandleId”                     : “ff56743”,     (M)
         "subscriptionResponse"           : {              (M)
             "subscriptionType"           : "delete",      (M)
             “subscriptionRequestStatus”  : “success”,     (O)
             “subscriptionId”             : “cc77765sddf”, (O - Mandatory if subscriptionRequestStatus=success)
         }  
    }

}



NCMP register the new cmhandle subscription data.

Store the datastore subscription data in the cmhandle.  Also store the administrativeState that may be used to prevent race conditions between parallel subscriptions being applied.

CmHandle Data in NCMP DB
{
     "cmhandleId" : <cmhandle-id>
     "subscriptions" :  {
          "operational" : {    # 'local' ncmp subscription is required with the cmhandle - store separately?
                 "administrativeState" : "active",
                 "datastore-xpath-filter" : <xpath-filter>
            },
          "running" : {   # 'local' ncmp subscription is required with the cmhandle - store subId separately?
                 "administrativeState" : "active",
                 "datastore-xpath-filter" : <xpath-filter>
            },
          "passthrough-operational" : {
                 "administrativeState" : "running"             →   'inactive'
                 "subscriptionId" : "cc77765sddf",          →   remove or set to ""
                 "datastore-xpath-filter" : <xpath-filter>  →  remove or set to ""
            },
          "passthrough-running" : {
                 "administrativeState" : "active",
                 "subscriptionId" : <sub-id-of-device">,
                 "datastore-xpath-filter" : <xpath-filter>
            }
    }
}

Once NCMP receives the subscription response if shall update the cmhandle subscription state attributes as described in step 7.


Topics 

Topic types

Topic NameDescriptionDirection
cm-avc[-<targetgroup>] topic for publication of events from NCMP to clients (suggest to rename existing topic from 'ncmp-events' to cm-avc).  In preparation to future needs the there may be separate topics for different security groups.  NCMP solution should consider this in its design this upfront.  By default cm-avc is the default topic where ALL events are published by default.  Different client may subscribe to different topics (ALL and/or target group specific).  It is their responsibility to subscribe to the appropriate topics as new target groups come into existance.  For now the only topic in scope of this story is the 'cm-avc' topic.ncmp → client consumer
dmi-cm-eventsinternal topic for communications between dmi-plugin ↔ ncmp for avc and subscription eventsncmp→ dmi 
cm-avc-subscriptiontopic for avc event subscriptions.  This topic may be (pre)defined external to NCMP.  If externally defined topic then it shall be possible to tell NCMP the topic name and where to find it.client → ncmp
dmi-cm-avc-subscriptiontopic for avc event subscriptions event toward dmi plugin(s).  This topic is internally defined by NCMP.  All CM AVC subscriptions will be sent from NCMP to DMI plugins via this topic.

ncmp → dmi for  subscription LCM

dmi → ncmp register device subscription in cmhandle

Topic partitioning

All events for the same cmhandleId should be sent to the same partition.  This will guarantee ordering.  Ordering is only guaranteed per topic, not across topic (future consideration).

DataBase Design, Subscription and Filter Tables

The client subscriptions may be stored in a simple Subscriptions table.

The individual cm filter parts that make up the subscription may be stored in their own rows in the database.

Once an individual cm subscription filter has been applied to a cmhandle (after subscription 'success' event is sent from dmi to ncmp) the matching filter row in the Filters table can be updated with the cmhandle that has taken on the new filter. 

Each time a client subscription filter applies to a cmhandle the filterUsage count is incremented.  A filter that could already have been applied to a cmhandle would get its filterUsageCount incremented if a new subscription applies to a cmhandle that already has the filter applied to it.  This helps when removing subscriptions... the filterUsageCount will be decremented each time a filter is removed.  When filterUsageCount goes to zero the associated cmhandles can be easily identified for update to remove that filter.  Simple DB design to identify affected cmhandles - can also use foreign keys to do similar.

                       

CmHandle related tables in the DB will be updated to store the subscription state for the appropriate cmhandle datastores

Subscription Data

Data similar to the below needs to be reflected in the related cmhandle data model

CmHandle Data Model
{
     "cmhandleId" : <cmhandle-id>
     "subscriptions" :  {
          "operational" : {    # 'local' ncmp subscription is required with the cmhandle - store separately?
                 "administrativeState" : "active",
                 "datastore-xpath-filter" : <xpath-filter>
            },

          "running" : {   # 'local' ncmp subscription is required with the cmhandle - store subId separately?
                 "administrativeState" : "active",
                 "datastore-xpath-filter" : <xpath-filter>
            },

          "passthrough-operational" : {
                 "administrativeState" : "updating",   # if 'updating' then block/queue new sub's until 'active'
                 "subscriptionId" : "cc77765sddf",
                 "datastore-xpath-filter" : <xpath-filter>
            },

          "passthrough-running" : {
                 "administrativeState" : "active",
                 "subscriptionId" : <sub-id-of-device">,
                 "datastore-xpath-filter" : <xpath-filter>
            }

    }
}

The subscriptionId in the passthrough-* datastore subscription information represents the subscription created on the remote device.

DMI plugin sends this subscriptionId of the device to NCMP in the subscription response event after successful subscription on the device.

The subscription may be different on the device if setting up a subscription toward different datastores.  If the device only supports running datastore (non-NMDA) then all subscriptions are directed toward the running datastore regardless of whether passthrough-running or passthrough-operational is specified in the subscription.  It would mean that the same notification would be sent twice but the eventSource would differ (passthrough-operational and passthrough-running).  However, this is a concern for the dmi plugin only - not for NCMP.