References
CPS-1065 - Getting issue details... STATUS
Open Issues & Decisions
| # | Issue | Notes | Decision |
|---|---|---|---|
| 1 | List scenarios | Need to clarify scenarios | Refer to Internal study |
| 2 | Topics | what topics are used in what scenario (topic for each Datastore ?) | See Topics section below.
cm-events topic name should be configurable in deployment settings.
|
| 3 | Handle unknown event schemas | Wrap event? | Unsupported for now. Thrown exception and log error. |
Event Flow Sketch
CM Event Specification Outline
{ "eventId" : "9999", # some generic event uuid generated by DMI Plugin "event": { } |
|---|
RFC 8641 : yang-datastore notification definition
notifications: +---n push-update | +--ro id? sn:subscription-id | +--ro datastore-contents? <anydata> | +--ro incomplete-update? empty +---n push-change-update {on-change}? +--ro id? sn:subscription-id +--ro datastore-changes | +--ro yang-patch | +--ro patch-id string | +--ro comment? string | +--ro edit* [edit-id] | +--ro edit-id string | +--ro operation enumeration | +--ro target target-resource-offset | +--ro point? target-resource-offset | +--ro where? enumeration | +--ro value? <anydata> +--ro incomplete-update? empty |
|---|
'operation' type definition
Event Payload Example
| Proposed CM event |
|---|
{ "eventId" : "9999", # some generic event uuid generated by DMI Plugin "event": { "push-change-update" : { "datastore-changes" : { {
} } } |
CM AVC Event Subscription
A client app may subscribe on change notification on different datastores. From a client app perspective each client subscription will be treated independently BUT internally the subscriptions will be merged and published on one or more collective CM topics based on target groups. The CM ALL target group is the default and ALL CM notifications are directed to the associated cm-events topic.
Create Subscription
The Create Subscription flow is as follows :
Subscription Steps :
- NCMP is configured at startup with the cm subscription topic information (cm topic name, kafka addressing info).
Some app sends a 'CreateSubscription' event to the public cm subscription topic (cm-event-subscription).
Usecase Participants Schema Create Example Create Subscription client app → ncmp Protocol : Kafka Event
Topic : cm-avc-subscription{
"version": "<event type version>",
"eventType": "subscriptionCreated",
"event": {
"subscription": {
"clientID": "<unique identifier for the client >",
"name": "<unique subscription name per client>",
"isTagged": "<yes|no>, optional parameter, default is no"
},
"dataType": {
"dataspace": "<data space>",
"dataCategory": "<data category type>",
"dataProvider": "<data provider type>"
"schemaName": "<schema name>"
"schemaVersion": "<schema version>"
},
"predicates": {
"<parameter>": "<value>",
"param2": [
"value21",
"value22"
]}
}
}{
"version": "1.0",
"eventType": "subscriptionCreated",
"event": {
"subscription": {
"clientID": "SCO-9989752",
"name": "cm-subscription-001"
},
"dataType": {
"dataspace": "ALL",
"dataCategory": "CM",
"dataProvider": "CM-SERVICE"
"schemaName": "org.onap.ncmp:cm-network-avc-event.rfc8641"
"schemaVersion": "1.0"
},
"predicates": {
"datastore": “passthrough-operational",
"datastore-xpath-filter": "//_3gpp-nr-nrm-gnbdufunction:GNBDUFunction/
_3gpp-nr-nrm-nrcelldu:NRCellDU/ | //_3gpp-nr-nrm-gnbcuupfunction:GNBCUUPFunction// |
//_3gpp-nr-nrm-gnbcucpfunction:GNBCUCPFunction/_3gpp-nr-nrm-nrcelldu:NRCellCU// |
//_3gpp-nr-nrm-nrsectorcarrier:NRSectorCarrier//”}
}
}Table 1 : Create Subscription request from App
- NCMP receives consumes the create cm subscription event and processes it as follows :
- Persist the subscription and cm-subscription-filter information to db
- Read all CM handles that match the cm-filter-subscription (model matching, cmhandleId matching)
- If a subscription is already ongoing for the same cmhandle datastore then separate them from the list for later processing (after the existing ongoing cmhandle subscription has completed - See table 3 below, administrativeState : running->active).
Use the administrativeState of cmhandle 'subscriptions' as shown in table 3 below. This can be used to know if a cmhandle subscription update is ongoing. - Merge the existing cmhandle filter (if one exists from a previous subscription) with the new cm-filter-subscription
- Record the cmhandles whose existing cm subscription filter has been modified after filter merging in step c)
- For the cmhandle(s) with a new or modified subscription filter, group them according to their controlling dmi-plugin and send one or more bulk subscription request(s) to the appropriate dmi-plugin(s).
- Change the subscription administrativeState to 'updating' for the associated datastore. Do not modify the other subscription attributes (e.g. datastore-xpath-filter) until after successful response received from dmi.
- send the a bulk subscription REST request to each dmi-plugin containing either a 'create' (where no previous subscription exists on the cmhandle) or an 'update' subscription info for each cmhandle as per table 2 below
- the REST request to the dmi-plugin is asynchronous. The dmi-plugin shall process the each of the subscription requests per cmhandle and send an event response on the dmi-cm-avc-subscription-events topic for each cmhandle.
| Usecase | Participants | Request Schema | Request Example |
|---|---|---|---|
| Register Subscriptions | ncmp → dmi | Protocol : REST Request Body: { Response : 202 Accepted { } | Protocol : REST Request Body: { Response : 202 Accepted { } |
Table 2 : Register Subscriptions with DMI plugin
5. dmi-plugin loops through the cmhandle subscriptions and creates a new subscription or modifies an existing subscription for the remote 'device' associated with the cmhandle.
6. dmi-plugin sends a subscription response event (org.onap.ncmp:cm-avc-subscription-event.response) on the dmi-cm-avc-subscription-events topic notifying NCMP of the subscription request status for each cmhandle
7. NCMP receives the subscription response event for the cmhandle and updates the cmhandle subscription data to reflect the response status (success/fail)
- Update the cmhandle subscription data for the appropriate datastore
- If subscription 'success' response, set the subscriptionId and the datastore-xpath-filter to that in the response event. Also set administrativeState to 'active' in the same update transaction
- If subscription 'fail' response, leave the subscriptionId and the datastore-xpath-filter as they were and modify the administrativeState from 'updating' to 'active'. If there is no subscriptionId in the subscription response then it means there was no active subscriptions toward the remote device. In this case the "passthrough-operational" node should either be removed from the NCMP DB completely OR the administrativeState shall be set from 'updating' to 'inactive'.
- This completes the subscription cycle.
| Usecase | participants | Request Schema / Example | NCMP Updates |
|---|---|---|---|
| Notify of create subscription request success on passthrough-operational datastore -success case | dmi → ncmp | Protocol : Kafka Event {
} | NCMP register the new cmhandle subscription data. Store the datastore subscription data in the cmhandle. Also store the administrativeState that may be used to prevent race conditions between parallel subscriptions being applied. cmhandle data in NCMP DB "subscriptions" : { "running" : { # 'local' ncmp subscription is required with the cmhandle - store subId separately? "passthrough-operational" : { "passthrough-running" : { } Once NCMP receives the subscription response if shall update the cmhandle subscription state attributes as described in step 7. |
| Notify of modify subscription request success on passthrough-operational datastore -success case | dmi → ncmp | Protocol : Kafka Event {
} |
Table 3. DMI plugin sends Subscription Response to NCMP
Queued Subscriptions
Any cmhandle subscription requests that were queued during an ongoing subscription request on the same datastore shall be processed once the previous subscription has completed (cmhandle.subscriptions.<datastore>.administrativeState goes to active).
The cmhandle subscription filters will need to be re-merged with the newly registered cmhandle subscription filters before being applied to the appropriate datastore.
Update Subscription
Update is quite similar to the create case above. The only difference is the eventType = subscriptionUpdated
| Usecase | Participants | Subscription Schema | Subscription Updated Example |
|---|---|---|---|
| Modify Subscription | client app → ncmp | Protocol : Kafka Event { } | { } |
Table 4 Update Subscription request from client app
The only real difference is that an update subscription from the client app might result in some filters getting removed for a cmhandle.
This means that a subscription request for a cmhandle may subscriptionType=delete in the request to the dmi plugin in order to remove the subscription which may no longer be applicable for the cmhandle.
See below Request example showing the delete example (in green bold)
| Usecase | Participants | Request Schema | Request Example |
|---|---|---|---|
| Register Subscriptions | ncmp → dmi | Protocol : REST Request Body: { Response : 202 Accepted { } | Protocol : REST Request Body: { Response : 202 Accepted { } |
CM AVC Subscription Response - delete dmi subscription case
| Usecase | participants | Request Schema / Example | NCMP Updates |
|---|---|---|---|
| Notify of delete subscription request success on passthrough-operational datastore -success case | dmi → ncmp | Protocol : Kafka Event {
} | NCMP register the new cmhandle subscription data. Store the datastore subscription data in the cmhandle. Also store the administrativeState that may be used to prevent race conditions between parallel subscriptions being applied. cmhandle data in NCMP DB "subscriptions" : { "running" : { # 'local' ncmp subscription is required with the cmhandle - store subId separately? "passthrough-operational" : { "passthrough-running" : { } Once NCMP receives the subscription response if shall update the cmhandle subscription state attributes as described in step 7. |
Topics
Topic types
| Topic Name | Description | Direction |
|---|---|---|
| cm-avc[-<targetgroup>] | topic for publication of events from NCMP to clients (suggest to rename existing topic from 'ncmp-events' to cm-avc). In preparation to future needs the there may be separate topics for different security groups. NCMP solution should consider this in its design this upfront. By default cm-avc is the default topic where ALL events are published by default. Different client may subscribe to different topics (ALL and/or target group specific). It is their responsibility to subscribe to the appropriate topics as new target groups come into existance. For now the only topic in scope of this story is the 'cm-avc' topic. | ncmp → client consumer |
| dmi-cm-events | internal topic for communications between dmi-plugin ↔ ncmp for avc and subscription events | ncmp→ dmi |
| cm-avc-subscription | topic for avc event subscriptions. This topic may be (pre)defined external to NCMP. If externally defined topic then it shall be possible to tell NCMP the topic name and where to find it. | client → ncmp |
| dmi-cm-avc-subscription | topic for avc event subscriptions event toward dmi plugin(s). This topic is internally defined by NCMP. All CM AVC subscriptions will be sent from NCMP to DMI plugins via this topic. | ncmp → dmi for subscription LCM dmi → ncmp register device subscription in cmhandle |
Topic partitioning
All events for the same cmhandleId should be sent to the same partition. This will guarantee ordering. Ordering is only guaranteed per topic, not across topic (future consideration).
DataBase Design
Subscription and Filter Tables
The client subscriptions may be stored in a simple Subscriptions table.
The individual cm filter parts that make up the subscription may be stored in their own rows in the database.
Once an individual cm subscription filter has been applied to a cmhandle (after subscription 'success' event is sent from dmi to ncmp) the matching filter row in the Filters table can be updated with the cmhandle that has taken on the new filter.
Each time a client subscription filter applies to a cmhandle the filterUsage count is incremented. A filter that could already have been applied to a cmhandle would get its filterUsageCount incremented if a new subscription applies to a cmhandle that already has the filter applied to it. This helps when removing subscriptions... the filterUsageCount will be decremented each time a filter is removed. When filterUsageCount goes to zero the associated cmhandles can be easily identified for update to remove that filter. Simple DB design to identify affected cmhandles - can also use foreign keys to do similar.
