Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 31 Next »

References

CPS-1870 - Getting issue details... STATUS

CPS-1812 - Getting issue details... STATUS


Assumptions

#

Issue

Notes

Decisions

1





Issues & Decisions

#

Issue

Notes

Decisions

1

Whether we need to persist the raw request for auditing purpose?



2

Keep track of the correlation id

Request to and from DMI Plugin has to be tracked hence we need to have a correlation id for this. ( to match the request and response )

Overview

This page only contains the details used for starting the development on the Merging of Subscriptions use case. Just the straight forward scenarios will be covered and the overall solution will be build on top of it.


Schema Definitions


  1. DME to NCMP Subscription Request


    Name

    Parent

    Type

    Example

    		Mandatory ?

    Notes

    1idheaderString
    		Y
    2versionheaderString 1.0Y
    3

    source

    		headerStringDMEY
    4

    type

    		headerStringsubscriptionCreateRequestY
    5

    dataschema

    		headerURIorg.onap.ncmp.cm.subscription:1.0.0Y
    6

    data


    		Object
    		N
    7

    subscriptionId

    		dataString
    		Y
    8

    predicates

    		dataArray
    		Y
    9

    targetFilter

    		predicatesArray of String
    		Y
    10

    scopeFilter

    		predicatesObject
    		Y
    11

    datastore

    		scopeFilterString
    		Y
    12

    xpathFilter

    		scopeFilterArray of String
    		Y


    Example

    DME to NCMP Subscription Request 
    id : random-UUID
version : "1.0"
source : "DME"
type : "subscriptionCreateRequest",
dataschema : org.onap.ncmp.cm.subscription:1.0.0  {
  "data": {
    "subscriptionId": "unique subscription id", // mandatory
    "predicates": [
	{
        “targetFilter” :  list of cmhandles, // mandatory
		"scopeFilter"  :  { 
	       "datastore": “ncmp-datastore:passthrough-operational or ncmp-datastore:passthrough-running", // optional. default is passthrough-operational
           "xpath-filter": list of valid xpaths  // mandatory
      }
    }
	]
}

  2. NCMP to DMI-Plugin Subscription Request


    Name

    Parent

    Type

    Example

    Mandatory ?

    Notes

    1idheaderString
    		Y
    2versionheaderString 1.0Y
    3

    source

    		headerStringDMEY
    4

    type

    		headerStringsubscriptionCreateRequestY
    5

    dataschema

    		headerURIorg.onap.ncmp.dmi.cm.subscription:1.0.0Y
    6

    correlationid

    		headerStringsubscriptionId#dmiPluginNameYConcatenation of subscriptionId and dmi plugin name
    7

    data


    		Object
    		Y
    8

    cmhandles

    		dataArray
    		Y
    9

    cmhandleId

    		cmhandlesString
    		Y
    10

    privateProperties

    		cmhandlesMap
    		Y
    11

    predicates

    		dataArray
    		Y
    12

    targetFilter

    		predicatesArray of String
    		Ylist of cm handle ids
    13

    scopeFilter

    		predicatesObject
    		Y
    14

    datastore

    		scopeFilterString
    		Yncmp-datastore:passthrough-operational OR ncmp-datastore:passthrough-running 
    15

    xpathFilter

    		scopeFilterArray of String
    		Y

    Example

    NCMP to DMI-Plugin Subscription Request 
    id : random UUID
version : 1.0
source : "NCMP"
type : "subscriptionCreateRequest"
dataschema : org.onap.ncmp.dmi.cm.subscription:1.0.0
correlationId ( concatenation of subscriptionId and dmi-plugin-name with a separator)

{
  "data": {
   
	"cmhandles" : [ // mandatory
	{
		"cmHandleId": "cmhandle1",
		"privateProperties": {...} 
	},
	{
		"cmHandleId": "cmhandle2",
		"privateProperties": {...}
	},
	{
		"cmHandleId": "cmhandle3",
		"privateProperties": {...}
	}
	]
    "predicates": [
	{
        “targetFilter” :  [“cmHandle1”, “cmHandle2”, "cmhandle3"], // mandatory
		"scopeFilter" : { 
	       "datastore": “ncmp-datastore:passthrough-operational or ncmp-datastore:passthrough-running", // optional. default is passthrough-operational
           "xpathFilter": list of valid xpaths  // mandatory
      }
    },  
	{
        “targetFilter” :  [“cmHandle1”, “cmHandle2”, "cmhandle3"],
		"scopeFilter" : { 
	       "datastore": “ncmp-datastore:passthrough-operational or ncmp-datastore:passthrough-running", // optional. default is passthrough-operational
           "xpathFilter": list of valid xpaths  *
      }
    }
  ]
}

  3. DMI-Plugin to NCMP Subscription Response

    DMI-Plugin to NCMP Subscription Response 
    id : random UUID
version : "1.0"
source : <dmi-plugin-name>
type : "subcriptionCreateResponse"
dataschema : org.onap.ncmp.dmi.cm.subscription:1.0.0
correlation-id : ( concatenation of subscriptionId and dmi-plugin-name with a separator) 
{ 
"data" : {
   "statusCode": "1/104",  // mandatory
   "statusMessage" : "ACCEPTED/REJECTED" // mandatory
 }
}

  4. NCMP to DME Subscription Response

    NCMP to DME Subscription Response 
    id : random UUID
version : "1.0"
source : "NCMP"
type : "subcriptionCreateResponse",
dataschema : org.onap.ncmp.cm.subscription:1.0.0,
correlationId : <subscriptionId> {
  "data": {
		"subscriptionId": "sample-subscription-id", // mandatory
	    "accepted-targets" : ["ch-1", ...], // optional
		"rejected-targets" : ["ch-1", ...], // optional
		"pending-targets" : ["ch-1", ...],  // optional
  }
}


Storage Solutions

Proposed Data Storage

Use a combination of

  1. Database (yang modelled) for permanent subscription details, subscribers, predicates
  2. In-Memory (hazelcast) for transient subscription details:  state (acceptep, rejected, pending)

Example Flow 1: First Subscription

  • We have the below configuration managed by CPS-NCMP

    #

    cmhandle

    dmi-plugin

    1ch-1dmi-1
    2ch-2dmi-1
    3ch-3dmi-2
    4ch-4dmi-2
    5ch-5non-responding-dmi

  1. We get a Subscription Create Request with unique subscription-id and a list of predicates(list of targets , datastore and list of filters)

    Subscription Create Message 
    {
"subscriptionId" : "A-10",
"predicates" : [
  {
    "targets": [ch-1,ch-2],
    "datastore" : "ncmp-datastore:passthrough-operational",
    "datastore-xpath-filter" : ["p1/c1","p2/c2"]
    },
  { "targets": [ch-3],
    "datastore" : "ncmp-datastore:passthrough-operational",
    "datastore-xpath-filter" : ["p3/c3"]
    }
] 
}

  2. We maintain a distributed datastructure in Hazelcast to keep track of the request and response to the client.

    SubscriptionId

    dmi-plugin

    affectedCmHandles

    status

    A-10dmi-1["ch-1","ch-2"]PENDING
    A-10dmi-2["ch-3"]PENDING
  3. Read the requests from the distributed data structure and form the request for the DMI-Plugin and publish it to the internal kafka channel.
  4. If the DMI-Plugin responds back within the configured time , so the request will either be ACCEPTED or REJECTED.
    1. if whole subscription request is ACCEPTED , update the distributed data structure and then store the subscription to the database.
    2. if whole subscription request is REJECTED , update the distributed data structure and send back the response to the client and no need to store anything to database.

    3. if the DMI Plugin fails to respond then the status would remain PENDING only in the distributed datastructure as there is no change.

      SubscriptionId

      dmi-plugin

      affectedCmHandles

      status

      A-10dmi-1["ch-1","ch-2"]ACCEPTED
      A-10dmi-2["ch-3"]REJECTED


      If there are no more PENDING requests , then we can right away send the response without updating the cache.

  5. We store the "ActiveSubscriptions" using a new model which will look like below.

    SubscriptionId

    cmHandle

    filter

    datastore

    A-10ch-1p1/c1ncmp-datastore:passthrough-operational
    A-10ch-1p2/c2ncmp-datastore:passthrough-operational
    A-10ch-2p1/c1ncmp-datastore:passthrough-operational
    A-10ch-2p2/c2ncmp-datastore:passthrough-operational
  6. Now based on whatever we have in the distributed map , we will send the response to the clients and may be clear the in-memory structure if required.


Example Flow 2: Second Subscription (merge)


  1. We get a new subscription create request.

    2nd Subscription Create Message 
    {
"subscriptionId" : "B-52",
"predicates" : [
  {
    "targets": [ch-2],
    "datastore" : "ncmp-datastore:passthrough-operational",
    "datastore-xpath-filter" : ["p2/c2" , "p3/c3"]
    }
]
}


  2. We now look at the "ActiveSubscriptions" in the database if we already have an ongoing subscription either for a combination of "cmhandle and xpath filter and the datastore".

    Current view of ActiveSubscriptions

    SubscriptionId

    cmHandle

    filter

    datastore

    A-10ch-1p1/c1ncmp-datastore:passthrough-operational
    A-10ch-1p2/c2ncmp-datastore:passthrough-operational
    A-10ch-2p1/c1ncmp-datastore:passthrough-operational
    A-10ch-2p2/c2ncmp-datastore:passthrough-operational

    Check if we have ch-2 , p2/c2 and ncmp-datastore:passthrough-operational then directly add to the ActiveSubscription. Similarly check for ch-3 , p3/c3 and ncmp-datastore:passthrough-operational ( which in this case will not be directly to the active subscription)


  3. Now the state of ActiveSubscription is :

    SubscriptionId

    cmHandle

    filter

    datastore

    A-10ch-1p1/c1ncmp-datastore:passthrough-operational
    A-10ch-1p2/c2ncmp-datastore:passthrough-operational
    A-10ch-2p1/c1ncmp-datastore:passthrough-operational
    A-10ch-2p2/c2ncmp-datastore:passthrough-operational
    B-52ch-2p2/c2ncmp-datastore:passthrough-operational

    and state of the in-memory data structure is as follows.

    SubscriptionId

    dmi-plugin

    affectedCmHandles

    status

    B-52dmi-1["ch-2"]PENDING
  4. Now we send the request to DMI plugin (dmi-1) with just the private properties of the ch-2 with the delta xpath.
    1. DMI Plugin could either apply the whole subscription request or reject the whole subscription. i.e respond with accepted/rejected

  5. Based on the response from DMI plugin the in-memory structure is updated and database also updated if the request is ACCEPTED and response is sent back to the clients.

    if DMI says it is accepted then : 

    SubscriptionId

    dmi-plugin

    affectedCmHandles

    status

    B-52dmi-1["ch-2"]ACCEPTED


    if DMI says it is rejected then : 

    SubscriptionId

    dmi-plugin

    affectedCmHandles

    status

    B-52dmi-1["ch-2"]REJECTED


    if we get no response within the configured time then :

    SubscriptionId

    dmi-plugin

    affectedCmHandles

    status

    B-52dmi-1["ch-2"]PENDING


    But we still send some response within the pre-configured time.


  6. if the above request was accepted then we should add it to the list of "ActiveSubscriptions" as well in the database.

    SubscriptionId

    cmHandle

    filter

    datastore

    A-10ch-1p1/c1ncmp-datastore:passthrough-operational
    A-10ch-1p2/c2ncmp-datastore:passthrough-operational
    A-10ch-2p1/c1ncmp-datastore:passthrough-operational
    A-10ch-2p2/c2ncmp-datastore:passthrough-operational
    B-52ch-2p2/c2ncmp-datastore:passthrough-operational
    B-52ch-2p3/c3ncmp-datastore:passthrough-operational
  7. If the status was accepted/rejected then after sending the response to the client we can clear the in-memory structure for subscriptionId B-52.


    Storage Solution Discussion


      We had an internal meeting and concluded to do a POC with the below YANG schema to efficiently store the subscriptions.


Summary for Grooming

Note : This is the latest approach we are following. (Priyank Maheshwari as of  )



Tracking Subscriptions in memory


  • No labels