...
- persistently (up to 7 days) deliver all the policy-update notifications per subscriber_topic
- component will listen for the subscriber_topic of the policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
...
Requirements on component
Option - 5-1 - Less requirements on component - policy-handler to do the subscription with PDP for the component
on startup – get the subscriber_topic=“policies_DCAE_tca_<service-component-name>” from config-binding-service
listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component
calculate the delta between the current set of policies and the received shapshot of policies
...
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
@startuml r5-1_proposed_policy_update_flow allowmixing scale 4096*4096 skinparam title { FontSize 24 FontColor Blue FontStyle Bold BorderRoundCorner 15 BorderThickness 2 } skinparam roundcorner 20 skinparam component { BackgroundColor Snow } skinparam note { FontColor Black BackgroundColor azure } title = R5-1 El Alto proposal for \n policy flows between the new PDP and DCAE component \n PDP does the pub-sub for policies per component \n %date[yyyy-MM-dd HH:mm]% package "<&dollar> Policy-Engine" as policy_engine #88ff88 { component "<&dollar> new Policy-Access-Point (<b>new PAP</b>)" as PAP component "<&dollar> new Policy-Decision-Point (<b>new PDP</b>)\n==pub-sub==\n maintains the table of policy-update subscribers \nwith <&crop> policy-filters and subscriber_topic\n--subscribe--\n[0.3.2] insert-update the record for the <b>subscriber_topic</b>\n[0.3.3] notify the subscriber_topic \nwith the latest snapshot of policies that match to any of policy-filters\n--unsubscribe--\n[99.21] delete the record for the <b>subscriber_topic</b>\n==[1] on policy push/delete from PAP==\n[1.0] select all subscribers that match to the pushed/deleted policies \n by any <&crop> policy-filter\n[1.1] for each affected subscriber retrieve all the latest snapshot of policies \n[1.2] notify each subscriber_topic separately \nwith the latest snapshot of policies" as PDP database "<&list> <b>policy-update subscribers</b> (table in database)\n..unique topic per each component instance..\n<b>subscriber_topic</b> TEXT <&key> PK -- "policies_DCAE_tca_<service-component-name>"\n..\n action TEXT -- "configure"\nONAPName TEXT -- "DCAE"\nONAPComponent TEXT -- "tca"\nONAPInstance TEXT -- "<service-component-name>"\n..list of policy-filters on subscriber_topic (component)..\n<&list> <&crop> <b>policy-filters</b> JSON -- list of the <b>"resource"</b> objects of /decision/v1: \n[{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}, ...]" as policy_update_subscribers } package "<&dollar> DCAE" as DCAE { package "<&signpost> DCAE-Controller" as DCAE_Controller #eeffee { package "<&cog> deployment" #eeeeff { component "<&aperture> <b>deployment_handler</b>\n--\n[0] install component through cloudify for the blueprint+inputs\n--\n[99] uninstall component through cloudify" as deployment_handler component "<&aperture> <b>policy_handler</b>\n--[0.3] on startup of component - /subscribe--\n[0.3.1] get <b>subscriber_topic</b> and all policy-filters from req from plugin\n[0.3.2] subscribe with PDP for policy-updates on all <&crop> policy-filters and \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n--\n==[99.21] on shutdown of component - /unsubscribe==\n[99.21] unsubscribe <b>subscriber_topic</b> from PDP" as policy_handler package "<b>cloudify</b>" as cloudify_server #00ffff { control cloudify component "<&aperture> <b>k8s_plugin decorated with @policies_gather</b>\n--[0] install component--\n[0.0] <b>k8s_plugin</b>: save config for component into consul-kv under <service-component-name>\n[0.1] <b>@policies_gather</b>: gather policy-filters assigned to component in blueprint+inputs\n..\n[0.2] <b>@policies_gather</b>: save <service policy subscriber for component into consul-kv \n<service-component-name>:<b>subscriber<b>policies/subscriber_topic</b>='policies_DCAE_tca_<service-component-name>' into consul-kv\n..\n[0.3] <b>@policies_gather</b>: subscribe the component for policy-updates through policy_handler\n pass <b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>" \nand multiple policy-filters for component as [{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}]\n..\n[0.4] <b>k8s_plugin</b>: install and start component\n--[99] uninstall component--\n[99.01] <b>@policies_gather</b>: delete recordsunsubscribe <b>subscriber_topic</b> from consul-kvPDP\n[99.12] <b>k8s<b>@policies_plugin<gather</b>: delete records from consul-kv\n[99.23] <b>k8s_plugin</b>: stop and uninstall component\n[99.4] <b>k8s_plugin</b>: delete records from consul-kv" as k8s_plugin } } database "<&target> <b>consul-kv</b>\n--config for component--\n<service-component-name>={...config...}\n<servicen-component-name>:<b>subscriber-policy subscriber_topic for component--\n<service-component-name>:<b>policies/subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'" as consul_kv component "<&aperture> <b>config_binding_service</b>" as config_binding_service } component "<&target> <b>DCAE component like TCA</b>\n--[0.4] on startup - run policy-update receiver--\n[0.4.1] get subscriber_topic along with the config from CBS\n[0.4.2] listen for policy-update notifications from <b>MR of DMaaP</b> on \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n with long-collect-polling time like 15 seconds to grab the push notification\n--\n[0.3.3] and [1.2] on receiving the push-notification of policy-update, \n[1.3] calculate the delta - compare the collection of existing policies \n versus the latest snapshot of policies and figure out \nwhat policies are added/updated/removed \n[1.4] handle and consume the added/updated/removed policies" as dcae_component #eeeeff } component "<&rss> <b>Message Router of DMaaP</b>\n--\n [0.3.3] and [1.2] persistently delivers \nthe policy-update notification \nto each <b>subscriber_topic</b> \nwith the latest snapshot of policies" as DMaaP #ff8888 PAP .down.> PDP : [1] push/delete policies PDP .down.> policy_update_subscribers : [0.3.2] subscribe\n insert/update \nsubscriber record PDP .down.> policy_update_subscribers : [1.0] iterate through \nall subscribers PDP ..> policy_update_subscribers : [99.21] unsubscribe \n delete <b>subscriber_topic</b> PDP .down.> DMaaP : [0.3.3] and [1.2] push policy-update \n for DCAE component instance DMaaP .up.> dcae_component : [0.3.3] and [1.2] push policy-update for DCAE component instance dcae_component .right.> config_binding_service : [0.4.1] get subscriber_topic along with the config config_binding_service .right.> consul_kv : [0.4.1] get config and subscriber_topic policy_handler ..> PDP : [0.3.2] POST /decision/v1/decision/<b>subscription</b>/<<b>subscriber_topic</b>> policy_handler ..> PDP : [99.21] DELETE /decision/v1/decision/<b>subscription</b>/<<b>subscriber_topic</b>> deployment_handler .down.> cloudify : [0] install component deployment_handler .down.> cloudify : [99] uninstall \ncomponent cloudify .down.> k8s_plugin : [0] install \ncomponent cloudify .down.> k8s_plugin : [99] uninstall \ncomponent k8s_plugin .down.> consul_kv : [0.0] and [0.2] save config and <b>subscriber_topic</b> for component k8s_plugin .down.> consul_kv : [99.02] and [99.14] delete records k8s_plugin .up.> policy_handler : [0.3] subscribe k8s_plugin .up.> policy_handler : [99.21] unsubscribe k8s_plugin .left.> dcae_component : [0.4] start component k8s_plugin .left.> dcae_component : [99.23] stop component 'left note "[99.2] http DELETE /decision/<b>subscription</b>/v1/<<b>subscriber_topic</b>>" as message_unsub_to_PDP ' dcae_component .down.> message_unsub_to_PDP ' message_unsub_to_PDP ..> PDP footer <&thumb-up> no middleman (DCAE-Control) for policy update flow <&thumb-up> minimal number of messages and volume of data <&thumb-up> Message Router of DMaaP persistently delivers the push-notification of the policy-update per each component instance globally identified by subscriber_topic <b>requirements on new PDP -- see the diagram</b> <&task> maintain pub-sub table per subscriber_topic with policy-filters <&task> notify about the policy added/updated/removed through Message Router of DMaaP <b>requirements on DCAE Component -- see the diagram</b> <&task> on start, get + subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>") <&task> provide handler for the push-notification of the polciy-update received from Message Router of DMaaP <&task> listen for the policy-update notification on subscriber_topic coming through the Message Router of DMaaP with long polling collect-time <&task> calc delta of policy-update and consume it <b>requirements on Config-Binding-Service - new data from consul-kv</b> <&task> get from consul-kv and return the + subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>") <b>requirements on @policies_gather -- see the diagram</b> <b>requirements on k8s_plugin -- see the diagram</b> <b>requirements on policy_handler -- see the diagram</b> endfooter @enduml |
...
Option - 5-2 - Less requirements on component - config-binding-service to call the policy-handler to do the subscription with PDP for the component
on startup – get the subscriber_topic=“policies_DCAE_tca_<service-component-name>” and the latest policies from config-binding-service
- initialize the policies in the component with policies received from config-binding-service - as it does now.
listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component
calculate the delta between the current set of policies and the received shapshot of policies
Config-binding-service to do the following on request from component
- get policy data from consul-kv
- request policy-handler to subscribe or get policies
- policy-handler will ask PDP to add the subscription and return the policies for the component or just get the policies for the component
- policy-handler to return the policies to config-binding-service
See inside the diagram for more details, data structures, and flow steps
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
@startuml r5-2_proposed_policy_update_flow
allowmixing
scale 4096*4096
skinparam title {
FontSize 24
FontColor Blue
FontStyle Bold
BorderRoundCorner 15
BorderThickness 2
}
skinparam roundcorner 20
skinparam component {
BackgroundColor Snow
}
skinparam note {
FontColor Black
BackgroundColor azure
}
title = R5-2 El Alto proposal for \n policy flows between the new PDP and DCAE component \n PDP does the pub-sub for policies per component \n %date[yyyy-MM-dd HH:mm]%
package "<&dollar> Policy-Engine" as policy_engine #88ff88 {
component "<&dollar> new Policy-Access-Point (<b>new PAP</b>)" as PAP
component "<&dollar> new Policy-Decision-Point (<b>new PDP</b>)\n==pub-sub==\n maintains the table of policy-update subscribers \nwith <&crop> policy-filters and subscriber_topic\n--subscribe or get - returns policies for subscribed/get--\n[0.3.2] insert-update the record for the <b>subscriber_topic</b>\n[0.3.3] notify the subscriber_topic \nwith the latest snapshot of policies that match to any of policy-filters \n - both in http response and through DMaaP\n--unsubscribe--\n[99.1] delete the record for the <b>subscriber_topic</b>\n==[1] on policy push/delete from PAP==\n[1.0] select all subscribers that match to the pushed/deleted policies \n by any <&crop> policy-filter\n[1.1] for each affected subscriber retrieve all the latest snapshot of policies \n[1.2] notify each subscriber_topic separately \nwith the latest snapshot of policies" as PDP
database "<&list> <b>policy-update subscribers</b> (table in database)\n..unique topic per each component instance..\n<b>subscriber_topic</b> TEXT <&key> PK -- "policies_DCAE_tca_<service-component-name>"\n..\n action TEXT -- "configure"\nONAPName TEXT -- "DCAE"\nONAPComponent TEXT -- "tca"\nONAPInstance TEXT -- "<service-component-name>"\n..list of policy-filters on subscriber_topic (component)..\n<&list> <&crop> <b>policy-filters</b> JSON -- list of the <b>"resource"</b> objects of /decision/v1: \n[{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}, ...]" as policy_update_subscribers
}
package "<&dollar> DCAE" as DCAE {
package "<&signpost> DCAE-Controller" as DCAE_Controller #eeffee {
package "<&cog> deployment" #eeeeff {
component "<&aperture> <b>deployment_handler</b>\n--\n[0] install component through cloudify for the blueprint+inputs\n--\n[99] uninstall component through cloudify" as deployment_handler
package "<b>cloudify</b>" as cloudify_server #00ffff {
control cloudify
component "<&aperture> <b>k8s_plugin decorated with @policies_gather</b>\n--[0] install component--\n[0.0] <b>k8s_plugin</b>: save config for component into consul-kv under <service-component-name>\n[0.1] <b>@policies_gather</b>: gather policy-filters assigned to component in blueprint+inputs\n..\n[0.2] <b>@policies_gather</b>: save policy subscriber and filters for component into consul-kv \n<service-component-name>:<b>policies/subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'\n<service-component-name>:<b>policies/pdp_msg_header</b>={"action": "configure", "ONAPName": "DCAE", \n "ONAPComponent": "tca", "ONAPInstance": "<service-component-name>"}\n<service-component-name>:<b>policies/filters</b>/<filter_id>:{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}\n..\n[0.3] <b>k8s_plugin</b>: install and start component\n--[99] uninstall component--\n[99.1] <b>@policies_gather</b>: unsubscribe <b>subscriber_topic</b> from PDP\n[99.2] <b>@policies_gather</b>: delete records from consul-kv\n[99.3] <b>k8s_plugin</b>: stop and uninstall component\n[99.4] <b>k8s_plugin</b>: delete records from consul-kv" as k8s_plugin
}
}
component "<&aperture> <b>config_binding_service</b>\n..\n[0.3] <b>@policies_gather</b>: subscribe the component for policy-updates through policy_handler\n pass <b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>" \nand multiple policy-filters for component as [{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}]" as config_binding_service
database "<&target> <b>consul-kv</b>\n--config for component--\n<service-component-name>={...config...}\n--policy filters for component--\n<service-component-name>:<b>policies/subscriber_topic</b>='policies_DCAE_tca_<service-component-name>'\n<service-component-name>:<b>policies/pdp_msg_header</b>={"action": "configure", "ONAPName": "DCAE", \n "ONAPComponent": "tca", "ONAPInstance": "<service-component-name>"}\n--multiple policy-filters records (<b>"resource"</b> objects of /decision/v1 API in PDP) per component--\n<service-component-name>:<b>policies/filters</b>/<filter_id>:{"policy-id": ["onap.scaleout.tca", "onap.restart.tca"]}" as consul_kv
component "<&aperture> <b>policy_handler</b>\n--[0.3] on startup of component - subscribe or get policies by filters--\n[0.3.1] get <b>subscriber_topic</b> and all policy-filters from req\n[0.3.2] subscribe or get with PDP for policy-updates on all <&crop> policy-filters and \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n--\n==[99.1] on shutdown of component - unsubscribe==\n[99.1] unsubscribe <b>subscriber_topic</b> from PDP" as policy_handler
}
component "<&target> <b>DCAE component like TCA</b>\n--[0.3] on startup - run policy-update receiver--\n[0.3.0.1] get subscriber_topic along with the config \n and initial policies from CBS\n[0.3.0.2] listen for policy-update notifications from <b>MR of DMaaP</b> on \n<b>subscriber_topic</b>="policies_DCAE_tca_<service-component-name>"\n with long-collect-polling time like 15 seconds to grab the push notification\n--\n[0.3.3] and [1.2] on receiving the push-notification of policy-update, \n[1.3] calculate the delta - compare the collection of existing policies \n versus the latest snapshot of policies and figure out \nwhat policies are added/updated/removed \n[1.4] handle and consume the added/updated/removed policies" as dcae_component #eeeeff
}
component "<&rss> <b>Message Router of DMaaP</b>\n--\n [0.3.3] and [1.2] persistently delivers \nthe policy-update notification \nto each <b>subscriber_topic</b> \nwith the latest snapshot of policies" as DMaaP #ff8888
PAP .down.> PDP : [1] push/delete policies
PDP .down.> policy_update_subscribers : [0.3.2] subscribe\n insert/update \nsubscriber record
PDP .down.> policy_update_subscribers : [1.0] iterate through \nall subscribers
PDP ..> policy_update_subscribers : [99.1] unsubscribe \n delete <b>subscriber_topic</b>
PDP .up.> DMaaP : [0.3.3] and [1.2] push policy-update \n for DCAE component instance
DMaaP .> dcae_component : [0.3.3] and [1.2] push policy-update for DCAE component instance
dcae_component ..> config_binding_service : [0.3.0.1] get subscriber_topic \n along with the config and policies
config_binding_service .right.> consul_kv : [0.3.0.1] get config and subscriber_topic and policy-filters
policy_handler .> PDP : [0.3.2] POST /decision/v1/<b>subscription</b>/<<b>subscriber_topic</b>>
policy_handler .> PDP : [0.3.2] POST /decision/v1/ - get policies if not subscribed
policy_handler .> PDP : [99.1] DELETE /decision/v1/<b>subscription</b>/<<b>subscriber_topic</b>>
deployment_handler .down.> cloudify : [0] install component
deployment_handler .down.> cloudify : [99] uninstall \ncomponent
cloudify .down.> k8s_plugin : [0] install \ncomponent
cloudify .down.> k8s_plugin : [99] uninstall \ncomponent
k8s_plugin .down.> consul_kv : [0.0] and [0.2] save config and <b>subscriber_topic</b> for component
k8s_plugin .down.> consul_kv : [99.2] and [99.4] delete records
config_binding_service ..> policy_handler : [0.3] subscribe or get policies by filters
k8s_plugin ..> policy_handler : [99.1] unsubscribe
k8s_plugin .left.> dcae_component : [0.3] start component
k8s_plugin .left.> dcae_component : [99.3] stop component
left footer
<&thumb-up> no middleman (DCAE-Control) for policy update flow
<&thumb-up> minimal number of messages and volume of data
<&thumb-up> Message Router of DMaaP persistently delivers the push-notification of the policy-update per each component instance globally identified by subscriber_topic
<b>requirements on new PDP -- see the diagram</b>
<&task> maintain pub-sub table per subscriber_topic with policy-filters
<&task> notify about the policy added/updated/removed through Message Router of DMaaP
<b>requirements on DCAE Component -- see the diagram</b>
<&task> on start, get
+ subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
<&task> provide handler for the push-notification of the polciy-update received from Message Router of DMaaP
<&task> listen for the policy-update notification on subscriber_topic coming through the Message Router of DMaaP with long polling collect-time
<&task> calc delta of policy-update and consume it
<b>requirements on Config-Binding-Service - new data from consul-kv</b>
<&task> get from consul-kv and return the
+ subscriber_topic (globally unique for the component instance like "policies_DCAE_tca_<service-component-name>")
<b>requirements on @policies_gather -- see the diagram</b>
<b>requirements on k8s_plugin -- see the diagram</b>
<b>requirements on policy_handler -- see the diagram</b>
endfooter
@enduml |
...
Option -
...
5 - More requirements on component (initial proposal) - too much to ask the components
- on startup – subscribe to PDP with the policy-filter the component is interested in and the subscriber_topic=“policies_DCAE_tca_<service-component-name>” that that uniquely identifies the component instance
- listen for subscriber_topic of policy-update notification from MR of DMaaP with long-collect-polling time like 15 seconds to grab the push notification
on receiving the policy-update pushed notification from DMaaP, handle the policy-update that contains the full snapshot of all the policies that match to the component
calculate the delta between the current set of policies and the received shapshot of policies
- n stop, unsubscribe subscriber_topic from PDP
...