The idea of using control loops to automatically (or autonomously) perform network management has been the subject of much research in the Network Management research community, see this paper for some background. However, it is only with the advent of ONAP that we have a platform that supports control loops for network management. Before ONAP, Control Loops have been implemented by hard-coding components together and hard coding logic into components. ONAP has taken a step forward towards automatic implementation of Control Loops by allowing parameterization of Control Loops that work on the premise that the Control Loops use a set of analytic, policy, and control components connected together in set ways.

The goal of the work is to extend and enhance the current ONAP Control Loop support to provide a complete open-source framework for Control Loops. This will enhance the current support to provide TOSCA based Control Loop definition and development, commissioning and run-time management. The participants that comprise a Control Loop and the metadata needed to link the participants together to create a Control Loop are specified in a standardized way using the OASIS TOSCA modelling language. The TOSCA description is then used to commission, instantiate, and manage the Control Loops in the run time system.

We consider Control Loops at Design Time and Run Time.

At Design Time, there are two capabilities to be supported:

Participant Metadata Definition. This capability allows external users and systems (such as SDC or DCAE-MOD) to define participants that can take part in a control loop and to define the metadata that can be used on and configured on a participant when it is taking part in in a control loop. The post condition of an execution of this capability is that a participant is defined in the Control Loop Design Time Catalogue together with sets of metadata that can be used with this participant in control loops.
Control Loop Composition. This capability allows users and other systems to create a control loop type by connecting a chain of participants together from the participants that are available in the Control Loop Design Time Catalogue. In an execution of this capability, a user will define the control loop chain, define the connections between participants, will select the correct metadata sets for each participant, and will define the overall control loop metadata. The post condition for an execution of this capability is a Control Loop definition in TOSCA stored in the Control Loop Design Time Catalogue.

At Run Time, the following capabilities are to be supported:

Participant Registration. This capability allows participants to register and deregister with CLAMP. The post condition for an execution of this capability is that a participant is available for participation in a control loop.
Control Loop Commissioning. This capability allows version controlled Control Loops to be taken from the Control Loop Design Time Catalogue and be placed in the Control Loop Run Time Catalogue. It also allows configuration of parameters that apply to the Control Loop Type, that is parameters that will apply to all control loop instances. Further, it allows control loop types to be commissioned on participants. Data that applies to all instances of a control loop type on a participant is sent to a participant. The participant can then take whatever actions it need to do to support the control loop type in question. The post condition for an execution of this capability is that the Control Loop definition is in the Control Loop Run Time Catalogue and all participants in this control loop type are commissioned, that is they are prepared to run instances of this control loop type.
Control Instantiation. This capability allows an instance of a control loop to be created. The control loop instance can The control loop definition is read from the Control Loop Run Time Catalogue and values are assigned to the parameters defined for the control loop. The control loop instance is created by instantiating the control loop instance parts in each participant by sending each participant its metadata. Once the control loop is instantiated on each participant, the Control Loop instance is marked as instantiated in the Control Loop Run Time Catalogue. The post condition for an execution of this capability is that the Control Loop instance is running on participants and is processing events.
Control Loop Monitoring. This capability allows control loops to be monitored. Users can check the status of a control loop instances and the status of each participant in a control loop instance. Control loop participants report their overall status and domain status periodically to CLAMP. Clamp aggregates these status reports into an aggregated control loop instance status record, which is available for monitoring. The post condition for an execution of this capability is that control loop instances are being monitored.

Once a control loop definition has been commissioned, instances of the control loop can be created, updated, and deleted. The system manages the lifecycle of control loops following the state transition diagram below.

1: Overall Architecture

The diagram below shows an overview of the architecture of TOSCA based Control Loop management.

The Design Time Catalogue contains the metadata primitives and control loop definition primitives for composition of control loops. As shown in the figure above, the Design Time component provides a system where Control Loops can be designed and defined in metadata. This means that a Control Loop can have any arbitrary structure and the Control Loop developers can use whatever analytic, policy, or control participants they like to implement their Control Loop. At composition time, the user parameterises the Control Loop and stores it in the design time catalogue. This catalogue contains the primitive metadata for any participants that can be used to compose a Control Loop. A Control Loop SDK is used to compose a Control Loop by aggregating the metadata for the participants chosen to be used in a Control Loop and by constructing the references between the participants.

Composed Control Loops are commissioned on the run time part of the system, where they are stored in the run time catalogue and are available for instantiation.

When a user wishes to instantiate a Control Loop, they set values for the parameters of the Control Loop. Once the parameterization has been carried out, the Control Loop instantiated, with the metadata and whatever other artifacts are required being passed to the participants in the Control Loop. At runtime, the Control Loop can be monitored and analysed. It can also be updated as required and can be deleted when it is on longer needed.

The Control Loop Runtime Management will use ONAP services for non-functional aspects such as inventory, topology and data delivery.

1.1: Class Diagrams

1.1.1 Design Time

1.1.2 Runtime

1.2: ERD

2: Control Loop Modelling

Joseph O'Leary to pad out this section

A Control Loop service template is made up of several components, those which represent applications, those which represent dynamic config schemas, and the actual node_templates which makes up the loop itself.

Applications can be a DCAE microservice, an operational policy, or any other application as long as it can be modeled, and the targeted ecosystem to has a participant client waiting for the event distributions from CLAMP via DMaaP Message Router.

Dynamic config on the other hand can be a monitoring policy, or any other resource that provides config to parts of the loop, can be updated after the run time phase has started and is supported by the components hosting the applications in the control loop.

2.1: Control Loop TOSCA file definition

2.1.1 Control Loop Component Definition

A Control Loop Component that can be part of a control loop, it defines the components that partake in a control loop, and are implemented at run time by participants. The control loop component definition is truly dynamic and, as long as the participant that the control loop component definition relates to understands its definition, it can be anything. However, we have designed a base control loop component attribute that's generic and that can act as a good starting point.

Control Loop Node Definition

node_types:
  org.onap.CL_Component:
    properties:
      component_name:
        type: string
        description: Human readable name for the component.
        required: true
      provider:
        type: string
        description: Provider of the component and of the descriptor.
        required: true
      component_version:
        type: string
        description: Software version of the component.
        required: true
      resource_id:
        type: string
        description: >The ID of the resource, 
          should be provided if the resource was uploaded to the entity's inventory already.
        required: false
      resource_content:
        type: string
        description: the contents of the component resource, to be uploaded during commssioning phase of loop.
        required: false
      monitoring_policy:
        type: string
        description: A reference to the monitoring policy if applicable.
        required: false
    version: 0.0.1
    derived_from: tosca.nodes.Root

2.1.2 Loop Definition

The loop definition is implicit in the node_templates within the topology_template, any node contained within is expected to be a part of the control loop managed by CLAMP.

The below example doesn't explicitly include any order, ordering of control loop execution is to be considered in the future which likely would lead to changes to this

Loop Definition

topology_template:
  inputs:
    pm_docker_image:
      type: string
      description: docker image used for instantiating the PMSH service.
      required: true
    operational_policy_name:
      type: string
      description: name of the operational policy to be triggered on MR when subscription is created.
      required: true
    outbound_topic_name:
      type: string
      description: Name of topic used for messages to policy.
      required: true
    feedback_topic_name:
      type: string
      description: Name of topic used for feedback from policy.
      required: true
    inventory_notification_topic_name:
      type: string
      description: Name of topic used for inventory inventory notifications.
      required: true
    pmsh_resource_id:
      type: string
      description: ID of the PMSH blueprint in the inventory.
      required: true

  node_templates:
    org.onap.PM_Subscription_Handler:
      type: org.onap.CL_Component
      properties:
        application_name: PMSH
        provider: DCAE
        application_Version: '1.0.1'
        resource_id: { get_input: pmsh_resource_id }
        monitoring_policy: 'monitoring_policy_id'
        application_config:
          tag_version: { get_input: pm_docker_image }
          operational_policy_name: { get_input: operational_policy_name }
          pmsh_publish_topic_name: { get_input: outbound_topic_name }
          policy_feedback_topic_name: { get_input: feedback_topic_name }
          aai_notification_topic_name: { get_input: inventory_notification_topic_name }

2.1.3 Example

Example Control Loop

tosca_definitions_version: tosca_simple_yaml_1_3
capability_types:
  org.onap.EventProducer:
    properties:
      carrier_protocol_type:
        type: string
        required: true
        constraints:
        - valid_values:
          - DMAAP_message_router
          - SOMETHING_ELSE
          - REST
      data_format:
        type: string
        required: true
        constraints:
        - valid_values:
          - JSON
          - YAML
          - JMS
      event_format:
        type: string
        required: true
      event_format_version:
        type: string
        required: false
      config_keys:
        type: list
        required: false
        entry_schema:
          type: string
          constraints:
          - valid_values:
            - all valid values should be added here
            - if not specified, events of any config key may be generated
            - 'examples for config_key: ves-measurement, ves-syslog, tca_handle_out,
              etc.'
    version: 0.0.1
    derived_from: tosca.capabilities.Root
  org.onap.EventConsumer:
    properties:
      responding_capability:
        type: string
        required: false
      carrier_protocol_type:
        type: string
        required: true
        constraints:
        - valid_values:
          - DMAAP_message_router
          - SOMETHING_ELSE
          - REST
      data_format:
        type: string
        required: true
        constraints:
        - valid_values:
          - JSON
          - YAML
          - JMS
          - all valid values should be added here
      event_format:
        type: string
        description: 'examples for event_format: Ves_specification, LinkUp, VnfConfigured,
          etc.'
        required: true
      event_format_version:
        type: string
        description: 'examples for event_format_version: 5.28.4, 7.30.1, etc.'
        required: false
      config_keys:
        type: list
        required: false
        entry_schema:
          type: string
          constraints:
          - valid_values:
            - all valid values should be added here
            - if not specified, events of any config key may be generated
            - 'examples for config_key: ves-measurement, ves-syslog, tca_handle_out,
              etc.'
    version: 0.0.1
    derived_from: tosca.capabilities.Root
node_types:
  org.onap.DynamicConfig:
    properties:
      application_name:
        type: string
        description: Value used to tie the config to an application ? should we be
          using a relationship here instead?
        required: true
      application_version:
        type: string
        required: true
      application_provider:
        type: string
        required: false
      data_types:
        type: object
        required: false
      schema:
        type: object
        required: false
    version: 0.0.1
    derived_from: tosca.nodes.Root
  org.onap.APP:
    properties:
      application_name:
        type: string
        description: Human readable name for the application Product
        required: false
      provider:
        type: string
        description: Provider of the application and of the descriptor
        required: true
      application_version:
        type: string
        description: Software version of the application
        required: true
      blueprint_id:
        type: string
        description: A reference to the app blueprint
        required: false
      monitoring_policy:
        type: string
        description: A reference to the monitoring policy
        required: false
    requirements:
    - receive:
        capability: org.onap.EventProducer
        relationship: org.onap.PropagateEvent
        occurrences:
        - 0.0
        - UNBOUNDED
    - send:
        capability: org.onap.EventConsumer
        relationship: org.onap.PropagateEvent
        occurrences:
        - 0.0
        - UNBOUNDED
    version: 0.0.1
    derived_from: tosca.nodes.Root
  org.onap.EventRelay:
    properties:
      event_format:
        type: string
        description: 'examples for event_format: Ves_specification, etc.'
        required: true
      event_format_version:
        type: string
        description: 'examples for event_format_version: 5.28.4, 7.30.1, etc.'
        required: true
      config_keys:
        type: list
        required: false
        entry_schema:
          type: string
          constraints:
          - valid_values:
            - all valid values should be added here
            - if not specified, events of any config key is relayed
            - 'examples for config_key: ves-measurement, ves-syslog, tca_handle_out,
              etc.'
      supported_carrier_protocols:
        type: map
        description: 'A map describing supported carrier protocols and translations.
          The tuples define what protocol combinations are supported on the producer
          and consumer side: e.g. { REST: REST, DMAAP: REST, DMAAP: DMAAP}'
        required: true
        key_schema:
          type: string
          constraints:
          - valid_values:
            - DMAAP_message_router
            - SOMETHING_ELSE
            - REST
            - all valid values should be added here
        entry_schema:
          type: string
          constraints:
          - valid_values:
            - DMAAP_message_router
            - SOMETHING_ELSE
            - REST
            - all valid values should be added here
      supported_data_formats:
        type: map
        description: 'Is a map describing supported data formats and translation.
          The tuples define what protocol combinations are supported on the producer
          and consumer side: e.g. { JSON: JSON, JMS: JSON, YAML:YAML }'
        required: true
        key_schema:
          type: string
          constraints:
          - valid_values:
            - JSON
            - JMS
            - YAML
            - etc
            - all valid values should be added here
        entry_schema:
          type: string
          constraints:
          - valid_values:
            - JSON
            - JMS
            - YAML
            - etc
            - all valid values should be added here
    requirements:
    - receive:
        capability: org.onap.EventProducer
        relationship: org.onap.PropagateEvent
        occurrences:
        - 0.0
        - UNBOUNDED
    - send:
        capability: org.onap.EventConsumer
        relationship: org.onap.PropagateEvent
        occurrences:
        - 0.0
        - UNBOUNDED
    version: 0.0.1
    derived_from: tosca.nodes.Root
relationship_types:
  org.onap.PropagateEvent:
    properties:
      config_keys:
        type: list
        description: The relationship type used on requirements to org.onap.EventProducer
          and org.onap.EventConsumer capabilities. Filters events by specific config_keys
          to be transferred by this relationship. That is, any event with a specific
          config_key found in the list is transferred. If list is not defined or is
          empty, events with all config_keys are transferred.
        required: false
        entry_schema:
          type: string
    version: 0.0.1
    derived_from: tosca.relationships.Root
topology_template:
  inputs:
    pm_subscription_topic:
      type: string
    pm_subscription_response_topic:
      type: string
    pm_subscription_handler_blueprint_id:
      type: string
    pm_subscription_operational_policy_id:
      type: string
    pm_subscription_cds_blueprint_id:
      type: string
    enable_tls:
      type: string
  node_templates:
    org.onap.PM_Subscription_Handler:
      type: org.onap.APP
      properties:
        application_name: PM Subscription Handler
        provider: Ericsson
        application_version: 1.0.0
        artifact_id:
          get_input: pm_subscription_handler_blueprint_id
          description: Is this a reference to the DCAE Cloudify Blueprint that is
            already stored(or will be stored before CL configuration & instatiation)
            in DCAE Inventory?
        artifact_config:
          enable_tls:
            get_input: enable_tls
          pmsh_publish_topic_name:
            get_input: pm_subscription_topic
      capabilities:
        pm-subscription-event-publisher:
          properties:
            carrier_protocol_type: DMAAP_message_router
            data_format: JSON
            event_format: pm-subscription-event-format
            event_format_version: 1.0.0
          attributes:
            type: org.onap.EventProducer
          occurrences:
          - 0.0
          - UNBOUNDED
        pm-subscription-event-receiver:
          properties:
            carrier_protocol_type: DMAAP_message_router
            data_format: JSON
            event_format: pm-subscription-event-response-format
            event_format_version: 1.0.0
            relationships:
            - type: tosca.relationships.DependsOn
            - description: any ideas on a better realtionship ? or is it better to
                just use the root realtionship ?
            - target: org.onap.PM_Monitoring_Policy
          attributes:
            type: org.onap.EventConsumer
          occurrences:
          - 0.0
          - UNBOUNDED
    org.onap.PM_Monitoring_Policy:
      type: org.onap.DynamicConfig
      properties:
        application_name: PM Subscription Handler
        application_version: 1.0.0
        provider: Ericsson
        data_types:
          measurementType:
            type: string
          DN:
            type: string
          nfFilter:
            properties:
              nfNames:
                type: list
                entry_schema: string
              modelInvariantIDs:
                type: list
                entry_schema:
                  type: string
              modelVersionIDs:
                type: list
                entry_schema:
                  type: string
          measurementGroup:
            properties:
              masurementTypes:
                type: list
                entry_schema:
                  type: measurementType
              managedObjectDNsBasic:
                type: list
                entry_schema:
                  type: DN
        schema:
          subscription:
            subscriptionName:
              type: string
              required: true
            administrativeState:
              type: string
              required: true
            filebasedGP:
              type: integer
              required: true
            fileLocation:
              type: string
              required: true
            nfFilter:
              type: nfFilter
            measurementGroups:
              type: list
              entry_schema:
                type: measurementGroup
      description: Should I be showing a dependency between PM Subscription Handler
        and the PM Monitoring Policy
    org.onap.PM_Policy:
      type: org.onap.APP
      properties:
        application_name: PM Subscription Operational Policy
        provider: Ericsson
        application_version: 1.0.0
        artifact_id:
          get_input: pm_subscription_operational_policy_id
        artifact_config: NOT_DEFINED
      requirements:
      - receive_0:
          capability: pm-subscription-event-publisher
          node: org.onap.PM_Subscription_Handler
          relationship: NOT_DEFINED
          properties:
            config_keys:
            - topic_name:
                get_input: pm_subscription_topic
      - send_0:
          capability: cds-rest-receive
          node: org.onap.CDS
      - receive_1:
          capability: cds-rest-response
          node: org.onap.CDS
      - send_1:
          capability: pm-subscription-event-receiver
          node: org.onap.PM_Subscription_Handler
          relationship: NOT_DEFINED
          properties:
            config_keys:
            - topic_name:
                get_input: pm_subscription_response_topic
      capabilities:
        pm-subscription-response-event-publisher:
          properties:
            type: org.onap.EventProducer
            carrier_protocol_type: DMAAP_message_router
            data_format: JSON
            event_format: pm-subscription-event-response-format
            event_format_version: 1.0.0
          occurrences:
          - 0.0
          - UNBOUNDED
    org.onap.PM_CDS_Blueprint:
      type: org.onap.APP
      properties:
        application_name: PM Subscription CDS Blueprint
        provider: Ericsson
        application_version: 1.0.0
        artifact_id:
          get_input: pm_subscription_cds_blueprint_id
      capabilities:
        cds-rest-receive:
          properties:
            type: org.onap.EventConsumer
            protocol_type: REST
            data_format: JSON
            event_format: cds_action_format
            event_format_version: 1.0.0
            responding_capability: cds-rest-response
          occurrences:
          - 0.0
          - UNBOUNDED
        cds-rest-response:
          properties:
            type: org.onap.EventProducer
            protocol_type: REST
            data_format: JSON
            event_format: cds_action_response_format
            event_format_version: 1.0.0
          occurrences:
          - 0.0
    org.onap.controlloop0:
      type: org.onap.APP
      properties:
        application_name: Test Control Loop
        provider: Ericsson
        application_version: 1.0.0
        status: NOT_DEPLOYED

2.2: Modelling from TOSCA to Commissioned Data in Run Time Catalogue

2.3: Modelling from TOSCA to Instance Data Run Time Catalogue

3: APIs and Sequence Diagrams

3.1: Commissioning

Ajay Deep Singh to pad out this section

This section defines Commissioning/CRUD Operations that can be performed on ControlLoops.

A Client, in this case CLAMP, can perform CRUD operations or can commission ControlLoops from DesignTime to RunTime Catalogue Database.

DesignTime/RunTime Catalogue Database stores ControlLoop definitions, CRUD operations on database supported by REST Endpoints like Get, Delete, Create allowing selection of a particular ControlLoop to be addressed, below sequence diagram will help you understand flow how a client(Clamp) application can initiate Rest call for performing different operations on Database.

API_Gateway Service is for interacting to different database DesignTime/RunTime and should be responsible for responding success or failure status on different operations.

The commissioning of ControlLoops definition from DesignTime to RunTime Catalogue Database can we achived using the commissioning Rest Endpoint, in this process when a rest request is initiated from a client(Clamp) the API_Gateway Service take cares of fetching ControlLoops metadata from DesignTime and creates in RunTime Catalogue Database, Commissioning API ControlLoop Sequence diagram will help you understand the flow.

In future commissioning Rest Endpoint might be updated to push ControlLoops not only in RunTime Database but to the participants involved in ControlLoop.

3.1.1: Commissioning REST API

Initial Swagger Draft for Deployment API

openapi: 3.0.3
info:
  description: >-
    The ControlLoop Commissioning API allows users to search and perform
    CRUD action on Controlloop Tosca exported data sets.
  version: 1.0.0
  title: Control Loop Commissioning API
  license:
    name: Apache 2.0
    url: 'http://www.apache.org/licenses/LICENSE-2.0'
  contact:
    email: lego@est.tech
servers:
  - url: '{scheme}://onap/controlloop/v2/'
    variables:
      scheme:
        description: 'The Data Set API is accessible via https and http'
        enum:
          - 'https'
          - 'http'
        default: 'http'
tags:
  - name: Control Loop 
    description: Commissioning API
  - name: design-catalogue
    description: CRUD actions on an designtime Control Loop data sets.
  - name: runtime-catalogue
    description: CRUD actions on an runtime Control Loop data sets.
paths:
  /design-catalogue/loops:
    get:
      tags:
        - design-catalogue
      operationId: getAllLoopsUsingGET
      summary: Returns a list of all available Control Loops.
      responses:
        '200':
          description: OK
          content:
            application/yaml:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ControlLoop'
                  
  /design-catalogue/loop/{controlLoopId}:
    get:
      tags:
        - design-catalogue
      summary: >-
        Returns a specific control loop.
      description: >-
        This GET API returns a specific control loop.
      operationId: getLoopByIdUsingGET
      parameters:
        - name: controlLoopId
          in: path
          description: 'Id of the control loop.'
          required: true
          example: "org.onap.PM_CDS_Blueprint"
          schema:
            type: string
      responses:
        '200':
          description: OK
          content:
            application/yaml:
              schema:
                $ref: '#/components/schemas/ControlLoop'
  
    delete:
      tags:
        - design-catalogue
      summary: Delete a specific Control Loop by ID.
      operationId: deleteLoopByIdUsingDELETE
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not present

    post:
      tags:
        - design-catalogue
      summary: >-
        Accepts a Tosca Service Template representing a Control Loop.
      description: >-
        This API Create a Control Loop.
      operationId: createLoopWithIDUsingPOST
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '409':
          description: Control Loop already exist.
     
    put:
      tags:
        - design-catalogue
      summary: Updates a Control Loop.
      operationId: updateLoopWithIDUsingPUT
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not found
          
  /runtime-catalogue/commission:
    post:
      tags:
        - runtime-catalogue
      summary: >-
        Commissions Tosca Service Template in RunTime Catalogue.
      description: >-
        This API commissions Tosca Service Template in RunTime Catalogue DB.
      operationId: commissionLoopWithPOST
      responses:
        '200':
          description: OK
        '404':
          description: No matching record found for the given criteria.
          
  /runtime-catalogue/loop/{controlLoopId}:
    get:
      tags:
        - runtime-catalogue
      summary: >-
        Returns a specific control loop.
      description: >-
        This GET API returns a specific control loop.
      operationId: getLoopByIdUsing
      parameters:
        - name: controlLoopId
          in: path
          description: 'Id of the control loop.'
          required: true
          example: "org.onap.PM_CDS_Blueprint"
          schema:
            type: string
      responses:
        '200':
          description: OK
          content:
            application/yaml:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ControlLoop'
  
    delete:
      tags:
        - runtime-catalogue
      summary: Delete a specific Control Loop by ID.
      operationId: deleteLoopByIdUsing
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not present

    post:
      tags:
        - runtime-catalogue
      summary: >-
        Accepts a Tosca Service Template representing a Control Loop.
      description: >-
        This API Create a Control Loop.
      operationId: createLoopWithIDUsing
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '409':
          description: Control Loop already exist.
     
    put:
      tags:
        - runtime-catalogue
      summary: Updates a Control Loop.
      operationId: updateLoopWithIDUsing
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not found

components:
  schemas:
    ControlLoop:
      title: ControlLoop
      type: object
      properties:
        id:
          type: string
        ControlLoop:
          type: string

3.1.2: Commissioning Sequence Diagrams

GET, DELETE, CREATE API ControlLoop Sequence Diagram
Commission API ControlLoop Sequence Diagram

3.2: Instantiation

Robertas Rimkus to pad out this section

This section refers to Instantiation of a Commissioned control loop. A client, in this case CLAMP (potentially DCAEMOD, etc in the future) will render the commissioned control loops allowing selection of a particular control loop to be instantiated. User will then provide the configurations needed to instantiate the selected control loop which will be sent onto the CL_Instance_Control Service. The service will then distribute the configurations to DMaaP topic. Participants (agents) will pull the event containing the config and pick out their control loop components to be instantiated and start/set up those particular components. CL_Instance_Control Service will be waiting for a response back from all participants involved in the instantiation of the control loop, in regards to the state of instantiation. In successful response case the service will store the CL Instance LCM (Life Cycle management) data into the runtime DB as well as providing a message back to the client of the successful instantiation. In failure to receive the response case, a timeout will be called, which will result in a teardown event being sent to DMaaP. The participants will then receive the event and proceed to teardown the components that were instantiated or check that they have failed to instantiate in the first place and send a Teardown ACK back to the CL_Instance_Control Service. No CL Instance LCM data will be stored and a message indicating failure to instantiate the CL along side with the error will be sent back to the client (CLAMP).

3.2.1: Instantiation REST API

Initial Swagger for Initiation API

openapi: 3.0.3
info:
  title: Swagger Control Loop Instantiation
  description: ""
  version: 1.0.0
servers:
  - url: '{scheme}://onap/controlloop/v2/'
    variables:
      scheme:
        description: 'The Data Set API is accessible via https and http'
        enum:
          - 'https'
          - 'http'
        default: 'http'
tags:
- name: instantiate
  description: Control Loop Instantiation
paths:
  /runtime-catalogue/instantiate:
    post:
      tags:
      - instantiate
      summary: Instantiate a commissioned control loop
      operationId: instantiateLoop
      requestBody:
        description: Control loop that is to be instantiated
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ControlLoop'
        required: true
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ApiResponse'
        404:
          description: No Control Loop found
        405:
          description: Invalid input
  /runtime-catalogue/loops/instantiated:
    get:
      tags:
      - instantiate
      summary: Get a list of instantiated control loops
      responses:
        200:
          description: Success
          content:            
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ControlLoop'
        404:
          description: No instantiated control loops found
  /runtime-catalogue/loops/instantiated/{loop-id}:
    get:
      tags:
      - instantiate
      summary: Get an instantiated control loop by id
      parameters:
      - name: loop-id
        in: path
        description: ID of instantiated loop to return
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/ControlLoop'
        404:
          description: Control Loop not found
    put:
      tags:
      - instantiate
      summary: Update an instantiated control loop
      parameters:
      - name: loop-id
        in: path
        description: ID of instantiated loop
        required: true
        schema:
          type: string
      requestBody:
        description: Control loop to be updated
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ControlLoop'
        required: true
        
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/ControlLoop'
        404:
          description: Control Loop not found
    delete:
      tags:
      - instantiate
      summary: Delete an instantiated control loop
      parameters:
      - name: loop-id
        in: path
        description: ID of instantiated loop
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
        404:
          description: Control Loop not found
components:
  schemas:
    ControlLoop:
      title: ControlLoop
      type: object
      properties:
        id:
          type: string
        controlLoopTosca:
          type: string
    ApiResponse:
      type: object
      properties:
        code:
          type: integer
          format: int32
        status:
          type: string

3.2.2: Instantiation Sequence Diagrams

3.2.3: Instantiation DMaaP API

Initial Thought for an event to be sent from CL_Instance_Control onto DMaaP for Participants to consume. The event would go onto an output topic which the Participants would be polling/subscribed to

e.g url : https://{{ONAPIP}}:{{DMaaPPort}}/events/CL_INSTANCE_CONTROL_OUTPUT

CL_Instance_Control Event to DmaaP

{
  "instance-id":"myCLInstance1",
  "action":"INSTANTIATE",
  "configurations":[
    {
      "participant-id":"Participant_DCAE",
      "applications":[
        {
          "application-id":"example.pmsh",
          "config":{
            
          }
        },
        {
          "application-id":"example.dfc",
          "config":{
            
          }
        }
      ]
    },
    {
      "participant-id":"Participant_Policy",
      "applications":[
        {
          "application-id":"example.OperationalPolicy",
          "config":{
            
          }
        }
      ]
    }
  ]
}

*Preferred solution is to send TOSCA in the body. Meaning we could reuse the parsing code which is already present and provide it to the participant.

3.2.4: Instantiation Participant API

*Suggestion was to put JAVA API code in this section for the participant talking to DMaaP. TBD

3.3: Monitoring

In this case it refers to monitoring the data that the participants will provide to DMaaP. Participants will send events to DMaaP which will be pulled by the CL_Monitoring_Service. The data provided should include a reference id to the control loops that are instantiated on the participant, as well as the applications that have been instantiated as a part of that control loop for that participant. Data should also include the time that the application has started, state of it (running/terminated) and any other critical information which would help to determine the health of an instantiated control loop and its components. Idea is for the participant to provide events every certain period of time, similar to a health-check, in order to provide consistent monitoring.

3.3.1: Monitoring REST API

Initial Swagger for Monitoring API

openapi: 3.0.3
info:
  title: Swagger Control Loop Monitoring
  description: ""
  version: 1.0.0
servers:
  - url: '{scheme}://onap/controlloop/v2/'
    variables:
      scheme:
        description: 'The Data Set API is accessible via https and http'
        enum:
          - 'https'
          - 'http'
        default: 'http'
tags:
- name: monitoring
  description: Control Loop Monitoring
paths:
  /runtime-catalogue/monitoring:
    post:
      tags:
      - monitoring
      summary: Input monitoring data into the database
      requestBody:
        description: Monitoring data to be input
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/MonitoringData'
        required: true
      responses:
        200:
          description: OK
        405:
          description: Invalid input
    get:
      tags:
      - monitoring
      summary: Get monitoring data for all participants
      responses:
        200:
          description: Success
          content:            
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/MonitoringData'
        404:
          description: No monitoring data found
  /runtime-catalogue/monitoring/{participant-id}:
    get:
      tags:
      - monitoring
      summary: Get monitoring data for a participant by participant id
      parameters:
      - name: participant-id
        in: path
        description: ID of a participant 
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/MonitoringData'
        404:
          description: Monitoring data for the participant not found
    put:
      tags:
      - monitoring
      summary: Update monitoring data for a participant by participant id
      parameters:
      - name: participant-id
        in: path
        description: ID of a participant
        required: true
        schema:
          type: string
      requestBody:
        description: Monitoring data to be updated
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/MonitoringData'
        required: true
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/MonitoringData'
        404:
          description: Monitoring data for the participant not found
    delete:
      tags:
      - monitoring
      summary: Delete monitoring data for a participant by participant id
      parameters:
      - name: participant-id
        in: path
        description: ID of a participant
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
        404:
          description: Monitoring data for the participant not found
components:
  schemas:
    MonitoringData:
      properties:
        participant-id:
          type: string
        control-loops:
          type: array
          items:
            properties:
              cl-instance-id:
                type: string
              applications:
                type: array
                items:
                  properties:
                    application-id:
                      type: string
                    status:
                      type: string
                      enum:
                        - INSTANTIATION_STARTED
                        - INSTANTIATED_RUNNING
                        - INSTANTIATED_FAILED
                    other-data:
                      type: string

3.3.2: Monitoring Sequence Diagrams

3.3.3: Monitoring DMaaP API

Participants will send an event containing monitoring data to a DMaaP topic at a set interval after participant has received an event to instantiate a control loop

e.g url: https://{{ONAPIP}}:{{DMaaPPort}}/events/CL_MONITORING_SERVICE_INPUT

Example event from Participant to DmaaP

{
  "timestamp":"1605689254",
  "participant-id":"Participant_DCAE",
  "control-loops":[
    {
      "instance-id":"myCLInstance1",
      "applications":[
        {
          "application-id":"example.pmsh",
          "status":"INSTANTIATION_STARTED"
        },
        {
          "application-id":"example.dfc",
          "status":"INSTANTIATION_STARTED"
        }
      ]
    },
    {
      "instance-id":"myCLInstance2",
      "applications":[
        {
          "application-id":"example.pmsh",
          "status":"INSTANTIATED"
        }
      ]
    }
  ]
}

3.3.4: Monitoring Participant API

Presume similar thinking to Instantiation Participant API

*Suggestion was to put JAVA API code in this section for the participant talking to DMaaP. TBD

3.4: Supervision

Supervision is responsible for ensuring that

control loops are established once their initiation has been ordered
control loops are running correctly once their initiation is completed
control loops are correctly removed once their removal has been ordered

3.4.1: Supervision Sequence Diagrams

3.4.2: Supervision APIs to other components

4: Design

4.1: Server Side

4.1.1 Database Schema and JPA

4.1.2: TOSCA Processing

4.1.3: Instance Control

4.1.4: Execution Monitoring

4.2: Participant Side

4.2.1: Participant Agent

4.2.2: Policy Participant Agent

4.2.3: CDS Participant Agent

4.2.4: DCAE Participant Agent

4.2.5: SO Participant Agent

4.3: Client Side

4.3.1: Client SDK: Composition of Control Loop Tosca

4.3.2: Client User Interface

4.4 Other Considerations

4.4.1 Upgrade

Performing a hot upgrade of the Control Loop at run time as well as handling an upgrade of the software in one or more of the participants in an Control Loop is a particularly challenging issue because upgrading must handle the following cases without tearing down the Control Loop:

Upgrade and changes of the configuration data of participants
Addition of or removal of participants in an Control Loop
Upgrade of software in one or more participants in an Control Loop
Maintenance of compatibility between participants when an update of more than one participant must be done together to ensure compatibility, for example, when a protocol being used by two participants to communicate is upgraded

5: Goals

5.1: MVP

key	summary	assignee	priority	status

Loading...

Refresh

Design Time
- Support design of multiple control loops*
- Support design of individual control loop component**
- Support composition of control loops**
Runtime
- Participant registration and participant deregistration
- Support commissioning of control loops
  - Ingestion with artifact references*
  - Ingestion with artifact embedded**
- Support instantiation of control loop
  - Support instantiation of control loop TOSCA to DMaaP MR*
  - Support instantiation of config for the control loop*
- Support monitoring of control loops
  - Receive control loop heartbeat events (heartbeat starts when component of control loop is running)*
- Support supervision of control loops
  - Periodically check monitored data, and update state of control loop*
Participants
- Agent library*
- Reference(test) participant*
- CDS participant*
- DCAE participant*
- Policy participant*
Demo**
- Throwaway Monitoring/Control GUI

TOSCA Defined Control Loop: Architecture and Design