This page is Work in Progress

The idea of using control loops to automatically (or autonomously) perform network management has been the subject of much research in the Network Management research community, see this paper for some background. However, it is only with the advent of ONAP that we have a platform that supports control loops for network management. Before ONAP, Control Loops have been implemented by hard-coding components together and hard coding logic into components. ONAP has taken a step forward towards automatic implementation of Control Loops by allowing parameterization of Control Loops that work on the premise that the Control Loops use a set of analytic, policy, and control components connected together in set ways.

The goal of the work is to extend and enhance the current ONAP Control Loop support to provide a complete open-source framework for Control Loops. This will enhance the current support to provide TOSCA based Control Loop definition and development, commissioning and run-time management. The participants that comprise a Control Loop and the metadata needed to link the participants together to create a Control Loop are specified in a standardized way using the OASIS TOSCA modelling language. The TOSCA description is then used to commission, instantiate, and manage the Control Loops in the run time system.

1. Terminology

Control Loop: A set of executing elements on which Life Cycle Management (LCM) is executed collectively. For example, a set of microservices may be spawned and executed together to deliver a service. This collection of services is a control loop

Control Loop Element: A single executing entitity, with its Life Cycle being managed as part of the overall control loop. For example, a single microservice that is executing as one microservice in a service.

Participant: A system or framework that runs Control Loop Elements. A participant chooses to partake in control loops, to manage Control Loop Elements for CLAMP, and to receive, send and act on LCM messages for the CLAMP runtime.

Control Loop Type: A definition of a Control Loop in the TOSCA language. This definition describes a certain type of a control loop. The life cycle of instances of a Control Loop Type are managed by CLAMP.

Control Loop Instance: An instance of a Control Loop Type. The life cycle of a Control Loop Instance is managed by CLAMP.

Property: Metadata defined in TOSCA that is associated with a Control Loop, a Control Loop Element, or a Participant.

Property Type: The TOSCA defintion of the type of a property. A property can have a generic type such as string or integer or can have a user defined TOSCA data type.

Property Value: The value of a Property Type. Property values are assigned at run time in CLAMP.

Common Property Type: Property Types that apply to all instances of a Control Loop Type.

Common Property Value: The value of a Property Type. It is assigned at run time once for all instances of a Control Loop Type.

Instance Specific Property Type: Property Types that apply to an individual instance of a Control Loop Type.

Instance Property Value: The value of a Property Type that applies to an individual instance of a Control Loop Type. The value is assigned at run time for each control loop instance.

2. Capabilities

We consider the capabilities of Control Loops at Design Time and Run Time.

At Design Time, two capabilities are supported:

Participant Metadata Definition. This capability allows external users and systems (such as SDC or DCAE-MOD) to define participants that can take part in a control loop and to define the metadata that can be used on and configured on a participant when it is taking part in in a control loop. The post condition of an execution of this capability is that a participant is defined in the Control Loop Design Time Catalogue together with sets of metadata that can be used with this participant in control loops.
Control Loop Composition. This capability allows users and other systems to create Control Loop Type definitions by connecting a chain of participants together from the participants that are available in the Control Loop Design Time Catalogue. In an execution of this capability, a user defines the control loop chain for the Control Loop Type and defines the connections between participants. The user also selects the correct metadata sets for each participant in the Control Loop Type and defines the overall Control Loop Type metadata. The user also specifies the Common Property Types that apply to all instances of a control loop type and the Instance Specific Property Types that apply to individual instances of a Control Loop Type. The post condition for an execution of this capability is a Control Loop definition in TOSCA stored in the Control Loop Design Time Catalogue.

At Run Time, the following capabilities are supported:

Participant Registration. This capability allows participants to register and deregister with CLAMP. The post condition for an execution of this capability is that a participant is available for participation in a control loop. Participants can explicitly register with CLAMP at any point after they start, or they can implicitly register when they respond to a control loop initiation request.
Control Loop Commissioning. This capability allows version controlled Control Loop Type definitions to be taken from the Control Loop Design Time Catalogue and be placed in the Control Loop Run Time Inventory. It also allows the values of Common Property Types that apply to all instances of a Control Loop Type to be set. The post condition for an execution of this capability is that the Control Loop definition is in the Control Loop Run Time Inventory.
Control Loop Priming on Participants. A participant is primed to support a Control Loop Type. The definition of a control loop and the values of Common Property Types that apply to all instances of a control loop type on a participant is sent to a participant. The participant can then take whatever actions it need to do to support the control loop type in question. Control Loop Priming can take place explicitly as a separate operation on participants or can be done implicitly in the instantiation message for the first instance of a Control Loop Type. The post condition for an execution of this capability is that all participants in this control loop type are commissioned, that is they are prepared to run instances of this control loop type.
Control Loop Life Cycle Management. This capability allows a Control Loop Instance to be created. The Control Loop Type definition is read from the Control Loop Run Time Inventory and values are assigned to the Instance Specific Property Types defined for instances of the Control Loop Type in the same manner as the existing CLAMP client does. A Control Loop Instance that has been created but has not yet been instantiated on participants is in state UNINITIALIZED. In this state, the Instance Specific Property Type values can be revised and updated as often as the user requires. Once the user is happy with the property values, the Control Loop Instance is instantiated on participants and the Control Loop Elements for this Control Loop Instance are started by participants using the control loop metadata. Once the Control Loop Instance is instantiated on each participant, the Control Loop Instance is set as being in state PASSIVE in the Control Loop Run Time Inventory. The user can now order the participants to change the state of the Control Loop Instance to state RUNNING. Each participant begins accepting and processing control loop events and the Control Loop Instance is set to state RUNNING in the control loop inventory. The post condition for an execution of this capability is that the Control Loop instance is running on participants and is processing events.
Control Loop Monitoring. This capability allows Control Loop Instances to be monitored. Users can check the status of Participants, Control Loop Instances, and Control Loop Elements. Participants report their overall status and the status of Control Loop Elements they are running periodically to CLAMP. Clamp aggregates these status reports into an aggregated Control Loop Instance status record, which is available for monitoring. The post condition for an execution of this capability is that Control Loop Instances are being monitored.

3. Life Cycle State Management

When a control loop definition has been commissioned, instances of the control loop can be created, updated, and deleted. The system manages the lifecycle of control loops and control loop elements following the state transition diagram below.

This page is updated for Istanbul to this point, the information below this point may or may not be correct for Istanbul.

1: Overall Architecture

The diagram below shows an overview of the architecture of TOSCA based Control Loop management.

The Design Time Catalogue contains the metadata primitives and control loop definition primitives for composition of control loops. As shown in the figure above, the Design Time component provides a system where Control Loops can be designed and defined in metadata. This means that a Control Loop can have any arbitrary structure and the Control Loop developers can use whatever analytic, policy, or control participants they like to implement their Control Loop. At composition time, the user parameterises the Control Loop and stores it in the design time catalogue. This catalogue contains the primitive metadata for any participants that can be used to compose a Control Loop. A Control Loop SDK is used to compose a Control Loop by aggregating the metadata for the participants chosen to be used in a Control Loop and by constructing the references between the participants.

Composed Control Loops are commissioned on the run time part of the system, where they are stored in the run time inventory and are available for instantiation.

When a user wishes to instantiate a Control Loop, they set values for the parameters of the Control Loop. Once the parameterization has been carried out, the Control Loop instantiated, with the metadata and whatever other artifacts are required being passed to the participants in the Control Loop. At runtime, the Control Loop can be monitored and analysed. It can also be updated as required and can be deleted when it is on longer needed.

The Control Loop Runtime Management will use ONAP services for non-functional aspects such as inventory, topology and data delivery.

1.1: Class Diagrams

1.1.1 Design Time

1.1.2 Runtime

Comment

1.2: ERD

2: Control Loop Modelling

Joseph O'Leary to pad out this section

A Control Loop service template is made up of several components, those which represent applications, those which represent dynamic config schemas, and the actual node_templates which makes up the loop itself.

Applications can be a DCAE microservice, an operational policy, or any other application as long as it can be modeled, and the targeted ecosystem to has a participant client waiting for the event distributions from CLAMP via DMaaP Message Router.

Dynamic config on the other hand can be a monitoring policy, or any other resource that provides config to parts of the loop, can be updated after the run time phase has started and is supported by the components hosting the applications in the control loop.

2.1: Control Loop TOSCA file definition

2.1.1 Control Loop Component Definition

A Control Loop Component that can be part of a control loop, it defines the components that partake in a control loop, and are implemented at run time by participants. The control loop component definition is truly dynamic and, as long as the participant that the control loop component definition relates to understands its definition, it can be anything. However, we have designed a base control loop component attribute that's generic and that can act as a good starting point.

node_types:
  org.onap.CL_Component:
    properties:
      component_name:
        type: string
        description: Human readable name for the component.
        required: true
      provider:
        type: string
        description: Provider of the component and of the descriptor.
        required: true
      component_version:
        type: string
        description: Software version of the component.
        required: true
      resource_id:
        type: string
        description: >The ID of the resource, 
          should be provided if the resource was uploaded to the entity's inventory already.
        required: false
      resource_content:
        type: string
        description: the contents of the component resource, to be uploaded during commssioning phase of loop.
        required: false
      monitoring_policy:
        type: string
        description: A reference to the monitoring policy if applicable.
        required: false
    version: 0.0.1
    derived_from: tosca.nodes.Root

2.1.2 Loop Definition

The loop definition is explicit in the node_templates within the topology_template, a Control Loop node template is specified and any node tempalte specified in the Control Loop node tepolcate is part of the control loop managed by CLAMP.

The below example doesn't explicitly include any order, ordering of control loop execution is to be considered in the future which likely would lead to changes to this

tosca_definitions_version: tosca_simple_yaml_1_3
data_types:
  onap.datatypes.ToscaConceptIdentifier:
    derived_from: tosca.datatypes.Root
    properties:
      name:
        type: string
        required: true
      version:
        type: string
        required: true
node_types:
  org.onap.policy.clamp.controlloop.Participant:
    version: 1.0.1
    derived_from: tosca.nodetypes.Root
    properties:
      provider:
        type: string
        requred: false
  org.onap.policy.clamp.controlloop.ControlLoopElement:
    version: 1.0.1
    derived_from: tosca.nodetypes.Root
    properties:
      provider:
        type: string
        requred: false
      participant_id:
        type: onap.datatypes.ToscaConceptIdentifier
        requred: true
  org.onap.policy.clamp.controlloop.ControlLoop:
    version: 1.0.1
    derived_from: tosca.nodetypes.Root
    properties:
      provider:
        type: string
        requred: false
      elements:
        type: list
        required: true
        entry_schema:
          type: onap.datatypes.ToscaConceptIdentifier
  org.onap.policy.clamp.controlloop.DCAEMicroserviceControlLoopElement:
    version: 1.0.1
    derived_from: org.onap.policy.clamp.controlloop.ControlLoopElement
    properties:
      dcae_blueprint_id:
        type: onap.datatypes.ToscaConceptIdentifier
        requred: true
  org.onap.policy.clamp.controlloop.PolicyTypeControlLoopElement:
    version: 1.0.1
    derived_from: org.onap.policy.clamp.controlloop.ControlLoopElement
    properties:
      policy_type_id:
        type: onap.datatypes.ToscaConceptIdentifier
        requred: true
  org.onap.policy.clamp.controlloop.CDSControlLoopElement:
    version: 1.0.1
    derived_from: org.onap.policy.clamp.controlloop.ControlLoopElement
    properties:
      cds_blueprint_id:
        type: onap.datatypes.ToscaConceptIdentifier
        requred: true
topology_template:
  node_templates:
    org.onap.dcae.controlloop.DCAEMicroserviceControlLoopParticipant:
      version: 2.3.4
      type: org.onap.policy.clamp.controlloop.Participant
      type_version: 1.0.1
      description: Participant for DCAE microservices
      properties:
        provider: ONAP
    org.onap.policy.controlloop.MonitoringPolicyControlLoopParticipant:
      version: 2.3.1
      type: org.onap.policy.clamp.controlloop.Participant
      type_version: 1.0.1
      description: Participant for DCAE microservices
      properties:
        provider: ONAP
    org.onap.policy.controlloop.OperationalPolicyControlLoopParticipant:
      version: 3.2.1
      type: org.onap.policy.clamp.controlloop.Participant
      type_version: 1.0.1
      description: Participant for DCAE microservices
      properties:
        provider: ONAP
    org.onap.ccsdk.cds.controlloop.CdsControlLoopParticipant:
      version: 2.2.1
      type: org.onap.policy.clamp.controlloop.Participant
      type_version: 1.0.1
      description: Participant for DCAE microservices
      properties:
        provider: ONAP
    org.onap.domain.pmsh.PMSH_DCAEMicroservice:
      version: 1.2.3
      type: org.onap.policy.clamp.controlloop.DCAEMicroserviceControlLoopElement
      type_version: 1.0.0
      description: Control loop element for the DCAE microservice for Performance Management Subscription Handling
      properties:
        provider: Ericsson
        participant_id:
          name: org.onap.dcae.controlloop.DCAEMicroserviceControlLoopParticipant
          version: 2.3.4
        dcae_blueprint_id:
          name: org.onap.dcae.blueprints.PMSHBlueprint
          version: 1.0.0
    org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement:
      version: 1.2.3
      type: org.onap.policy.clamp.controlloop.PolicyTypeControlLoopElement
      type_version: 1.0.0
      description: Control loop element for the monitoring policy for Performance Management Subscription Handling
      properties:
        provider: Ericsson
        participant_id:
          name: org.onap.policy.controlloop.PolicyControlLoopParticipant
          version: 2.3.1
        policy_type_id:
          name: onap.policies.monitoring.pm-subscription-handler
          version: 1.0.0
    org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement:
      version: 1.2.3
      type: org.onap.policy.clamp.controlloop.PolicyTypeControlLoopElement
      type_version: 1.0.0
      description: Control loop element for the operational policy for Performance Management Subscription Handling
      properties:
        provider: Ericsson
        participant_id:
          name: org.onap.policy.controlloop.PolicyControlLoopParticipant
          version: 2.3.1
        policy_type_id:
          name: onap.policies.operational.pm-subscription-handler
          version: 1.0.0
    org.onap.domain.pmsh.PMSH_CDS_ControlLoopElement:
      version: 1.2.3
      type: org.onap.policy.clamp.controlloop.ControlLoopElement
      type_version: 1.0.0
      description: Control loop element for CDS for Performance Management Subscription Handling
      properties:
        provider: Ericsson
        participant_Id:
          name: org.onap.ccsdk.cds.controlloop.CdsControlLoopParticipant
          version: 3.2.1
        cds_blueprint_id:
          name: org.onap.ccsdk.cds.PMSHCdsBlueprint
          version: 1.0.0
    org.onap.domain.pmsh.PMSHControlLoopDefinition:
      version: 1.2.3
      type: org.onap.policy.clamp.controlloop.ControlLoop
      type_version: 1.0.0
      description: Control loop for Performance Management Subscription Handling
      properties:
        provider: Ericsson
        elements:
        - name: org.onap.domain.pmsh.PMSH_DCAEMicroservice
          version: 1.2.3
        - name: org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement
          version: 1.2.3
        - name: org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement
          version: 1.2.3
        - name: org.onap.domain.pmsh.PMSH_CDS_ControlLoopElement
          version: 1.2.3

2.1.3 Example of two Control Loop instances

{
    "controlLoops": [
        {
            "name": "PMSHInstance0",
            "version": "1.0.1",
            "definition": {
                "name": "org.onap.domain.pmsh.PMSHControlLoopDefinition",
                "version": "1.0.0"
            },
            "state": "UNINITIALISED",
            "orderedState": "UNINITIALISED",
            "description": "PMSH control loop instance 0",
            "elements": [
                {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c20",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_DCAEMicroservice",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "DCAEParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "DCAE Control Loop Element for the PMSH instance 0 control loop"
                },
                {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c21",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "PolicyParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "Monitoring Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c22",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "PolicyParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "Operational Policy Control Loop Element for the PMSH instance 0 control loop"
                },
                {
                    "id": "709c62b3-8918-41b9-a747-d21eb79c6c23",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_CDS_ControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "CDSParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "CDS Control Loop Element for the PMSH instance 0 control loop"
                }
            ]
        },
        {
            "name": "PMSHInstance1",
            "version": "1.0.1",
            "definition": {
                "name": "org.onap.domain.pmsh.PMSHControlLoopDefinition",
                "version": "1.0.0"
            },
            "state": "UNINITIALISED",
            "orderedState": "UNINITIALISED",
            "description": "PMSH control loop instance 1",
            "elements": [
                {
                    "id": "709c62b3-8918-41b9-a747-e21eb79c6c24",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_DCAEMicroservice",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "DCAEParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "DCAE Control Loop Element for the PMSH instance 1 control loop"
                },
                {
                    "id": "709c62b3-8918-41b9-a747-e21eb79c6c25",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_MonitoringPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "PolicyParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "Monitoring Policy Control Loop Element for the PMSH instance 1 control loop"
                },
                {
                    "id": "709c62b3-8918-41b9-a747-e21eb79c6c26",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_OperationalPolicyControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "PolicyParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "Operational Policy Control Loop Element for the PMSH instance 1 control loop"
                },
                {
                    "id": "709c62b3-8918-41b9-a747-e21eb79c6c27",
                    "definition": {
                        "name": "org.onap.domain.pmsh.PMSH_CDS_ControlLoopElement",
                        "version": "1.2.3"
                    },
                    "participantId": {
                        "name": "CDSParticipant0",
                        "version": "1.0.0"
                    },
                    "state": "UNINITIALISED",
                    "orderedState": "UNINITIALISED",
                    "description": "CDS Control Loop Element for the PMSH instance 1 control loop"
                }
            ]
        }
    ]
}

2.2: Modelling from TOSCA to Commissioned Data in Run Time Inventory

2.3: Modelling from TOSCA to Instance Data Run Time Inventory

2.4: Swagger REST APIs for Control Loop

ControlLoop Runtime Swagger REST APIs:

ControlLoop_Runtime_Swagger_API.yml

Participant Swagger REST APIs:

Participant_Swagger_API.yml

3: APIs and Sequence Diagrams

3.1: Commissioning

Ajay Deep Singh to pad out this section

This section defines Commissioning/CRUD Operations that can be performed on ControlLoops.

A Client, in this case CLAMP, can perform CRUD operations or can commission ControlLoops from DesignTime to RunTime Inventory Database.

DesignTime/RunTime Catalogue/Inventory Database stores ControlLoop definitions, CRUD operations on database supported by REST Endpoints like Get, Delete, Create allowing selection of a particular ControlLoop to be addressed, below sequence diagram will help you understand flow how a client(Clamp) application can initiate Rest call for performing different operations on Database.

API_Gateway Service is for interacting to different database DesignTime/RunTime and should be responsible for responding success or failure status on different operations.

The commissioning of ControlLoops definition from DesignTime Catalogue to RunTime Inventory Database can we achived using the commissioning Rest Endpoint, in this process when a rest request is initiated from a client(Clamp) the API_Gateway Service take cares of fetching ControlLoops metadata from DesignTime and creates in RunTime Inventory Database, Commissioning API ControlLoop Sequence diagram will help you understand the flow.

In future commissioning Rest Endpoint might be updated to push ControlLoops not only in RunTime Database but to the participants involved in ControlLoop.

3.1.1: Commissioning REST API

openapi: 3.0.3
info:
  description: >-
    The ControlLoop Commissioning API allows users to search and perform
    CRUD action on Controlloop Tosca exported data sets.
  version: 1.0.0
  title: Control Loop Commissioning API
  license:
    name: Apache 2.0
    url: 'http://www.apache.org/licenses/LICENSE-2.0'
  contact:
    email: lego@est.tech
servers:
  - url: '{scheme}://onap/controlloop/v2/'
    variables:
      scheme:
        description: 'The Data Set API is accessible via https and http'
        enum:
          - 'https'
          - 'http'
        default: 'http'
tags:
  - name: Control Loop 
    description: Commissioning API
  - name: design-catalogue
    description: CRUD actions on an designtime Control Loop data sets.
  - name: runtime-inventory
    description: CRUD actions on an runtime Control Loop data sets.
paths:
  /design-catalogue/loops:
    get:
      tags:
        - design-catalogue
      operationId: getAllLoopsUsingGET
      summary: Returns a list of all available Control Loops.
      responses:
        '200':
          description: OK
          content:
            application/yaml:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ControlLoop'
                  
  /design-catalogue/loop/{controlLoopId}:
    get:
      tags:
        - design-catalogue
      summary: >-
        Returns a specific control loop.
      description: >-
        This GET API returns a specific control loop.
      operationId: getLoopByIdUsingGET
      parameters:
        - name: controlLoopId
          in: path
          description: 'Id of the control loop.'
          required: true
          example: "org.onap.PM_CDS_Blueprint"
          schema:
            type: string
      responses:
        '200':
          description: OK
          content:
            application/yaml:
              schema:
                $ref: '#/components/schemas/ControlLoop'
  
    delete:
      tags:
        - design-catalogue
      summary: Delete a specific Control Loop by ID.
      operationId: deleteLoopByIdUsingDELETE
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not present

    post:
      tags:
        - design-catalogue
      summary: >-
        Accepts a Tosca Service Template representing a Control Loop.
      description: >-
        This API Create a Control Loop.
      operationId: createLoopWithIDUsingPOST
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '409':
          description: Control Loop already exist.
     
    put:
      tags:
        - design-catalogue
      summary: Updates a Control Loop.
      operationId: updateLoopWithIDUsingPUT
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not found
          
  /runtime-inventory/commission:
    post:
      tags:
        - runtime-inventory
      summary: >-
        Commissions Tosca Service Template in RunTime Inventory.
      description: >-
        This API commissions Tosca Service Template in RunTime Inventory DB.
      operationId: commissionLoopWithPOST
      responses:
        '200':
          description: OK
        '404':
          description: No matching record found for the given criteria.
          
  /runtime-inventory/loop/{controlLoopId}:
    get:
      tags:
        - runtime-inventory
      summary: >-
        Returns a specific control loop.
      description: >-
        This GET API returns a specific control loop.
      operationId: getLoopByIdUsing
      parameters:
        - name: controlLoopId
          in: path
          description: 'Id of the control loop.'
          required: true
          example: "org.onap.PM_CDS_Blueprint"
          schema:
            type: string
      responses:
        '200':
          description: OK
          content:
            application/yaml:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ControlLoop'
  
    delete:
      tags:
        - runtime-inventory
      summary: Delete a specific Control Loop by ID.
      operationId: deleteLoopByIdUsing
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not present

    post:
      tags:
        - runtime-inventory
      summary: >-
        Accepts a Tosca Service Template representing a Control Loop.
      description: >-
        This API Create a Control Loop.
      operationId: createLoopWithIDUsing
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '409':
          description: Control Loop already exist.
     
    put:
      tags:
        - runtime-inventory
      summary: Updates a Control Loop.
      operationId: updateLoopWithIDUsing
      parameters:
        - name: controlLoopId
          in: path
          description: Identity for the control loop
          required: true
          style: simple
          schema:
            type: string
      responses:
        '200':
          description: OK
        '404':
          description: Control Loop not found

components:
  schemas:
    ControlLoop:
      title: ControlLoop
      type: object
      properties:
        id:
          type: string
        ControlLoop:
          type: string

3.1.2: Commissioning Sequence Diagrams

GET, DELETE, CREATE API ControlLoop Sequence Diagram

@startuml

actor CLAMP
participant API_Gateway
database Database
note over Database: It can be DesiginTime Catalogue Or RunTime Inventory Database


CLAMP -> API_Gateway: Rest API Request (JSON/Yaml)
API_Gateway -> Database: Get, Delete Or Create ControlLoops
activate API_Gateway
note over API_Gateway,Database: Pull, Delete existing Or Create New ControlLoops in Database

alt successful case

    Database -> API_Gateway : Return List of ControlLoops Or able to Delete, Create ControlLoop in Database
    API_Gateway -> CLAMP: Return Success

else failure case

    API_Gateway <- Database: Return either Empty List for Get ControlLoops Or Failure Status for Delete, Create Request
	deactivate API_Gateway
    note over Database: No ControlLoops exist Or Failed in Deleting, Creating ControlLoops
    API_Gateway -> CLAMP : Return Failure

end
@enduml

Commission API ControlLoop Sequence Diagram

@startuml

actor CLAMP
participant API_Gateway
database DesignTime_Catalogue_Database
database RunTime_Inventory_Database
participant Dmaap
collections Participants

CLAMP -> API_Gateway: Rest Commission API Request (JSON/Yaml)
API_Gateway -> DesignTime_Catalogue_Database: Fetch ControlLoop from DesignTime Catalogue Database
activate API_Gateway
note over API_Gateway,DesignTime_Catalogue_Database: Fetch existing ControlLoops from DesignTime Catalogue Database

alt successful case
    DesignTime_Catalogue_Database -> API_Gateway : Return List of existing ControlLoops from DesignTime Catalogue Database
    note over DesignTime_Catalogue_Database, RunTime_Inventory_Database: Populate RunTime Inventory with ControlLoops existing in DesignTime Catalogue Database
    API_Gateway -> RunTime_Inventory_Database: Push ControlLoop to RunTime Inventory Database
    note over Dmaap: Putting ControlLoop Metadata for individual Participant
    API_Gateway -> Dmaap: Push ControlLoop to individual Participants(App) over Dmaap
    note over Dmaap,Participants: Asynchronous communication
    Dmaap -> Participants
    API_Gateway -> CLAMP: Return Success

else failure case

    API_Gateway <- DesignTime_Catalogue_Database: Return Empty ControlLoop List, No ControlLoop exist in DesignTime Catalogue
	deactivate API_Gateway
    note over API_Gateway, DesignTime_Catalogue_Database: DesignTime Catalogue Database is Empty, cannot commission ControLoop in RunTime Inventory Database 
    API_Gateway -> CLAMP : Return Failure

end
@enduml

3.2: Instantiation

Robertas Rimkus to pad out this section

This section refers to Instantiation of a Commissioned control loop. A client, in this case CLAMP (potentially DCAEMOD, etc in the future) will render the commissioned control loops allowing selection of a particular control loop to be instantiated. User will then provide the configurations needed to instantiate the selected control loop which will be sent onto the CL_Instance_Control Service. The service will then distribute the configurations to DMaaP topic. Participants (agents) will pull the event containing the config and pick out their control loop components to be instantiated and start/set up those particular components. CL_Instance_Control Service will be waiting for a response back from all participants involved in the instantiation of the control loop, in regards to the state of instantiation. In successful response case the service will store the CL Instance LCM (Life Cycle management) data into the runtime DB as well as providing a message back to the client of the successful instantiation. In failure to receive the response case, a timeout will be called, which will result in a teardown event being sent to DMaaP. The participants will then receive the event and proceed to teardown the components that were instantiated or check that they have failed to instantiate in the first place and send a Teardown ACK back to the CL_Instance_Control Service. No CL Instance LCM data will be stored and a message indicating failure to instantiate the CL along side with the error will be sent back to the client (CLAMP).

3.2.1: Instantiation REST API

openapi: 3.0.3
info:
  title: Swagger Control Loop Instantiation
  description: ""
  version: 1.0.0
servers:
  - url: '{scheme}://onap/controlloop/v2/'
    variables:
      scheme:
        description: 'The Data Set API is accessible via https and http'
        enum:
          - 'https'
          - 'http'
        default: 'http'
tags:
- name: instantiate
  description: Control Loop Instantiation
paths:
  /runtime-inventory/instantiate:
    post:
      tags:
      - instantiate
      summary: Instantiate a commissioned control loop
      operationId: instantiateLoop
      requestBody:
        description: Control loop that is to be instantiated
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ControlLoop'
        required: true
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ApiResponse'
        404:
          description: No Control Loop found
        405:
          description: Invalid input
  /runtime-inventory/loops/instantiated:
    get:
      tags:
      - instantiate
      summary: Get a list of instantiated control loops
      responses:
        200:
          description: Success
          content:            
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/ControlLoop'
        404:
          description: No instantiated control loops found
  /runtime-inventory/loops/instantiated/{loop-id}:
    get:
      tags:
      - instantiate
      summary: Get an instantiated control loop by id
      parameters:
      - name: loop-id
        in: path
        description: ID of instantiated loop to return
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/ControlLoop'
        404:
          description: Control Loop not found
    put:
      tags:
      - instantiate
      summary: Update an instantiated control loop
      parameters:
      - name: loop-id
        in: path
        description: ID of instantiated loop
        required: true
        schema:
          type: string
      requestBody:
        description: Control loop to be updated
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ControlLoop'
        required: true
        
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/ControlLoop'
        404:
          description: Control Loop not found
    delete:
      tags:
      - instantiate
      summary: Delete an instantiated control loop
      parameters:
      - name: loop-id
        in: path
        description: ID of instantiated loop
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
        404:
          description: Control Loop not found
components:
  schemas:
    ControlLoop:
      title: ControlLoop
      type: object
      properties:
        id:
          type: string
        controlLoopTosca:
          type: string
    ApiResponse:
      type: object
      properties:
        code:
          type: integer
          format: int32
        status:
          type: string

3.2.2: Instantiation Sequence Diagrams

@startuml

actor CLAMP
participant Commissioning_API
database Runtime_DB
control CL_Instance_Control
participant DmaaP
collections Participants

CLAMP -> Commissioning_API : Fetch all Commissioned Control Loops
Commissioning_API -> Runtime_DB : Rest call to get the Commissioned Control Loops
Runtime_DB -> Commissioning_API : Return all Commissioned Loops which reside in the database
Commissioning_API -> CLAMP : Provide the data
CLAMP -> CLAMP : Select Control Loop to be instantiated and provide configurations for it
CLAMP -> CL_Instance_Control : Instantiate CL POST REST API call

alt case where CLAMP sends commissioned CL id instead of the full CL in the body

CL_Instance_Control -> Runtime_DB : Fetch the Commissioned Control Loop from Models table
Runtime_DB -> CL_Instance_Control : Return the requested Commissioned Control Loop data

end

CL_Instance_Control -> DmaaP : INSTANTIATE Event (JSON/Yaml)
note over CL_Instance_Control, DmaaP : List of Participant ID's containing Participant Metadata Blocks
Participants -> DmaaP : Pulls its part of CL and tries to Instantiate

alt failure case

note over CL_Instance_Control : TIMEOUT, Instantiation ACK not received by monitoring service \nSupervision_Service requests a teardown
CL_Instance_Control -> DmaaP : TEARDOWN Event (JSON/Yaml)
Participants -> DmaaP : Pulls its part of CL and starts Teardown

end
@enduml

3.2.3: Instantiation DMaaP API

Initial Thought for an event to be sent from CL_Instance_Control onto DMaaP for Participants to consume. The event would go onto an output topic which the Participants would be polling/subscribed to

e.g url : https://{{ONAPIP}}:{{DMaaPPort}}/events/CL_INSTANCE_CONTROL_OUTPUT

{
  "instance-id":"myCLInstance1",
  "action":"INSTANTIATE",
  "configurations":[
    {
      "participant-id":"Participant_DCAE",
      "applications":[
        {
          "application-id":"example.pmsh",
          "config":{
            
          }
        },
        {
          "application-id":"example.dfc",
          "config":{
            
          }
        }
      ]
    },
    {
      "participant-id":"Participant_Policy",
      "applications":[
        {
          "application-id":"example.OperationalPolicy",
          "config":{
            
          }
        }
      ]
    }
  ]
}

*Preferred solution is to send TOSCA in the body. Meaning we could reuse the parsing code which is already present and provide it to the participant.

3.2.4: Instantiation Participant API

*Suggestion was to put JAVA API code in this section for the participant talking to DMaaP. TBD

3.3: Monitoring

In this case it refers to monitoring the data that the participants will provide to DMaaP. Participants will send events to DMaaP which will be pulled by the CL_Supervision_Service in to the runtime database. Monitoring service provides APIs to display the statistics data from runtime database to the Monitoring GUI. The data provided should include a reference id to the control loops that are instantiated on the participant, as well as the applications that have been instantiated as a part of that control loop for that participant. Data should also include the time that the application has started, state of it (running/terminated) and any other critical information which would help to determine the health of an instantiated control loop and its components. Idea is for the participant to provide events every certain period of time, similar to a health-check, in order to provide consistent monitoring.

3.3.1: Monitoring REST API

openapi: 3.0.3
info:
  title: Swagger Control Loop Monitoring
  description: ""
  version: 1.0.0
servers:
  - url: '{scheme}://onap/controlloop/v2/'
    variables:
      scheme:
        description: 'The Data Set API is accessible via https and http'
        enum:
          - 'https'
          - 'http'
        default: 'http'
tags:
- name: monitoring
  description: Control Loop Monitoring
paths:
  /runtime-inventory/monitoring:
    post:
      tags:
      - monitoring
      summary: Input monitoring data into the database
      requestBody:
        description: Monitoring data to be input
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/MonitoringData'
        required: true
      responses:
        200:
          description: OK
        405:
          description: Invalid input
    get:
      tags:
      - monitoring
      summary: Get monitoring data for all participants
      responses:
        200:
          description: Success
          content:            
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/MonitoringData'
        404:
          description: No monitoring data found
  /runtime-inventory/monitoring/{participant-id}:
    get:
      tags:
      - monitoring
      summary: Get monitoring data for a participant by participant id
      parameters:
      - name: participant-id
        in: path
        description: ID of a participant 
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/MonitoringData'
        404:
          description: Monitoring data for the participant not found
    put:
      tags:
      - monitoring
      summary: Update monitoring data for a participant by participant id
      parameters:
      - name: participant-id
        in: path
        description: ID of a participant
        required: true
        schema:
          type: string
      requestBody:
        description: Monitoring data to be updated
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/MonitoringData'
        required: true
      responses:
        200:
          description: OK
          content:            
            application/json:
              schema:
                $ref: '#/components/schemas/MonitoringData'
        404:
          description: Monitoring data for the participant not found
    delete:
      tags:
      - monitoring
      summary: Delete monitoring data for a participant by participant id
      parameters:
      - name: participant-id
        in: path
        description: ID of a participant
        required: true
        schema:
          type: string
      responses:
        200:
          description: OK
        404:
          description: Monitoring data for the participant not found
components:
  schemas:
    MonitoringData:
      properties:
        participant-id:
          type: string
        control-loops:
          type: array
          items:
            properties:
              cl-instance-id:
                type: string
              applications:
                type: array
                items:
                  properties:
                    application-id:
                      type: string
                    status:
                      type: string
                      enum:
                        - INSTANTIATION_STARTED
                        - INSTANTIATED_RUNNING
                        - INSTANTIATED_FAILED
                    other-data:
                      type: string

3.3.2: Monitoring Sequence Diagrams

@startuml

control CL_Instance_Control
database Runtime_DB
participant Monitoring_Service
participant DMaaP
collections Participants

CL_Instance_Control -> DMaaP : INSTANTIATE Event (JSON/Yaml)
Participants -> DMaaP : Pulls its part of CL and starts instantiation of components

Participants -> DMaaP : Starts sending monitoring events once instantiation of components begins
note over DMaaP : Events are sent at set intervals

Supervision_service-> DMaaP : Consistently pulls events off the monitoring topic
Monitoring_Service -> Runtime_DB : Puts and updates the monitoring data on the DB

@enduml

3.3.3: Monitoring DMaaP API

Participants will send an event containing monitoring data to a DMaaP topic at a set interval after participant has received an event to instantiate a control loop

e.g url: https://{{ONAPIP}}:{{DMaaPPort}}/events/CL_MONITORING_SERVICE_INPUT

{
  "timestamp":"1605689254",
  "participant-id":"Participant_DCAE",
  "control-loops":[
    {
      "instance-id":"myCLInstance1",
      "applications":[
        {
          "application-id":"example.pmsh",
          "status":"INSTANTIATION_STARTED"
        },
        {
          "application-id":"example.dfc",
          "status":"INSTANTIATION_STARTED"
        }
      ]
    },
    {
      "instance-id":"myCLInstance2",
      "applications":[
        {
          "application-id":"example.pmsh",
          "status":"INSTANTIATED"
        }
      ]
    }
  ]
}

3.3.4: Monitoring Participant API

Presume similar thinking to Instantiation Participant API

*Suggestion was to put JAVA API code in this section for the participant talking to DMaaP. TBD

3.4: Supervision

Supervision is responsible for ensuring that

control loops are established once their initiation has been ordered
control loops are running correctly once their initiation is completed
control loops are correctly removed once their removal has been ordered

3.4.1: Supervision Sequence Diagrams

@startuml

actor Client
participant Commissioning_API
control CL_Instance_Control
database Runtime_DB
participant Supervision_Service
participant Monitoring_Service
participant DMaaP
collections Participants


==Instantation==

Client -> Commissioning_API : Fetch all Commissioned Control Loops
Commissioning_API -> Runtime_DB : Rest call to get Commissioned Control Loops
Runtime_DB -> Commissioning_API : Return Commissioned Control Loops
Commissioning_API -> Client : Provide the data
Client -> Client : Select CL for instantiation and provide config
Client -> CL_Instance_Control : Instantiate CL POST REST API call

alt case where Client sends commissioned CL id instead of the full CL in the body

CL_Instance_Control -> Runtime_DB : Fetch the Commissioned Control Loop from Models table
Runtime_DB -> CL_Instance_Control : Return the requested Commissioned Control Loop data

end

CL_Instance_Control -> DMaaP : INSTANTIATE Event (JSON/Yaml)
note over CL_Instance_Control, DMaaP : List of Participant ID's containing Participant Metadata Blocks
Participants -> DMaaP : Pulls its part of CL and tries to Instantiate


==Monitoring==

Participants -> DMaaP : Starts sending monitoring events \nOnce instantiation of components begins
note over DMaaP : Events are sent at set intervals, one per participant

Supervision_Service -> DMaaP : Consistently pulls events off the monitoring topic
Monitoring_Service -> Runtime_DB : Puts and updates the monitoring data on the DB


==Supervision==

Supervision_Service -> DMaap : Periodic check on monitoring data

alt CL successfully instantiated

Supervision_Service -> Client : Return Instantiation Successful

else CL failed to instantiate

... TIMEOUT CL state not changed to INSTANTIATED ...
Supervision_Service -> CL_Instance_Control : Timeout, error occured
CL_Instance_Control -> DMaaP : TEARDOWN Event (JSON/Yaml)
Participants -> DMaaP : Pulls its part of CL and starts Teardown
note over Monitoring_Service : Monitoring Service receives events and updates state
note over Supervision_Service : Supervision Service finds that status has changed
Supervision_Service -> Client : Return Instantiation Failed (http code and an error message) 

end
@enduml

3.4.2: Supervision APIs to other components

4: Design

4.1: Server Side

4.1.1 Database Schema and JPA

4.1.2: TOSCA Processing

4.1.3: Instance Control

4.1.4: Execution Monitoring

4.2: Participant Side

Participant is a component that acts as a bridge between Runtime and components like Policy-framework, DCAE, Kubernetes cluster etc.
It listens to Dmaap to receive messages from runtime and performs operations towards control loop components.

Every participant has two parts Participant-Intermediary and a Participant-Impl.
Participant-Intermediary is a common component that listens to Dmaap and acts on the messages, participant-impl handles the logic towards
control loop element.

4.2.1: Participant Message handling

Participant handles 4 types of messages

1. Participant State Change : This message handles states of a participant. Runtime can order participant for a state change.
ParticipantState can be set to one of the following
UNKNOWN : Control Loop execution is unknown.
PASSIVE : Control Loop execution is always rejected.
SAFE : Control Loop execution proceeds, but changes to domain state or context are not carried out.
The participant returns an indication that it is running in SAFE mode together with the action it would
have performed if it was operating in ACTIVE mode.
TEST : Control Loop execution proceeds and changes to domain and state are carried out in a test environment.
The participant returns an indication that it is running in TEST mode together with the action it has performed
on the test environment
ACTIVE : Control Loop execution is executed in the live environment by the participant.
TERMINATED : Control Loop execution is terminated and not available.

2. Control Loop Update: This message creates the control loop elements and brings them from UNINITIALIZED to PASSIVE state.
ControlLoopUpdate message contains full ToscaServiceTemplate describing all components participating in a control loop.
This acts as a template for any control loop to be created according to the template.
When participant-intermediary receives this message, it triggers creation of policy-types and policies in Policy-Framework by Policy-Participant,
and deploys DCAE from DCAE-participant

3. Control Loop State change: This message is used to order a state change in control loop element.
Runtime can order one of the following ordered states.
UNINITIALIZED : The control loop or control loop element should become uninitialized on participants, it should not exist on participants.
PASSIVE : The control loop or control loop element should initialized on the participants and be passive, that is,
it is not handling control loop messages yet.
RUNNING : The control loop or control loop element should running and is executing control loops. Once any of above states are ordered, then control loop element transitions into
UNINITIALIZED : The control loop or control loop element is not initialized on participants, it does not exist on participants.
UNINITIALIZED2PASSIVE : The control loop or control loop element is changing from uninitialized to passive,
it is being initialized onto participants.
PASSIVE : The control loop or control loop element is initialized on the participants but is passive, that is, it is not
handling control loop messages yet.
PASSIVE2RUNNING : The control loop or control loop element is changing from passive to running,
the participants are preparing to execute control loops.
RUNNING : The control loop or control loop element is running and is executing control loops.
RUNNING2PASSIVE : The control loop or control loop element is completing execution of current control loops but
will not start running any more control loops and will become passive.
PASSIVE2UNINITIALIZED : The control loop or control loop element is changing from passive to uninitialized,
the control loop is being removed from participants

4. Participant Healthcheck: This message is used to learn the health status of a participant.

As a response to any of the above message participant returns a Participant Status message, holding respective message response.
Runtime receives Participant Status message and stores relevant information in database, Or performs respective actions.

4.2.2: Policy Participant Agent

Policy participant receives messages through participant-intermediary common code, and handles them by invoking REST APIs towards policy-framework.

For example, When a ControlLoopUpdate message is received by policy participant, it contains full ToscaServiceTemplate describing all components
participating in a control loop. When the control loop element state changed from UNINITIALIZED to PASSIVE, Policy-participant triggers creation
of policy-types and policies in Policy-Framework.
When the state changes from PASSIVE to UNINITIALIZED, Policy-Participant deletes the policies, policy-types by invoking REST APIs towards policy-framework.

4.2.4: DCAE Participant Agent

DCAE participant receives messages through participant-intermediary common code, and handles them by invoking CLAMP DCAE methods,
which internally work towards DCAE.

For example, When a ControlLoopUpdate message is received by DCAE participant, it contains full ToscaServiceTemplate describing all components
participating in a control loop. When the control loop element state changed from UNINITIALIZED to PASSIVE, DCAE-participant triggers deploy
of DCAE.
When the state changes from PASSIVE to UNINITIALIZED, DCAE-Participant un-deploys DCAE by invoking methods towards CLAMP.

4.2.5: Kubernetes Participant Agent

Kubernetes participant receives messages through participant-intermediary common code, and handles them by invoking Kubernetes Open API.
For example, When a ControlLoopUpdate message is received by Kubernetes participant, When the control loop element state changed from UNINITIALIZED to PASSIVE, Kubernetes-participant triggers Kubernetes Open API and passes the HELM charts towards cluster.

4.3: Client Side

4.3.1: Client SDK: Composition of Control Loop Tosca

4.3.2: Client User Interface

4.4 Other Considerations

4.4.1 Upgrade

Performing a hot upgrade of the Control Loop at run time as well as handling an upgrade of the software in one or more of the participants in an Control Loop is a particularly challenging issue because upgrading must handle the following cases without tearing down the Control Loop:

Upgrade and changes of the configuration data of participants
Addition of or removal of participants in an Control Loop
Upgrade of software in one or more participants in an Control Loop
Maintenance of compatibility between participants when an update of more than one participant must be done together to ensure compatibility, for example, when a protocol being used by two participants to communicate is upgraded

4.4.2 Scalability

The system is designed to be inherently scalable. The control loop runtime server is stateless, all state is preserved in the run time inventory in the database. When the user requests a control loop operation (such as an instantiation, activation, passivation, or an ininitialization) the server broadcasts the request to participants over DMaaP and saves details of the request to the database. The server does not directly wait for responses to requests.

When a request is broadcast on DMaaP, the request is asynchronously picked up by participants of the types required for the control loop instance and those participants manage the life cycle of its control loop elements. Periodically, each participant reports back on the status of operations it has picked up for the control loop elements it controls, together with statistics on the control loop elements over DMaaP. On reception of these participant messages, the server stores this information to its database.

The server periodically runs a supervion function, which checks the status of all existing control loop instances and the status of outstanding requests. It builds a picture of the current status of each control loop instance from the reports on the elements of the control loop instances. Once the server has a full picture, it checks that the control loop instance is in the correct state as requested by the user of the system. If the control loop is not in the correct state, the supervision function can initiate actions such aas performing retries on operations or issuing alarms or notificaitons on control loop instances.

This approach makes it easy to scale control loop LCM. As control loop instance counts increase, more than one runtime server can be deployed and REST/supervision operations on control loop instances can run in parallel. The number of participants can scale because an asynchronous broadcast mechanism is used for server-participant communication and there is no direct connection or communication channel between participants and runtime servers. Participant state, control loop instance state, and control loop element state is held in the database, so any runtime server can handle operations for any participant. Because many participants of a particular type can be deployed and participant instances can load balance control loop element instances for different control loops of many types across themselves using a mechanism such as a Kubernetes cluster.

5: Goals

5.1: MVP

5.2: ControlLoop in Tosca LCM Istanbul Jiras

Design Time
- Support design of multiple control loops*
- Support design of individual control loop component**
- Support composition of control loops**
Runtime
- Participant registration and participant deregistration
- Support commissioning of control loops
  - Ingestion with artifact references*
  - Ingestion with artifact embedded**
- Support instantiation of control loop
  - Support instantiation of control loop TOSCA to DMaaP MR*
  - Support instantiation of config for the control loop*
- Support monitoring of control loops
  - Receive control loop heartbeat events (heartbeat starts when component of control loop is running)*
- Support supervision of control loops
  - Periodically check monitored data, and update state of control loop*
Participants
- Agent library*
- Reference(test) participant*
- CDS participant*
- DCAE participant*
- Policy participant*
Demo**
- Throwaway Monitoring/Control GUI