...

1 Terminology

This section describes the terminology used in the system.

...

At Run Time, the following Control Loop Life Cycle mangement management capabilities are supported:

...

Following the ONAP Reference Architecture, the architecture has a Design Time part and a Runtime part.

The Design Time part of the archtiecture architecture allows a user to specify metadata for participants. It also allows users to compose control loops. The Design Time Catalogue contains the metadata primitives and control loop definition primitives for composition of control loops. As shown in the figure above, the Design Time component provides a system where Control Loops can be designed and defined in metadata. This means that a Control Loop can have any arbitrary structure and the Control Loop developers can use whatever analytic, policy, or control participants they like to implement their Control Loop. At composition time, the user parameterises the Control Loop and stores it in the design time catalogue. This catalogue contains the primitive metadata for any participants that can be used to compose a Control Loop. A Control Loop SDK is used to compose a Control Loop by aggregating the metadata for the participants chosen to be used in a Control Loop and by constructing the references between the participants. The architecture of the Control Loop Design Time part will be elaborated in future releases.

...

In the figure above, five participants are shown. A Configuration Perisistence Persistence Participant manages Control Loop Elements that interact with the ONAP Configuration Persistence Service to store common data. The DCAE Participant runs Control Loop Elements that manage DCAE microservices. The Kubernetes Participant hosts the Control Loop Elements that are managing the life cycle of microservices in control loops that are in a Kubernetes ecosystem. The Policy Participant handles the Control Loop Elements that interact with the Policy Framework to manage policies for control loops. A Controller Participant such as the CDS Participant runs Control Loop Elements that load metadata and configure controllers so that they can partake in control loops. Any third party Existing System Participant can be developed to run Control Loop Elements that interact with any existing system (such as an operator's analytic, machine learning, or artificial intelligence system) so that those systems can partake in control loops.

4.

...

Other Considerations

4.1

...

Management

...

of

...

Control Loop Instance

...

• Changes of the configuration data of Control Loop Instances
• Coping with changes in Control Loop Definitions

...

Configurations

In order to keep management of versions of the configuration of control loop instances straightforward and easy to implement, the following version management scheme using semantic versioning is implemented. Each configuration of a Control Loop Instance and configuration of a Control Loop Element has a semantic version with 3 digits indicating the major.minor.patch number of the version.

Note that a configuration means a full set of parameter values for a Control Loop Instance.

...

When a request is broadcast on DMaaP, the request is asynchronously picked up by participants of the types required for the Control Loop Instance and those participants manage the life cycle of its control loop elements. Periodically, each participant reports back on the status of operations it has picked up for the Control Loop Elements it controls, together with statistics on the Control Loop Elements over DMaaP. On reception of these participant messages, the CLAMP runtime stores this information to its database.

The CLAMP runtime periodically runs a supervision function, which checks the status of all existing Control Loop Instances and the status of outstanding requests. It builds a picture of the current status of each Control Loop Instance from the reports on the elements of the Control Loop Instances. Once the CLAMP runtime has a full picture, it checks that each Control Loop Instance is in the correct state as requested by the user of the system. If a Control Loop Instance is not in the correct state, the supervision function can initiate actions such as performing retries on operations or issuing alarms or notifications on that Control Loop Instanceparticipant to use on a control loop can be selected from the registered participants in either of two ways:

1. Runtime-side Selection: The CLAMP runtime selects a suitable participant from the list of participants and sends the participant ID that should be used in the Participant Update message. In this case, the CLAMP runtime decides on which participant will run the Control Loop Element based on a suitable algorithm. Algorithms could be round robin based or load based.
2. Participant-side Selection: The CLAMP runtime sends a list of Participant IDs that may be used in the Participant Update message. In this case, the candidate participants decide among themselves which participant should host the Control Loop Element.

This approach makes it easy to scale Control Loop life cycle management. As Control Loop Instance counts increase, more than one CLAMP runtime can be deployed and REST/supervision operations on Control Loop Instances can run in parallel. The number of participants can scale because an asynchronous broadcast mechanism is used for runtime-participant communication and there is no direct connection or communication channel between participants and CLAMP runtime servers. Participant state, Control Loop Instance state, Control Loop Instance state, and Control Loop Element state is held in the database, so any CLAMP runtime server can handle operations for any participant. Because many participants of a particular type can be deployed and participant instances can load balance control loop element instances for different Control Loop Instances of many types across themselves using a mechanism such as a Kubernetes cluster.

4.3 API Gateway Support

4.4 Security and Multi Tenancy

and Control Loop Element state is held in the database, so any CLAMP runtime server can handle operations for any participant. Because many participants of a particular type can be deployed and participant instances can load balance control loop element instances for different Control Loop Instances of many types across themselves using a mechanism such as a Kubernetes cluster.

4.3 Sandboxing and API Gateway Support

At runtime, interaction between ONAP platform services and application microservices are relatively unconstrained, so interactions between Control Loop Elements for a given Control Loop Instance remain relatively unconstrained. A proposal to support access-controlled access to and between ONAP services will improve this. This can be complemented by intercepting and controlling services accesses between Control Loop Elements for Control Loop Instances for some/all Control Loop types.

API gateways such as Kong have emerged as a useful technology for exposing and controlling service endpoint access for applications and services. When a Control Loop Type is onboarded, or when Control Loop Instances are created in the Participants, CLAMP can configure service endpoints between Control Loop Elements to redirect through an API Gateway.

Authentication and access-control rules can then be dynamically configured at the API gateway to support constrained access between Control Loop Elements and Control Loop Instances.

The diagram below shows the approach for configuring API Gateway access at Control Loop Instance and Control Loop Element level.

At design time, the Control Loop type definition specifies the type of API gateway configuration that should be supported at Control Loop and Control Loop Element levels.

At runtime, the CLAMP can configure the API gateway to enable (or deny) interactions between Control Loop Instances and individually for each Control Loop Element. All service-level interactions in/out of a Control Loop Element, except that to/from the API Gateway, can be blocked by networking policies, thus sandboxing a Control Loop Element and an entire Control Loop Instance if desired. Therefore, a Control Loop Element will only have access to the APIs that are configured and enabled for the Control Loop Element/Instance in the API gateway.

For some Control Loop Element Types the Participant can assist with service endpoint reconfiguration, service request/response redirection to/from the API Gateway, or annotation of requests/responses.

Once the Control Loop instance is instantiated on participants, the participants configure the API gateway with the Control Loop Instance level configuration and with the specific configuration for their Control Loop Element.

Monitoring and logging of the use of the API gateway may also be provided. Information and statistics on API gateway use can be read from the API gateway and passed back in monitoring messages to the CLAMP runtime.

Additional isolation and execution-environment sandboxing can be supported depending on the Control Loop Element Type. For example: ONAP policies for given Control Loop Instances/Types can be executed in a dedicated PDP engine instances; DCAE or K8S-hosted services can executed in isolated namespaces or in dedicated workers/clusters; etc..

5 APIs and Protocols

The APIs and Protocols used by CLAMP for Control Loops are described on the pages below:

1. System Level Dialogues
2. Defining Control Loops in TOSCA for CLAMP
3. REST APIs for CLAMP Control LoopsAutomation Composition
4. The CLAMP Control Loop Automation Composition Participant Protocol

6 Design and Implementation

...