SO Monitoring Feature Requirements

• A design idea
  • Why not Camunda Cockpit as is: current Camunda Cockpit was designed from a BPMN process management perspective (note: need to study for TOSCA cases).
    • It does not meet service-level orchestration monitoring.
    • It is designed for BPMN definition/execution monitoring; require process knowledge for monitoring.
  • We need higher-level monitoring abstraction for both BPMN and TOSCA.
    • Associate Service Instance Id (or other keys) to the top-level process instance id. For the association, 
    • Could use a process variable holding the Service Instance id (or other keys), or
    • Could use a database holding the association
    • Allow VID, UUI or external apps monitor process workflow process (graphically and text-based) based on extensible search keys.
  • We need a platform level runtime and history process activity report capabilties out of the box.
    • Regardless use of Camunda Enterprise Edition or Community Edition.
    • Query to Camunda/ARIA database to extract activities. 
  • The following diagram depicts the high-level concept. 

• Rationale
  • Search is an Camunda enterprise feature, but we need to provide searching capabilitiy for non-enterprise edition.
    • Finding right process instance(s) for a NS/VNF service request is tedious and hassle.
    • To facilitate monitoring, we need more than what Camunda Community/Enterprise edition supports.
    • provides the process monitoring (instance-search) hyperlink to the SO clients for launching process monitoring.
      • Automates tedious manual steps for finding target process instance(s), bypassing tedious search starting from the process defintions.
      • Access customized SO Service List / Camunda Cockpit widgets from VID, UUI and external APIs.
  • If a service provider uses Camunda Enterprise edition, they can still utilize this SO monitoring on top of Camunda enterprise edition features.
  • Many of Camunda enterprise features such as CRUDV and version control of process definitions would be part of SDC.
    • ONAP separated workflow design and runtime.
    • Several enterprise features are not part of SO monitoring features and are not applicable.
  • What are current TOSCA orchestrator monitoring capabilities?
    • SO monitoring should cover both imperative and declarative orchestration.
    • Question: can we have a kind of uniform way of monitoring?

• High-Level Requirements
  • Dashboard views of Service lists
    • Filtering capabilities based on search criteria
    • Configurable search criteria
  • Dashboard views of statistics (donuts, pie charts, etc.) for filtered service instances
  • Service Instance Rendering and detail panel views
    • with sub-service instance drill-drown and drill-up capabilities
      • A service instance could be realized by multiple process instances
    • with process / task detail
    • Topology (workflow) views during/after orchestration
  • Input/output data views for process/task/service task (messages, parameters)
    • Display on service / task detail panel
    • Provide message log views (could be on a pop-up widget)
  • Color coding/visual indication of statistic and service type and status
  • Troubleshooting capabilities by manipulating the workflow during orchestration for troubleshooting and retry from the current location (stretch goal)
  • TOSCA orchestration monitoring (stretch goal)

• Widget Requirements and Design
  • Service List Widget 
    • Capabilities
      • Provides monitoring capabilities for processed services based on search criteria
        • Configurable Search Criteria filtering: Service ID, Operation Type, Status, User Id, Date/Time range
        • Actual filtering criteria fields could be changed based on configuration
      • The Service List panel will display the filtered Service list.
      • This widget can be triggered from other UIs (VID, UUI, external applications).
    • Design
      • The search criteria fields will be defined in a search criteria field configuration file (e.g., so-monitoring-config.yaml). So, they can be customized and support internationalization.
      • Based on the fields and buttons configuration, the Service List widget will render the corresponding fields and buttons.
      • The search criteria fields support the field validation based on the field type; e.g., length, date format, time format.
      • The wild characters are supported for the String field.
      • Example of so-monitoring-config.yaml

service-list-search-criteria-fields:

user-id:

name: User Id

type: String

max-length: 30

validation: alpha-numeric

service-id:

name: Service Id

type: String

max-length: 30

validation: alpha-numeric

operation-type:

name: Operation Type

type: String

max-length: 30

validation: alpha-numeric

status:

name: Status

type: combo-list

max-length: 30

default-value:

- status-item: finished

name: Finished

- status-item: processing

name: Processing

- status-item: error

name: Error

date-from:

name: Date From

type: date

validation: date-format

time-from:

name: Time From

type: time

validation: time-format

date-to:

name: Date To

type: date

validation: date-format

time-to:

name: Time To

type: time

valiation: time-format

filtering-expression:

name: Filtering Expression

type: string

validation: regular-expression

action-buttions:

search:

name: Search

statistic:

name: Statistic

monitor:

name: Monitor

previous:

name: Previous       # or <--

next:

name: Next             # or -->

• • • • The Service List panel supports pagination, which will be controlled by the Navigation buttons.
      • Queries a service list from the SO Request DB through the Monitoring REST APIs.
      • Action buttons are enabled/disabled based on data population and/or context.

• • Search Operations:
    • Capabilities
      • Retrieve a service list thru the Service List REST API, getServiceList(...)
      • Provides filtering, Service ID, Operation Type, Status User Id, Date/Time range (actual criteria could be configured).
      • Click the Search button, and the filtered service list will be displayed.
    • Design
      • Use a REST API, getServiceList with search criteria parameters

• Service List Get Service Instance Monitoring
  • Select a row and click the Monitor button. Then, it will go to the Service instance (= process instance in Camunda) monitoring widge for the selected Service instance.

• • Service Statistic Widget
    • When, the static button is clicked, the Service Statistic dashboard will be shown. 
    • Collect and display Service Instance statistic per filtered service list, for the filtered service range.

• • Service Instance Rendering and Detail View
    • Get a process definition XML through the Service Id and Process instance association.
    • use Camunda REST API, get/process-defintion/{id}xml
    • Get the state of a process instance from the activity-instances
    • Use Camunda REST API, get/process-instance/{id}/activity-instances
    • Render the BPMN XML with bpmn.io and places markers on top of it, and provide Service instance detail views.

• Rendering code example
  • Import BPMN XML string through Camunda REST APIs and render the BPMN workflow.
  • Attache overlay events on the Call Activities for drilling down and display details.

• • Task Drill-Down/Drill-Up and Detail View
    • Support Task (CAll Activity) drill-down/drill-up capabilties.
    • When a call activity task is selected, the Drill-Down button will be enabled.
    • Provide detail views for the selected Task.
    • When a task is selected including the call activity, the Detail view panel will display details for the selected task.

• • Sub-Service Instance Rendering and Detail View
    • Get a sub process defintion XML through the Service Id and Process Instance association.
    • Use Camunda REST API, get/process-definition/{sub-id}/xml
    • Get the state of a sub-process instance from the activity-instances.
    • Use Camunda REST API, get/process-instance/{sub-id}/activity-instances
    • Render the BPMN XML with bpmn.io and places markers on top of it, and provides service instance detail views.
    • This widget uses the same code of the Service Instance Rendering and Detail View with the sub-process instance id. 
    • The Drill-Up button will be enabled.

• REST APIs for providing data to UIs
  • Provides REST services, by utilzing 1) Camunda REST APIs, such as BPMN XML string, process activity data, process variable, statistic, and 2) SO Request DB APIs for a service list.
  • Consolidate data responses from multpile Camunda calls and feed them to UIs.
  • Use of HistoryService APIs, example, processEngine.getHistoryService().createHistoricProcessVariableQuery().xyz
  • Set a History level to ACTIVITY as a minimum; AUDIT (default) level for process variable tracing
  • Provides workflow tracing (between parent-child workflows, interaction with other services; service task in and out); example,
    • processEngine.getRuntimeService().createExecutionQuery().processVariableValueEquals("serviceInstanceId", serviceInstanceId).singleResult();
  • Custom Query
    • Custom Query against History ACT_HI_DETAIL database table, as needed
  • Create a REST API, getServiceList with search criteria
    • Enhance MsoRequestsDbAdapter and MsoRequestsDbAdapterImpl to implement getInfraRequest(...) with additional parameters
    • invoke getInfraRequest(...) to collect service list data based on search criteria.
• Custom History Event Producer
  • Populate additional data in history with extensibility

• • Estimates
  • Note: 
    • It is a rough estimate - to be refined
    • Estimate includes development and unit testing time