SO Monitoring Feature Design

• A design idea
  • Why not Camunda Cockpit as is: current Camunda Cockpit was designed from a BPMN process management perspective (note: need to study for TOSCA cases).
    • It does not meet service-level orchestration monitoring.
    • It is designed for BPMN definition/execution monitoring; require process knowledge for monitoring.
  • We need higher-level monitoring abstraction for both BPMN and TOSCA.
    • Associate Service Instance Id (or other keys) to the top-level process instance id. For the association, 
    • Could use a process variable holding the Service Instance id (or other keys), or
    • Could use a database holding the association
    • Allow VID, UUI or external apps monitor process workflow process (graphically and text-based) based on extensible search keys.
  • We need a platform level run-time and history process activity report capabilities out of the box.
    • Regardless use of Camunda Enterprise Edition or Community Edition.
    • Query to Camunda/ARIA database to extract activities. 
  • The following diagram depicts the high-level concept. 

• Rationale
  • Search is an Camunda enterprise feature, but we need to provide searching capabilitiy for non-enterprise edition.
    • Finding right process instance(s) for a NS/VNF service request is tedious and hassle.
    • To facilitate monitoring, we need more than what Camunda Community/Enterprise edition supports.
    • provides the process monitoring (instance-search) hyperlink to the SO clients for launching process monitoring.
      • Automates tedious manual steps for finding target process instance(s), bypassing tedious search starting from the process defintions.
      • Access customized SO Service List / Camunda Cockpit widgets from VID, UUI and external APIs.
  • If a service provider uses Camunda Enterprise edition, they can still utilize this SO monitoring on top of Camunda enterprise edition features.
  • Many of Camunda enterprise features such as CRUDV and version control of process definitions would be part of SDC.
    • ONAP separated workflow design and runtime.
    • Several enterprise features are not part of SO monitoring features and are not applicable.
  • What are current TOSCA orchestrator monitoring capabilities?
    • SO monitoring should cover both imperative and declarative orchestration.
    • Question: can we have a kind of uniform way of monitoring?

• High-Level Requirements
  • Dashboard views of Service lists
    • Filtering capabilities based on search criteria
    • Configurable search criteria
  • Dashboard views of statistics (donuts, pie charts, etc.) for filtered service instances
  • Service Instance Rendering and detail panel views
    • with sub-service instance drill-drown and drill-up capabilities
      • A service instance could be realized by multiple process instances
    • with process / task detail
    • Topology (workflow) views during/after orchestration
  • Input/output data views for process/task/service task (messages, parameters)
    • Display on service / task detail panel
    • Provide message log views (could be on a pop-up widget)
  • Color coding/visual indication of statistic and service type and status
  • Troubleshooting capabilities by manipulating the workflow during orchestration for troubleshooting and retry from the current location (stretch goal)
  • TOSCA orchestration monitoring (stretch goal)

• UI Widgets
  • Service List 
  • Service Statistic
  • Service Instance Rendering and Detail

• Widget Requirements and Design
  • *** Note 1: actual screen layout and display can be changed based on implementation decisions. ***
  • *** Note 2: the service list panel columns can be changed based on the database table columns. The important fields would be the service instance id, requester id, date/time for filtering.***
  • *** Note 3: we can add a new field, Proc_Inst-ID varchar(64) to the Infra_Active_Requests database table, to associate it (as a top-level process instance id) with the service instance id
  • Service List Widget 
    • Capabilities
      • Provides monitoring capabilities for processed services based on search criteria
        • Configurable Search Criteria filtering: Service ID, Operation Type, Status, User Id, Date/Time range
        • Actual filtering criteria fields could be changed based on configuration
      • The Service List panel will display the filtered Service list.
      • This widget can be triggered from other UIs (VID, UUI, external applications).
    • Design
      • The search criteria fields will be defined in a search criteria field configuration file (e.g., so-monitoring-config.yaml). So, they can be customized and support internationalization.
      • Based on the fields and buttons configuration, the Service List widget will render the corresponding fields and buttons.
      • The search criteria fields support the field validation based on the field type; e.g., length, date format, time format.
      • The Service List panel supports pagination, which will be controlled by the Navigation buttons.
      • Queries a service list from the SO Request DB through the Monitoring REST APIs.
      • Action buttons are enabled/disabled based on data population and/or context.
      • The wild characters are supported for the Search fields.
      • Note: for security (who, when and how access info), the requester info (User Id) is needed. It could be a future requirement.
        • The flexible filtering expression could be a stretch goal (it depends the BE Request DB query capabilities)

• • • • Example of so-monitoring-config.yaml                   // note: the flexibility depends on the SO getServiceList(...) support

service-list-search-criteria-fields:

user-id:

name: User Id

type: String

max-length: 30

validation: alpha-numeric

service-id:

name: Service Id

type: String

max-length: 30

validation: alpha-numeric

operation-type:

name: Operation Type

type: String

max-length: 30

validation: alpha-numeric

status:

name: Status

type: combo-list

max-length: 30

default-value:

- status-item: finished

name: Finished

- status-item: processing

name: Processing

- status-item: error

name: Error

date-from:

name: Date From

type: date

validation: date-format

time-from:

name: Time From

type: time

validation: time-format

date-to:

name: Date To

type: date

validation: date-format

time-to:

name: Time To

type: time

valiation: time-format

filtering-expression:

name: Filtering Expression

type: string

validation: regular-expression

action-buttions:

search:

name: Search

statistic:

name: Statistic

monitor:

name: Monitor

previous:

name: Previous       # or <--

next:

name: Next             # or -->

service-list:

max-row: 25000

service-instance-rendering-details:

max-page-num: 10

• • Search Operations:
    • Capabilities
      • Retrieve a service list thru the Service List REST API, getServiceList(...)
      • Provides filtering, Service ID, Operation Type, Status, Date/Time range (actual criteria could be configured).
      • Click the Search button, and the filtered service list will be displayed.
    • Design
      • After the user fills up the desired search crtieria fields, and click the Search button.
      • Use a REST API, getServiceList with search criteria parameters based on a new getInfraRequest(...)
      • Provide pagnation for the list, and enable/disable the navigation buttons based on the page location.
        • When the retreieved rows exceed the max-row configuration value, provide the user a warning to narrow the search scope.
      • When the Service List data is NOT populated, only the Search button is enabled; rest buttons are disabled.
      • When the Service List data is populated, enable the navigation, Statistic and Monitor buttons.
      • Stretch goals
        • sort the list ascending and descending order per column
        • change the column position

• Service List Get Service Instance Monitoring
  • Capabilities
    • Select a row and click the Monitor button. Then, it will open the Service instance (maps into process instance in Camunda) monitoring widget for the selected Service instance on the secondary/pop-up browser page.
  • Design
    • when a user clicks on a row, highlight the selected row and enable the Monitor button.
    • Once the Monitor button is clicked, go to the Service Instance Rendering and Deail widget, passing key fields such as service instance id, which will be mapped into the process instance id to find a matched process instance id in Camunda database in the Service Instnce Rendering and Detail widget.

• • Service Statistic Widget (stretch goal)
    • When, the static button is clicked, the Service Statistic dashboard will be shown. 
    • Collect and display Service Instance statistic per filtered service list, for the filtered service range.
    • Design
      • Query for service list data for a given search criteria. 
        • Use a REST API, getServiceList with search criteria parameters based on a new getServiceStatic(...) based on the  getInfraRequest(...)
      • Count the total number per status (Finished, Processing, Error)
      • Status Total / total number

• • Service Instance Rendering and Detail Widget
    • Capabilities
      • Get a process definition XML through the Service Id and Process instance association.
      • use Camunda REST API, GET /process-defintion/{id}/xml
      • Get the state of a process instance from the activity-instances
      • Use Camunda REST API, get/process-instance/{id}/activity-instances
      • Render the BPMN XML with bpmn.io and places markers on top of it, and provide Service instance detail views.
    • Design
      • This widget will be on the secondary/pop-up browser page. 
        • The Service List page will be the primary page, so the user can open multiple Service Instance Rendering and Detail pages.
      • The search criteria fields are for display only, and they cannot be changed.
      • The Detail panel is scrollable and display contents with the tag: value formt.
      • Query a process instance with the matched 'serviceInstanceId' process variable.
        • get a process instande id: query bases on the process variable, 
          • GET /history/variable-instance/{variableValue}, where {variableValue} = 'serviceIntanceId' ;
        • query BPML XML based on a process instance id
          • GET /process-definition/{id}/xml
        • query the process instance detail such as process variables
          • GET /process-instance/{id}/variables
          • Display the process variables on the Service Instance Detail panel
        • query and activity instance (tree) for the process instance id
          • GET /process-instance/{id}/activity-instances
          • GET /history/activity-instance/{id}
          • The response provides child activity instance info. By using this we can find the parent and child process instnce id and definition id.
            • If the current process instance id has its parent, enable the Drill-Up button.
          • When a user clicks on a task, display the task varables
            • GET /task/{id}variables
            • When the task activity type is call activity, it enables the Drill-Down button.
      • Render the process instance diagram with the retrieved XML.

        • use bpmn.io javascript library for the XML rendering and event attachments for process instance tasks.

bpmnViewer.importXML (xmlDiagram, function(err) {

if (err) { ... }

var canvas = bpmnViewer.get('canvas');

canvas.zoom ('fit-viewport');

var eventBus = bpmnViewer.get('eventBus');

// you may hook into any of the following events
var events = [
    'element.hover',
    'element.out',
    'element.click',
    'element.dblclick',
    'element.mousedown',
    'element.mouseup'
];

events.forEach(function(event) {

eventBus.on(event, function(e) {

log(event, 'on', e.element.id);

• • • • The Close button is always enabled.

• • The following is a possible alternative service instance rendering based on the Camunda Cockpit screen. If we go this path, the Camunda Cockpit customized plugin would be used. 

• • Task Drill-Down/Drill-Up and Detail View
    • Capabilities
      • Support Task (Call Activity) drill-down/drill-up capabilities.
      • When a call activity task is selected, the Drill-Down button will be enabled.
      • Provide detail views for the selected Task.
      • When a task is selected including the call activity, the Detail view panel will display details for the selected task.
    • Design
      • When a selected/clicked task activity type is the Call Activity, the Drill-Down button is enabled.

        • if (activityType.equals('CallActivity') {enableDrillDown();}

      • If the current process instance has its parent process instance (from the activity tree), the Drill-Up button is enabled.
        • if (parentActivityInstanceId != null) {enableDrillUp();}

• • Sub-Service Instance Rendering and Detail View
    • Capabilities
      • Get a sub process defintion XML through the Service Id and Process Instance association.
      • Use Camunda REST API, get/process-definition/{sub-id}/xml
      • Get the state of a sub-process instance from the activity-instances.
      • Use Camunda REST API, get/process-instance/{sub-id}/activity-instances
      • Render the BPMN XML with bpmn.io and places markers on top of it, and provides service instance detail views.
      • This widget uses the same code of the Service Instance Rendering and Detail View with the sub-process instance id. 
      • The Drill-Up button will be enabled.
    • Design
      • This widget code is the same as the Service Instance Rendering and Detail. 
      • The difference is the passed process id; the child process instance id will be passed.

• REST APIs for providing data to UIs
  • Capabilities
    • Provides REST services, by utilzing 1) Camunda REST APIs, such as BPMN XML string, process activity data, process variable, statistic, and 2) SO Request DB APIs for a service list.
    • Consolidate data responses from multple Camunda calls (both from process and history database) and feed them to UIs.
    • Use of HistoryService APIs, example, processEngine.getHistoryService().createHistoricProcessVariableQuery().xyz
    • Set a History level to ACTIVITY as a minimum; AUDIT (default) level for process variable tracing; currently, it is set to FULL
      • ProcessEngine processEngine = ProcessEngineConfiguration.createProcessEngineConfigurationFromResourceDefault().setHistory(ProcessEngineConfiguration.HISTORY_FULL).buildProcessEngine()
    • Provides workflow tracing (between parent-child workflows, interaction with other services; service task in and out); example,
      • processEngine.getRuntimeService().createExecutionQuery().processVariableValueEquals("serviceInstanceId", serviceInstanceId).singleResult();
    • Custom Query
      • Custom Query against History ACT_HI_DETAIL database table, as needed
    • Create a REST API, getServiceList with search criteria
      • Enhance MsoRequestsDbAdapter and MsoRequestsDbAdapterImpl to implement getInfraRequest(...) with additional parameters
      • invoke getInfraRequest(...) to collect service list data based on search criteria.
      • or use the RequestDB adapter directly.
  • Design
    • Write REST APIs in the new mso-api-handlers/mso-api-handler-monitoring

1. getServiceList(Map searchCriteria...)                                                 // invokes new getInfraRequest(Map searchCriteria...)
   1. getServiceStatistic(Map searchCriteria, String status);            // get the statistic percentage for the status
2. getProcessInstanceId(String serviceInstanceId);                               // get the processInstanceId through GET /history/variable-instance/{id}
3. getprocessDiagramXML(String processInstanceId);                          // get a process instance, get a process definition id, and get diagram
4. getprocessInstanceDetail(String processInstanceId); 
5. getHistoryActivityInstance(String processInstanceId);                       // GET /history/activity-instance?processInstanceId={processInstanceId}
6. getProcessInstanceParent (String processInstanceId);
7. getProcessInstanceChildren(String processInstanceId);
8. getProcessTaskDetail(String processInstanceId, String taskId);
9. getProcessTaskType(String processInstanceId, String taskid);        // query if this task type is "Call Activity" for drilling down
10. 
    
    • The following diagram depicts the API interaction.

• Custom History Event Producer  (stretch goal)
  • Populate additional data in history with extensibility
  • It is an optional component. We plan to use it when the default Camunda process and history data is not sufficient.

• Component Build Dependencies
  • The following diagram depicts the component build dependencies.
  • Components can be built in parallel with a certain level of sequential integration. 
  • Each estimate includes code development and unit testing plus some component integration and integration testing (since there are API dependencies).

• Estimates (T-Shirt Size: M)
  
  
  • Note: 
    • It is a rough estimate - to be refined
    • Estimate includes development and unit testing time
    • As mentioned in the above section, the components can be built in parallel. 
    • 5 hours / day