SO Monitoring Feature Design

• Rationale

  • From the service/network operator perspective, the BPMN Monitoring has been realized as a pain point since the Amsterdam release. Finding right process instance(s) and data for the  service / resource request is tedious and hassle.

  • Camunda is not an OSS vendor, and they don't have any business value to cover the OSS opearator-friendly monitoring.

  • There is Camunda cockpit, but regardless community or enterprise editions, they were designed from a BPMN process management perspective. 

    • They require the user BPMN definition and design knowledge for monitoring.
    • The user needs to know which service invokes which BPMN process definitions and how those process definitions are related to each other.
    • There could be thousands process instances daily, and a top-level process instance can have multiple sub-process instances (call activity instances).
  • • • It is not easy to find right process instances and data
  • We need more than what Camunda Community/Enterprise editions offer to
    • Facilitate the finding of associated process instances and data 
      • Allow VID, UUI or external application users monitor process workflow process (graphically and text-based) based on their search criteria.
    • Bypassing all the tedious manual search through process definitions and instances
    • Eliminating prerequisite knowledge of process definition relationships for monitoring - learn as you drill down and up
      • Needs drill down and drill up to cover sub-process instances graphically and text-based.
  • The following diagram depicts the high-level concept. 

• High-Level Requirements
  • Dashboard views of Service lists
    • Filtering capabilities based on search criteria
    • Configurable search criteria
  • Dashboard views of statistics (donuts, pie charts, etc.) for filtered request/service instances
  • Process / Service Instance Rendering and detail panel views
    • with sub-process/service instance drill-drown and drill-up capabilities
      • A process/service instance could be realized by multiple process instances
    • with process / task detail
    • Topology (workflow) views during/after orchestration
    • Provide process activity trails to identity the execution path and failure indication
  • Input/output data views for process/task/service task (messages, parameters)
    • Display on service / task detail panel
    • Provide message log views (could be on a pop-up widget)
  • Color coding/visual indication of statistic and task execution trails

• Design
  
  • Collect a service list based on search criteria through the REQUEST Database table.
    • Service Instance Id,
    • Request Id
    • Service Name
    • Status
    • Date
  • Display service list statistic (COMPLETED, IN_PROGRESS, FAILED, PENDING) for the selected service list.
  • Provides the hyperlinks between the service list to Camunda Process instances to
    • Automate tedious manual steps for finding target process instance(s), bypassing tedious search starting from the process definitions
    • Allow VID, UUI and external API users monitor their process instances and data
  • Use Camunda REST APIs to: 
    • Associate the Request Id / Service Instance Id (and other search criteria) to the top-level process instance without manual intervention
    • Use a process variable holding the Request Id for the process instance search
    • Get the process definition XML for workflow rendering
    • Get the process instance details such as activities and variables
    • User process activity data for 1) execution trail, 2) drill-down and drill-up.
  • Display and render collected process instance data on the pages
    • Workflow Diagram canvas (zoom in/out and scrolling)
    • Process Information panel
    • Activity Instances and Process variable panels
      
      

• UI Widgets
  • Service List 
  • Service Statistic
  • Process/Service Instance Rendering and Detail

• Widget Requirements and Design
  • *** Note 1: actual screen layout and display can be changed based on implementation decisions. ***
  • *** Note 2: the service list panel columns can be changed based on the database table columns. The important fields would be the request id / service instance id, requester id, date/time for filtering.***
  • *** Note 3: we can add a new field, Proc_Inst-ID varchar(64) to the Infra_Active_Requests database table, to associate it (as a top-level process instance id) with the service instance id
  • Service List Widget 
    • Capabilities
      • Provides monitoring capabilities for processed services based on search criteria
        • Configurable Search Criteria filtering: Service ID, Operation Type, Status, User Id, Date/Time range
        • Actual filtering criteria fields could be changed based on configuration
      • The Service List panel will display the filtered Service list.
      • This widget can be triggered from other UIs (VID, UUI, external applications).
    • Design
      • The search criteria fields will be defined in a search criteria field configuration file (e.g., so-monitoring-config.yaml). So, they can be customized and support internationalization.
      • Based on the fields and buttons configuration, the Service List widget will render the corresponding fields and buttons.
      • The search criteria fields support the field validation based on the field type; e.g., length, date format, time format.
      • The Service List panel supports pagination, which will be controlled by the Navigation buttons.
      • Queries a service list from the SO Request DB through the Monitoring REST APIs.
      • Action buttons are enabled/disabled based on data population and/or context.
      • The wild characters are supported for the Search fields.
      • Note: for security (who, when and how access info), the requester info (User Id) is needed. It could be a future requirement.
        • The flexible filtering expression could be a stretch goal (it depends the BE Request DB query capabilities)

<implementation>

• • • • Example of so-monitoring-config.yaml                   // note: the flexibility depends on the SO getServiceList(...) support

service-list-search-criteria-fields:

user-id:

name: User Id

type: String

max-length: 30

validation: alpha-numeric

service-id:

name: Service Id

type: String

max-length: 30

validation: alpha-numeric

operation-type:

name: Operation Type

type: String

max-length: 30

validation: alpha-numeric

status:

name: Status

type: combo-list

max-length: 30

default-value:

- status-item: finished

name: Finished

- status-item: processing

name: Processing

- status-item: error

name: Error

date-from:

name: Date From

type: date

validation: date-format

time-from:

name: Time From

type: time

validation: time-format

date-to:

name: Date To

type: date

validation: date-format

time-to:

name: Time To

type: time

valiation: time-format

filtering-expression:

name: Filtering Expression

type: string

validation: regular-expression

action-buttions:

search:

name: Search

statistic:

name: Statistic

monitor:

name: Monitor

previous:

name: Previous       # or <--

next:

name: Next             # or -->

service-list:

max-row: 25000

service-instance-rendering-details:

max-page-num: 10

• • Search Operations:
    • Capabilities
      • Retrieve a service list thru the Service List REST API, getServiceList(...)
      • Provides filtering, Service ID, Operation Type, Status, Date/Time range (actual criteria could be configured).
      • Click the Search button, and the filtered service list will be displayed.
    • Design
      • After the user fills up the desired search crtieria fields, and click the Search button.
      • Use a REST API, getServiceList with search criteria parameters based on a new getInfraRequest(...)
      • Provide pagnation for the list, and enable/disable the navigation buttons based on the page location.
        • When the retreieved rows exceed the max-row configuration value, provide the user a warning to narrow the search scope.
      • When the Service List data is NOT populated, only the Search button is enabled; rest buttons are disabled.
      • When the Service List data is populated, enable the navigation, Statistic and Monitor buttons.
      • Stretch goals
        • sort the list ascending and descending order per column
        • change the column position

• Service List Get Process / Service Instance Monitoring
  • Capabilities
    • Select a row and click the Monitor button. Then, it will open the process / Service instance (maps into process instance in Camunda) monitoring widget for the selected process / Service instance on the secondary/pop-up browser page.
  • Design
    • when a user clicks on a row, highlight the selected row and enable the Monitor button.
    • Once the Monitor button is clicked, go to the process / Service Instance Rendering and Deail widget, passing key fields such as process / service instance id, which will be mapped into the process instance id to find a matched process instance id in Camunda database in the Service Instance Rendering and Detail widget.

<Implementation>

• • Service Statistic Widget (stretch goal)
    • When, the static button is clicked, the Service Statistic dashboard will be shown. 
    • Collect and display Request / Service Instance statistic per filtered service list, for the filtered service range.
    • Design
      • Query for service list data for a given search criteria. 
        • Use a REST API, getServiceList with search criteria parameters based on a new getServiceStatic(...) based on the  getInfraRequest(...)
      • Count the total number per status (Finished, Processing, Error)
      • Status Total / total number

• • Process / Service Instance Rendering and Detail Widget
    • Capabilities
      • Get a process definition XML through the Service Id and Process instance association.
      • use Camunda REST API, GET /process-defintion/{id}/xml
      • Get the state of a process instance from the activity-instances
      • Use Camunda REST API, get/process-instance/{id}/activity-instances
      • Render the BPMN XML with bpmn.io (or equivalent) and places markers on top of it, and provide process / Service instance detail views.
    • Design
      • This widget will be on the secondary/pop-up browser page. 
        • The Service List page will be the primary page, so the user can open multiple Process / Service Instance Rendering and Detail pages.
      • The search criteria fields are for display only, and they cannot be changed.
      • The Detail panel is scrollable and display contents with the tag: value formt.
      • Query a process instance with the matched 'serviceInstanceId' process variable.
        • get a process instande id: query bases on the process variable, 
          • GET /history/variable-instance/{variableValue}, where {variableValue} = 'serviceIntanceId' ;
        • query BPML XML based on a process instance id
          • GET /process-definition/{id}/xml
        • query the process instance detail such as process variables
          • GET /process-instance/{id}/variables
          • Display the process variables on the Process / Service Instance Detail panel
        • query and activity instance (tree) for the process instance id
          • GET /process-instance/{id}/activity-instances
          • GET /history/activity-instance/{id}
          • The response provides child activity instance info. By using this we can find the parent and child process instnce id and definition id.
            • If the current process instance id has its parent, enable the Drill-Up button.
          • When a user clicks on a task, display the task varables
            • GET /task/{id}variables
            • When the task activity type is call activity, it enables the Drill-Down button.
      • Render the process instance diagram with the retrieved XML.

        • use bpmn.io javascript library for the XML rendering and event attachments for process instance tasks.

bpmnViewer.importXML (xmlDiagram, function(err) {

if (err) { ... }

var canvas = bpmnViewer.get('canvas');

canvas.zoom ('fit-viewport');

var eventBus = bpmnViewer.get('eventBus');

// you may hook into any of the following events
var events = [
    'element.hover',
    'element.out',
    'element.click',
    'element.dblclick',
    'element.mousedown',
    'element.mouseup'
];

events.forEach(function(event) {

eventBus.on(event, function(e) {

log(event, 'on', e.element.id);

• • • • The Close button is always enabled.

<Implementation>

• • The following is a possible alternative process / service instance rendering based on the Camunda Cockpit screen. If we go this path, the Camunda Cockpit customized plugin would be used. 
  • Calling the process instance cockpit page

    • get the processInstanceId from the getProcessInstanceId(String serviceInstanceId).

    • call the process instance cockpit page, e.g., following a function like below:

    • $scope.getProcessInstanceUrl = function(processInstance, params) {
        var path = '#/process-instance/' + processInstance.id;
        var searches = angular.extend({}, ($location.search() || {}), (params || {}));
      
        var keepSearchParams = [ 'viewbox' ];
        for (var i in params) {
          keepSearchParams.push(i);
        }
      
        return routeUtil.redirectTo(path, searches, keepSearchParams);
      };

• • Task Drill-Down/Drill-Up and Detail View
    • Capabilities
      • Support Task (Call Activity) drill-down/drill-up capabilities.
      • When a call activity task is selected, the Drill-Down button will be enabled.
      • Provide detail views for the selected Task.
      • When a task is selected including the call activity, the Detail view panel will display details for the selected task.
    • Design
      • When a selected/clicked task activity type is the Call Activity, the Drill-Down button is enabled.

        • if (activityType.equals('CallActivity') {enableDrillDown();}

      • If the current process instance has its parent process instance (from the activity tree), the Drill-Up button is enabled.
        • if (parentActivityInstanceId != null) {enableDrillUp();}

• • Sub-Process/Service Instance Rendering and Detail View
    • Capabilities
      • Get a sub process defintion XML through the Service Id and Process Instance association.
      • Use Camunda REST API, get/process-definition/{sub-id}/xml
      • Get the state of a sub-process instance from the activity-instances.
      • Use Camunda REST API, get/process-instance/{sub-id}/activity-instances
      • Render the BPMN XML with bpmn.io and places markers on top of it, and provides process/service instance detail views.
      • This widget uses the same code of the Process/Service Instance Rendering and Detail View with the sub-process instance id. 
      • The Drill-Up button will be enabled.
    • Design
      • This widget code is the same as the Process/Service Instance Rendering and Detail. 
      • The difference is the passed process id; the child process instance id will be passed.

• REST APIs for providing data to UIs
  • Capabilities
    • Provides REST services, by utilzing 1) Camunda REST APIs, such as BPMN XML string, process activity data, process variable, statistic, and 2) SO Request DB APIs for a service list.
    • Consolidate data responses from multple Camunda calls (both from process and history database) and feed them to UIs.
    • Use of HistoryService APIs, example, processEngine.getHistoryService().createHistoricProcessVariableQuery().xyz
    • Set a History level to ACTIVITY as a minimum; AUDIT (default) level for process variable tracing; currently, it is set to FULL
      • ProcessEngine processEngine = ProcessEngineConfiguration.createProcessEngineConfigurationFromResourceDefault().setHistory(ProcessEngineConfiguration.HISTORY_FULL).buildProcessEngine()
    • Provides workflow tracing (between parent-child workflows, interaction with other services; service task in and out); example,
      • processEngine.getRuntimeService().createExecutionQuery().processVariableValueEquals("serviceInstanceId", serviceInstanceId).singleResult();
    • Custom Query
      • Custom Query against History ACT_HI_DETAIL database table, as needed
    • Create a REST API, getServiceList with search criteria
      • Enhance MsoRequestsDbAdapter and MsoRequestsDbAdapterImpl to implement getInfraRequest(...) with additional parameters
      • invoke getInfraRequest(...) to collect service list data based on search criteria.
      • or use the RequestDB adapter directly.
  • Design
    • Write REST APIs in the new mso-api-handlers/mso-api-handler-monitoring
    • The following APIs are out-dated. Look at the SO-Monitor-APIs for Implementation detail, SO Monitoring API

1. getServiceList(Map searchCriteria...)                                                 // invokes new getInfraRequest(Map searchCriteria...)
   1. getServiceStatistic(Map searchCriteria, String status);            // get the statistic percentage for the status
2. getProcessInstanceId(String serviceInstanceId);                               // get the processInstanceId through GET /history/variable-instance/{id}
3. getprocessDiagramXML(String processInstanceId);                          // get a process instance, get a process definition id, and get diagram
4. getprocessInstanceDetail(String processInstanceId); 
5. getHistoryActivityInstance(String processInstanceId);                       // GET /history/activity-instance?processInstanceId={processInstanceId}
6. getProcessInstanceParent (String processInstanceId);
7. getProcessInstanceChildren(String processInstanceId);
8. getProcessTaskDetail(String processInstanceId, String taskId);
9. getProcessTaskType(String processInstanceId, String taskid);        // query if this task type is "Call Activity" for drilling down
10. 
    
    • The following diagram depicts the API interaction.

• Custom History Event Producer  (stretch goal)
  • Populate additional data in history with extensibility
  • It is an optional component. We plan to use it when the default Camunda process and history data is not sufficient.

• Component Build Dependencies
  • The following diagram depicts the component build dependencies.
  • Components can be built in parallel with a certain level of sequential integration. 
  • Each estimate includes code development and unit testing plus some component integration and integration testing (since there are API dependencies).

• Estimates (T-Shirt Size: M)
  
  
  • Note: 
    • It is a rough estimate - to be refined
    • Estimate includes development and unit testing time
    • As mentioned in the above section, the components can be built in parallel. 
    • 5 hours / day