POMBA Reporting Ideas

1 1. Upgrade of ELK
2 2. Data Enrichment
3 3. Data Preparation in Local Lab
4 4. Dashboard Ideas

LOG-395: Spike: Platform Knowledge on ReportingClosed

1. Upgrade of ELK

amdocs Bath team (in charge of search-data-service, @Colin Burns) is planning to upgrade elasticsearch to 6.1.2 by the end of June.

Current ELK versions: elasticsearch 2.4, kibana 4.6 (no logstash is being used)
To create the dashboards with enhanced Kibana features, upgrading to version 5.5 or above for all ELK stack is desired. (note: ONAP Logging project is using 5.5)
Upgrade from 2.x to 5.x or above requires "Full Cluster-restart Upgrade".
search-data-service should reflect this upgrade
- deploy/configure the right versions
- potentially update relevant API methods for the elasticsearch data management.

Specifically for POMBA use, we could provide:

Automatic deployment of a separate kibana (version 6.1.2) for POMBA (currently, it is manually installed) with all required configuration (kibana.yml, index pattern creation) and pre-installation of the POMBA dashboards
(Q) Would it be a good idea to use the kibana provided in onap-log pod?
- pros: no redundant install of kibana, an integrated place for all views
- cons: dependency on the onap-log (e.g., version), getting complex with all different types of dashboards
- to-do: only configuration and import of POMBA dashboards

2. Data Enrichment

Common Model Data Fields

The data fields that are currently available and will be available from the POMBA context builders are listed up in the page, Context Builders Mapping to Common Model (see the java field name).

Specifically the reporting should deal with the highlighted fields for the network discover as developing the Network Discovery Context Builder.

Data Enrichment

The following discusses any enrichment opportunities of the audit validation/violation data being pushed to elasticsearch. Most jobs could be done in the data-router micro-service code instead of using logstash.

Notice the two boxes below demonstrating a sample validation and violation event currently stored in ES, which will be the data source for the Kibana dashboards.

The violation event need to add more fields from the field "violations" available in the validation info including: modelName, violationDetails, indicator for manual vs automatic trigger
The field violationDetails (which would tell the exact discrepancies; see the sample event below inside the '?violations') need to be parsed and stored in separate fields. Such nested data cannot be directly used in the kibana visualizations (? mark indicates it).
We could further parse out the ONAP components involved with the violations to see the violation stats factored by component (from violationDetails)
"Audit duration" stats would be useful? time taken for the auditing itself (from trigger to result).
Any other meta-data that would be useful? e.g., who invoked the validation (user, dept)

3. Data Preparation in Local Lab

To create proper dashboards, we need to populate a certain amount of audit results data in our local lab environment which consist of various types of validation and violation cases. We want the data to be reflecting more likely the production reality that would help create more useful dashboards.

Approach 1: Copy the audit-results from IST to dev lab

Script A runs to collect a list of info that will be used as input parameters for the audit requests: serviceInstanceId, modelINvariantId, modelVersionId, customerId, serviceType
Script B runs to send audit requests with the data collected above: need to properly distribute the requests over time to make it more realistic
Manually collect the elasticsearch dump (which will contain all the audit validation/violation events) and import it to the elasticsearch in the dev lab

Approach 2: Component-level copy from IST to dev lab

Script X runs to GET all necessary info from each component of interest in the IST or production
Script Y prepares APIs to PUT the info into the components in the dev lab
Run Script A
Run Script B

After that, as necessary, we could manipulate the data to generate many different types of violations

Manually update the data in some components to generate special violation cases

4. Dashboard Ideas

The visualizations and dashboards will need to be designed and created according to the current and any potential use cases of the POMBA services - what the users want/need to check, how the system could help improve the whole platform integrity. We want the POMBA reporting to be informative, insightful, and intuitive from the user's perspective.

Challenges

We have created several validation rules but do not know all the rules to be created in the future by the users in the production. That means most likely we can create and provide some high-level dashboards - for the specific rules and details, we could only provide some sample dashboards to give an idea so that how the end users could create their own dashboards customized for their use cases.
For the Network Discovery, what specific audits will be executed and what kinds of audit results are expected

Dashboard List

(Note) One dashboard type could need multiple dashboard pages depending on the amount of visualizations.

	Dashboard Type	Description (What To Want to See)	Required Information To Show (Visualizations)

	Dashboard Type	Description (What To Want to See)	Required Information To Show (Visualizations)
1	Overall Audit Monitor	As a general admin, I want to see the whole platform integrity - health status in terms of all validation rules configured	validation total count for the specified time period violation total count for the specified time period validation count over time (trend) violation count over time (trend) violation count by component involved violation count by validation rule violation count by rule severity audit KPI trend: daily violation metric against total validation count (weighted measure considering the severity level) validation list violation list
2	Overall Audit Analysis	What kind of validations mostly executed against which models What kind of violations mostly occur in which components	validation stats (e.g., validationId or serviceInstanceId count) by modelInvariantId and modelVersionId which models made how many violations of which type violation stats by validation model, rule, etc. provide the views from the perspective of each component
3	Individual Audit Analysis	Given a validation job, the user wants to see and quickly recognize all relevant violations detected by POMBA	validation details and related violation details on the same page (stretch) given the same type of validation job, can we retrieve the historical violation results to give an idea if this is really unusual case or it used to happen?
4	Violation Summary Report	Provide a table of violation cases for any potential fixes	list of up-to-date violations with detail info helping to raise/fix any issues
5	Audit History	For the exactly same validation request, check the result change over time (invalidity could be decreasing with fixes, or a valid case before could turn to invalid?) For the same validation category (e.g., with the same rule and model id, component set?), the user wants to be ensured that the violation has been fixed and gone now, and wants to get an insight on how much the POMBA helps improve the overall system integrity	violation stats over time for the same validation job (e..g, 3 violations reduced to 2 and then 0 now) violation stats over time for the same validation category (e..g, 10 violations reduced to 2 now) KPIs showing the violation resolution trend (e.g., how many violations have been fixed on daily basis)

Features to support as necessary

Where necessary, provide links to switch the dashboards back and forth: e.g., from the violation page to the page displaying its validation info
Color coding for the critical violations