CD Health Check Kibana Dashboard Setup

The CD stack sends the Jenkins console output directly to Elasticsearch using the logstash plugin. For this reason scripted fields are used in Kibana to parse and derive fields on load time instead of using logstash.

The instructions to configure these fields and import the dashboard are as follows:

1. Update mappings to support using message.keyword field. 
   To do so on the Elasticsearch (ES) node run the following curl command included in the attached file:
   jenkins_elasticsearch_mappings.txt

2. ^{Add the following parameter to the elasticsearch.yml config file:}

   ^{script.painless.regex.enabled: true}

3. Restart ES

4. In Kibana create the index patter "logstash-jenkins". Set the timestamp field to "@buildTimestamp"

5. For that index pattern navigate to the scripted fields tab. Create a scripted field named "healthcheck". Language is painless. Type is string. Script is as follows:
   
   

   ^{def msg = doc['message.keyword'].value;
   if (doc['message.keyword'].value != null) {
   if (doc['message.keyword'].value =~ /Health Check/) {
   if (doc['message.keyword'].value =~ /PASS/) {
   return "PASS";
   }
   else {
   return "FAIL";
   }
   }
   }}

6. Create another scripted field named "component". Language is painless. Type is string. Script is as follows:
   
   

   ^{if (doc['message.keyword'].value != null) {
   if (doc['message.keyword'].value =~ /Health Check/) {
   def m = /^(.+?)(API)?\sHealth Check.*$/.matcher(doc['message.keyword'].value);
   if ( m.matches() ) {
   return m.group(1)
   }
   else {
   return "unknown";
   }
   }
   }
   return null;}

7. Import the following visualizations to Kibana:
   jenkins cd visualizations.json

Import the following dashboard to Kibana:
jenkins cd dashboard.json

Troubleshooting:

If you get the following error in Kibana:

"[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_per_minute] setting","bytes_wanted":0,"bytes_limit":0}}}}]},"status":400}

Execute the following request on the Elasticsearch VM: