Healthcheck tests evaluation

The goal of this page is ro review the existing healthcheck tests and evaluate their maturity level in order to ask for improvement if needed for G and later releases.

In fact the healthcheck tests are not homogeneous. A good healthcheck shall be trustable, for instance if the contact between a component and it DB is failing is shall report FAIL and not PASS just because one web server is answering.



Core components



Component

Nb of pods

Description

Logs

(for troubleshooting)

Comments

Integration team view

(1: weak trust - 5 full trust)

Component

Nb of pods

Description

Logs

(for troubleshooting)

Comments

Integration team view

(1: weak trust - 5 full trust)











MR







dmaap

  • onap-dmaap-bc-6bfbf489fb-v7qv9

  • onap-dmaap-dr-db-0

  • onap-dmaap-dr-db-1

  • onap-dmaap-dr-node-0

  • onap-dmaap-dr-prov-5f69668996-hc8pv

  • onap-sdnc-dmaap-listener-754b485cbd-qvt2b



2 tests in health

1) Basic DMAAP Message Router Health Check (/topics)

test a topic is well received (TEST_TOPIC_ACL)

2) Basic DMAAP Message Router PubSub Health Check (/events/TEST_TOPIC/g1/c4?timeout=5000)

Received response from message router ["{\"test\":\"success\",\"timestamp\":\"${timestamp}\"}"]



1 in postinstall

1) Basic DMAAP Message Router ACL Update Test

Runs MR create topic and update producer credentials

  • onap-dmaap-: log4j:WARN No appenders could be found for logger (org.onap.dmaap.dbcapi.aaf.DmaapPerm).
    log4j:WARN Please initialize the log4j system properly.
    log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

  • onap-dmaap-dr-node: ?

  • onap-dmaap-dr-prov-: OK

  • onap-dmaap-dr-db: OK but not real LOG level

  • onap-sdnc-dmaap-listener: OK but set at DEBUG











aai

  • onap-aai-babel-f69d8fdc8-nds4h

  • onap-aai-data-router-699bcc49c5-86tt7

  • onap-aai-elasticsearch-5894f8fbdd-n6sln

  • onap-aai-f68c4bf85-v6m5n

  • onap-aai-graphadmin-55bcc9654d-9hb55

  • onap-aai-modelloader-75556cd68-8jglx

  • onap-aai-resources-5674d9c49f-ngk5f

  • onap-aai-schema-service-7bcf64bd9-4fdpw

  • onap-aai-search-data-b68cbc498-vsc4l

  • onap-aai-sparky-be-868d55b4-rsj5s

  • onap-aai-traversal-58bc54c4f4-s4fq7

  • onap-pomba-aaictxbuilder-598d7b7dfb-77vp4



1 test in health

1) 1 global answer from A&AI

{"responseMessages":{"responseMessage":[{"messageId":"INF0001","text":"Success X-FromAppId=%1 X-TransactionId=%2 (msg=%3) (rc=%4)",

"variables":{"variable":["robot-ete","51707c75-ac59-49e7-8181-f6979db0dfa8","Successful health check:OK","0.0.0002"]}}]}}



1 test in postinstall

1) Basic AAI Service Design Models Size Test (/aai/v16/service-design-and-creation/models/)

Check that the number of models is > min value













portal

  • onap-portal-app-664dd47bc6-ldp5h

  • onap-portal-cassandra-8456f77bd6-4bj5v

  • onap-portal-db-84c9d4d7fc-rdx8z

  • onap-portal-sdk-55cb6cb779-xbkwc

  • onap-portal-widget-6f7ddf4bc4-dxjxz

  • onap-portal-zookeeper-7b65474bc-x272x

1 healthcheck test retrieving  BE, FE and DB status

{"healthCheckComponent":"BE","healthCheckStatus":"UP", "version":"", "description":"OK", "hostName":"onap-portal-app-664dd47bc6-ldp5h","ipAddress":"10.233.75.218","dbClusterStatus":"","dbPermissions":""},{"healthCheckComponent":"FE","healthCheckStatus":"UP","version":"","description":"OK","hostName":"","ipAddress":"","dbClusterStatus":"","dbPermissions":""},{"healthCheckComponent":"DB","healthCheckStatus":"UP","version":"","description":"OK","hostName":"","ipAddress":"","dbClusterStatus":"","dbPermissions":"OK"}]"}





Only 1 hostname mentioned



Possible to add another test healthportalapp to have Selenium tests on the GUI









sdc

  • onap-cds-sdc-listener-58944c9d89-v7fd4

  • onap-pomba-sdcctxbuilder-78755486f9-5j4sf

  • onap-sdc-be-59ddc8999b-gmlcw

  • onap-sdc-dcae-be-64795bdfc5-zszvx

  • onap-sdc-dcae-dt-5798d568cd-f8vh6

  • onap-sdc-dcae-fe-5c9ff8c47-2jvxd

  • onap-sdc-dcae-tosca-lab-7fcddbc459-dhcqb

  • onap-sdc-es-8898c646d-g522p

  • onap-sdc-fe-78f86d4bf7-d9rm6

  • onap-sdc-kb-6c64c9dd7c-gvzkn

  • onap-sdc-onboarding-be-5c84f754bc-s277l

  • onap-sdc-wfd-be-56c9c98494-pbq8f

  • onap-sdc-wfd-fe-56587d8dd4-6pv4w

  • onap-so-sdc-controller-5bb4f7c545-s4s7m

SDC Health Check (/sdc1/rest/healthCheck)

complete status

{"sdcVersion": "1.5.2",

"siteMode": "unknown",

"componentsInfo": [

{"healthCheckComponent": "BE",

"healthCheckStatus": "UP",

"version": "1.5.2",

"description": "OK"},

{"healthCheckComponent": "JANUSGRAPH",

"healthCheckStatus": "UP",

"description": "OK"},

{"healthCheckComponent": "ES",

"healthCheckStatus": "UP",

"version": "1.5.2",

"description": "OK"},

{"healthCheckComponent": "DE",

"healthCheckStatus": "UP",

"description": "OK"},

{"healthCheckComponent": "CASSANDRA",

"healthCheckStatus": "UP",

"description": "OK"},

{"healthCheckComponent": "ON_BOARDING",

"healthCheckStatus": "UP",

"version": "1.5.2",

"description": "OK",

"componentsInfo": [

{"healthCheckComponent": "ZU",

"healthCheckStatus": "UP",

"description": "OK"},

{"healthCheckComponent": "BE",

"healthCheckStatus": "UP",

"version": "1.5.2",

"description": "OK"},

{"healthCheckComponent": "CAS",

"healthCheckStatus": "UP",

"version": "2.2.14",

"description": "OK"},

{"healthCheckComponent": "FE",

"healthCheckStatus": "UP",

"version": "1.5.2",

"description": "OK"}]},

{"healthCheckComponent": "DCAE",

"healthCheckStatus": "UP",

"version": "1.3.2",

"description": "OK",

"componentsInfo": [

{"healthCheckComponent": "BE",

"healthCheckStatus": "UP",

"version": "1.3.2",

"description": "OK"},

{"healthCheckComponent": "TOSCA_LAB",

"healthCheckStatus": "UP",

"version": "1.3.0",

"description": "OK"},

{"healthCheckComponent": "FE",

"healthCheckStatus": "UP",

"description": "OK"}]},

{"healthCheckComponent": "FE",

"healthCheckStatus": "UP",

"version": "1.5.2",

"description": "OK"}]}













sdnc

  • onap-pomba-sdncctxbuilder-85969b6549-qhh27

  • onap-sdnc-0

  • onap-sdnc-ansible-server-55bcb99f94-qhhvl

  • onap-sdnc-dgbuilder-78699dc768-5p8c8

  • onap-sdnc-dmaap-listener-754b485cbd-qvt2b

  • onap-sdnc-ueb-listener-557b64d794-q45s8

  • onap-so-sdnc-adapter-685c66486d-nzlcx

  • onap-vfc-zte-sdnc-driver-5bfcd8db6c-r8dzd

1 test in health

1) SDNC Health Check (/restconf/operations/SLI-API:healthcheck)

wait for  200 OK from the SLI-API













so

  • onap-so-bpmn-infra-5b7fc8dbfd-cgtwc

  • onap-so-catalog-db-adapter-5486bbcdc7-7ssbk

  • onap-so-cbdf77b84-ptvrq

  • onap-so-monitoring-64c56566f6-td8tp

  • onap-so-openstack-adapter-84648b89b9-hcwkb

  • onap-so-request-db-adapter-695fb8c565-t2tmb

  • onap-so-sdc-controller-5bb4f7c545-s4s7m

  • onap-so-sdnc-adapter-685c66486d-nzlcx

  • onap-so-vfc-adapter-8758898db-zc2tl

  • onap-so-vnfm-adapter-6d66d5bd5-xsnvx



1 global test in health including 8 requests

SO Health Check

1) so

2) so-sdc-controller

3) so-bpmn-infra

4) so-catalog-db-adapter

5) so-openstack-adapter

6) so-request-db-adapter

7) so-sdnc-adapter

8) so-vfc-adapter

9) so-vnfm-adapter



all pods covered









cassandra

  • onap-cassandra-0

  • onap-cassandra-1

  • onap-cassandra-2

















maria-galera

  • onap-mariadb-galera-0

  • onap-mariadb-galera-1

  • onap-mariadb-galera-2