2019-08-14 AAI Meeting Notes

Recording:

Attendees:

START RECORDING

1	Kubernetes job problem	Keong Lim	Message to discuss list: https://lists.onap.org/g/onap-discuss/message/18553 We have a Dublin environment that had a Cassandra problem which caused the graphadmin-create-db-schema job to fail repeatedly. Question 1. Is there an ONAP-built-in monitoring solution for the shared Cassandra database? The Cassandra problem was a combination of: - node-0 had Out of Memory error in JVM - node-1 had error in commit log processing - node-2 had Out of Disk Space error on filesystem However, Kubernetes showed node-0 and node-2 still "Running" and apparently healthy (zero restarts), with node-1 in CrashLoopBackOff (many restarts). The failed Cassandra nodes caused the graphadmin-create-db-schema job to fail (many restarts), which causes all other AAI pods to wait in Init state. The Cassandra problems have been manually fixed now, but should ONAP have a monitoring solution built-in to detect it? Question 2. How can we re-run that kubernetes job? Deleting the failed job pods cleaned up the list but did not prompt any new job pods to be created. Deleting the graphadmin pod caused it to restart and wait for the job completion, but did not trigger a new job to start. I think the job has now hit its backoff limit, so it no longer runs, even though we have since fixed the Cassandra problem. Can we reset some parameter so that it will finally run to completion? Is there an appropriate helm command or kubectl command to re-run the job? The rest of AAI pods are obviously waiting for that job to complete before they can progress out of Init state.
2	Alpine images	Dmitry Puzikov (Deactivated)	Jenkins jobs needed to create / push the common alpine image that the AAI containers will use, still need this completed before we merge the outstanding commits. AAI-2574 - Getting issue details... STATUS
3	Microservices enabled by default in aai/oom	James Forsyth	The ones in bold will be turned "off" by default: aai-babel aai-elasticsearch aai-modelloader aai-search-data aai-traversal aai-champ aai-gizmo aai-resources aai-sparky-be aai-data-router aai-graphadmin aai-schema-service aai-spike We will add are release note about how to turn off the ones in italics for users that want a true minimum set without the UI AAI-2548 - Getting issue details... STATUS AAI-2549 - Getting issue details... STATUS
4	El Alto Review	James Forsyth	JIRA Epic/Story count: Getting issues...
5	Janusgraph 0.2.3	James Forsyth	AAI-2420 - Getting issue details... STATUS James Forsyth is going to merge updates to JanusGraph, 0.2.0 → 0.2.3 in master, should be a drop in replacement.
6	Performance Issue	James Forsyth	FREEMAN, BRIAN D reported that when there are ~2400 vservers returned in a closed loop custom query, AAI takes minutes to respond. We've discussed paging previously, any other ways to optimize this? PUT 'https://aai.onap:8443/aai/v16/query?format=resource' {"query":"query/closed-loop","start":"/cloud-infrastructure/cloud-regions/cloud-region/CloudOwner/RegionOne/tenants/tenant/28481f6939614cfd83e6767a0e039bcc/vservers/vserver/6ad9eb6b-d7fc-46d6-8617-4c9b46e7308b"}
7	Spring Boot 2	James Forsyth	AAI-2111 - Getting issue details... STATUS aai-common:1.5.2 (not released yet) will be the el-alto library for springboot 1.5.x microservices. There needs to be some urgency to migrate the microservices to spring boot 2, since 1.5.3+ will not be backward compatible and so dependent applications might not have security updates if they stay on 1.5.2. 1.6.0 (Frankfort, currently master branch) aai-common will be springboot 2 so microservices must be updated by Frankfurt. Springboot 1.5.21 is the latest (and last) version in the 1.5.x spring boot train.
8	Upcoming R6 Functional Requirements	Benjamin Cheung	Release 6 (Frankfurt) proposed use cases and functional requirements Use Case Realization Call: July 24, 2019
9	Round trip reverse engineering of the AAI data model	James Forsyth	Moved to new meeting: AAI Information Model Reverse Engineering Meeting
10	Vetting DSL Queries	Manisha Aggarwal William Reehil	Will discuss strategies AAI will employ to protect itself from rogue queries.
11	AAI History	Philip Blackwood	Discuss and review Scenarios for using historical inventory and topology
12	Casablanca to Dublin Migration	Ankit Bhatt	3rd July: raised AAI-2507 - Getting issue details... STATUS based on Ankit's analysis that found the change in EdgeRules.
13	Build styles	James Forsyth	We have too many ways to build and deploy artifacts, which makes it difficult and time-consuming. Some of our artifacts are not backward compatible and we need to move everything up to released versions.
14	Alternative meeting system		Just in case Zoom problems continue, try this instead: https://meet.jit.si/onapaai
15	New UI Features / Historical Tracking	William Reehil Manisha Aggarwal	The AT&T team has done an exciting POC at a sprint-a-thon event that they would like to share with the community. ATT wants to contribute additional UI views to ONAP, want to discuss path forward
16	Run AAI containers as non-root user	James Forsyth	https://wiki.onap.org/display/DW/Best+Practices The Docker and Kubernetes engines may run as root until such time as the products support non-root execution. Applications may run as root within a container. The process ID of a container must not run as the root ID with the exception of containers supporting ONAP features that require the container to run as the root ID. Containers may run with root privileges. Project containers that run as the root ID must document this in the release notes along with the functionality that requires the container to run as the root ID. AAI-2172 - Getting issue details... STATUS
17	Container Images	James Forsyth	Container Image Minimization Guidelines James Forsyth will ask integration team if we can use the AAI tenant space in windriver environment
18	Bug Review		JIRA Issue Count (in AAI project): Getting issues... JIRA Issue Count (talking about AAI): Getting issues...
19	Gerrit Review		https://gerrit.onap.org/r/#/q/projects:aai
20	Sonar Review		https://sonar.onap.org/projects?search=aai&sort=coverage
21	Jenkins CLM Review		https://jenkins.onap.org/view/CLM/
22	Code Coverage Status	James Forsyth	code coverage to 55% for Dublin AAI-2219 - Getting issue details... STATUS Champ slipped below 55% with latest commit. Tian Lee will do babel, model-loader, champ, gizmo, event-client, spike, rest-client, validation Arul Nambi will do sparky-be, search-data-service, data-router, logging-service, router-core James Forsyth create ticket for El Alto for sparky, rolling back because of sonar failure Venkata Harish Kajur will check aai-common, resources, traversal, graphadmin, cacher, schema-service (which probably already have the setting) Add thresholds to all repos that are above 55% so it doesn't happen again For setting the threshold on each repo, set it to the current percentage, so if it's is at 62.7%, set the threshold to 0.62 <jacoco.line.coverage.limit>0.68</jacoco.line.coverage.limit> Sonar job for Dublin has been changed to using line coverage. <execution> <id>default-check</id> <goals> <goal>check</goal> </goals> <configuration> <dataFile>${project.build.directory}/coverage-reports/jacoco.exec</dataFile> <rules> <!-- implementation is needed only for Maven 2 --> <rule implementation="org.jacoco.maven.RuleConfiguration"> <element>BUNDLE</element> <limits> <limit implementation="org.jacoco.report.check.Limit"> <counter>LINE</counter> <value>COVEREDRATIO</value> <minimum>${jacoco.line.coverage.limit}</minimum> </limit> </limits> </rule> </rules> </configuration> </execution>
23

Attendees:

Open Action Items

New Action items