2018-12-20 AAI Developers Meeting
Attachments (including meeting recording)
Chat Server
private group aai-dev on Rocketchat server: http://onap-integration.eastus.cloudapp.azure.com:3000/group/aai-dev
Agenda Items
START RECORDING
Title | Responsible | Status | Last discussed | Notes | |
---|---|---|---|---|---|
1 | CCVPN Extension Dublin | IN PROGRESS | 20th Dec 2018 | During the VF2F we have presented the CCVPN Use Case extension for Dublin Release. In particular, we have defined the E-LAN Service (EP-LAN. EVP-LAN) sub-Use Case. More details can be found at this page (the presentation given to the Use Case SubCommittee and during the VF2F can also be found in that page). We have also created 2 additional pages where we describe the: In the former page, we have identified Option 2 (for Service Creation) and Option A (for Service Modification) as the “ideal” implementation. The Impacts in the latter page are based on such an assumption. We would be grateful if you could help us with the assessment for A&AI and if you could confirm the project commitment to CCVPN. 14th Dec: Regarding the Options and Impact assessment linked above:
As far as I know, the ServiceComposite pattern hasn't even exited the Modelling team's discussions for R3 or R4 workshops Hui Deng : This would seem like a stretch goal given the impacts on all the downstream clients of AAI, if it would require reworking any existing schema elements. However, if you are looking for a zero-impact method of implementation, the EdgeRules already contain service-instance-to-service-instance relationship with label "org.onap.relationships.inventory.ComposedOf", so AAI can do the composite data with this relationship.
| |
2 | Multi-CPU architecture images | IN PROGRESS | 20th Dec 2018 | I am writting you because of your comment here: We have the following issue: https://jira.onap.org/browse/INT-759 Briefly put, do you think you can help us with the Dockerfile you use to build aai-common docker image, for Casablanca and maybe master if it differs ? Best Regards, Paul 14th Dec: See also 20th Dec: Clarified that the aaionap/aai-common docker image is not the same as the aai/aai-common gerrit repository, despite the shared name.
| |
3 | CIA images for AAI | DONE | 20th Dec 2018 | Adolfo Perez-Duran (Deactivated) / Adolfo Perez-Duran raised - INT-769Getting issue details... STATUS and its many sub-tasks. Most, but not all, of AAI container code is Java, so is it really feasible to be both platform-agnostic and CPU-agnostic? What is the actual range of platforms and CPUs targeted? 20th Dec: Docker has "fat manifests" that list out each image per-CPU architecture under the same name | |
4 | HAproxy in OOM | IN PROGRESS | 1st Nov 2018 | Why is the pod for HAproxy not named (hard to figure out that there is a proxy), unsure how it is logging and where James Forsyth creates JIRA tasks to 1. to have the pod named 2. add logging to the proxy - AAI-1807Getting issue details... STATUS - AAI-1810Getting issue details... STATUS | |
5 | AAI-DMaaP event problems? | OPEN | Just had a look at this DMaaP bug - DMAAP-896Getting issue details... STATUS which shows up two AAI-related exceptions when getting events: INFO org.onap.dmaap.dmf.mr.service.impl.EventsServiceImpl - KafkaConsumer is not safe for multi-threaded accesson AAI-EVENT datarouter ****** datarouter from Remote10.42.133.135 INFO org.onap.dmaap.dmf.mr.service.impl.EventsServiceImpl - [AAI-EVENT/datarouter/datarouter] INFO org.onap.dmaap.dmf.mr.service.impl.EventsServiceImpl - KafkaConsumer is not safe for multi-threaded accesson champRawEvents spike ****** spike from Remote10.42.121.41 INFO org.onap.dmaap.dmf.mr.service.impl.EventsServiceImpl - [champRawEvents/spike/spike] Anyone else seen or reported similar problems in AAI? | ||
6 | aai-cassandra performance issues | Keong Lim | OPEN | Michael O'Brien has documented performance issues in aai-cassandra:
hector has discovered that the stress test jar (liveness probe?) in aai-cassandra is hammering the cpu/ram/hd on the vm that aai is on - this breaks the etcd cluster (not the latency/network issues we suspected that may cause pod rescheduling) Is there something that should be tweaked in AAI config? Or documentation on the recommended setup to run the VM? I'll come to the next AAI meet (conflicts with pomba meet) - | |
7 | get notified of AAI Casandra issues automatically | OPEN | Mike Elliott wrote in OOM Meeting Notes - 2018-12-5 f. AAI team wanted to get notified of AAI Casandra issues automatically Keep an eye out for new issues! | ||
8 | Purpose of fields in AAI | OPEN | Dénes Németh wrote in - AAI-1104Getting issue details... STATUS : In think it would be good to answer what is the meaning of the field (collection of PEMs of the CA xor URL) Questions: 1. Is AAI intended to strictly prescribe how the fields are used and what contents are in the values? Even if (1) is true, AAI is not really in any position to enforce how clients use the data, so really (2) is always true and we need to consult the original producers of the data and the ultimate consumers of the data to document their intended meanings. How do we push to have documentation on the purpose and meaning of the fields in AAI? Where does all this documentation go? Should the documentation be backed up by validation code? See also discussion about AAI in 2018-11-28 ExtAPI Meeting notes 29th Nov: Started on new wiki page AAI Schema Producer-Consumer Pairings | ||
9 | Postel's Law / Robustness Principle for AAI data inputs | Keong Lim | OPEN | 6th Dec 2018 | While helping UUI team to debug a family of related issues such as - AAI-1915Getting issue details... STATUS , found that their JSON request bodies had inserted extra lines that were not acceptable to AAI, e.g. Not Accepted Input { ext-aai-network: { "aai-id": "VDF", "schema-version": "version-1", "resource-version": "1542082337501" } } vs Expected Input Format { "aai-id": "VDF", "schema-version": "version-1", "resource-version": "1542082337501" } Initially, I thought this could be due to XML-to-JSON translation error, but it could also have been a copy-paste from the output of a GET, e.g. > GET /aai/v14/network/ext-aai-networks HTTP/1.1 { "ext-aai-network": [ { "aai-id": "createAndDelete", "schema-version": "version-1", "resource-version": "1542247826990" }, { "aai-id": "aaiId-2", "schema-version": "version-2", "resource-version": "1542029867153" } ] } In the spirit of Postel's Law ("be liberal in what you accept"), could/should AAI accept both variations of the input data above? There could be additional validation that the element name inside the request body matches the element name in the URL of the API (similar validation is already performed for the key ID value). It would also allow for simple methods of data transfer/migration, where AAI output is directly accepted as AAI input. 7th Dec: As per Jimmy's comment below, the difference between the "Not Accepted Input" and "Expected Input Format" is the behaviour controlled by "SerializationFeature.WRAP_ROOT_VALUE" as described in:
|
10 | Copyright license header restricted to 1 company | OPEN | In the copyright /** * ============LICENSE_START======================================================= * org.onap.aai * ================================================================================ * Copyright © 2017-2018 AT&T Intellectual Property. All rights reserved. If there is a company other than AT&T the build fails saying the license header is wrong | ||
11 | Return codes and messages for WS | OPEN | Is there a guide for the description of the error message and the error codes? How are new error states (message + code) added? | ||
12 | OOM Artifacts | OPEN | Some of our top level OOM deployment artifacts are not unique (i.e. don’t take namespace into account as all other deployables), is that intentional? | ||
13 | AAI test data bootstrap | Keong Lim | OPEN | Looking at AAI usage in OOF - HPA guide for integration testing by Dileep Ranganathan, wondering whether there is a better way to bootstrap AAI test data?
Similarly, Scott Seabolt and J / Joss Armstrong wrote for APPC Sample A&AI Data Setup for vLB/vDNS for APPC Consumption and Script to load vLB into AAI:
Similarly:
Related - AAI-1948Getting issue details... STATUS on the brittleness of the ReadTheDocs links to data files. One for VIM: How-To: Register a VIM/Cloud Instance to ONAP and - AAI-1928Getting issue details... STATUS Potential issues:
| |
14 | AAI too slow for OOF/HAS | Keong Lim | OPEN | Under OOF Homing and Allocation Service (HAS) section, Dileep Ranganathan wrote about Project Specific enhancements:
See also - OPTFRA-268Getting issue details... STATUS / - OPTFRA-291Getting issue details... STATUS Similarly to the "AAI too slow for Holmes" item below, this introduction of extra caching of AAI data is a worrisome development and sad indictment of the performance of the system architecture. What can we do about this? Would the AAI Cacher - AAI-1337Getting issue details... STATUS help to improve performance? | |
15 | MultiCloud usage of AAI for HPA telemetry/time-series data to OOF | OPEN | Bin Yang and Lianhao Lu (Deactivated) wrote in - MULTICLOUD-274Getting issue details... STATUS : HPA telemetry data collection and make it persistent in A&AI, from which OOF can leverage during its decision making process.and
and The reason why we propose here is that VES mechanism doesn't store the telemetry data into A&AI. And OOF now can only get those kind of data from A&AI. Some concerns:
| ||
16 | Relationship between POMBA Common Model and AAI schema? | Keong Lim | OPEN | Under POMBA Common Model, Geora Barsky and Sharon Chisholm discuss objects that seem to overlap the AAI schema:
Curious to know what is the relationship between the POMBA Common Model and the AAI schema? There seems to be an overlap in these object definitions and relationships. Is POMBA a potential client for AAI schema services? 15th Nov: As per Geora's and Sharon's comments: POMBA is a client of AAI APIs. It retrieves certain objects from AAI and transforms it into POMBA COMMON model which is aimed to represent flat structure of service instance representation. and The POMBA model enables us to normalize data from different data sources to facilitate auditing. To save time and work, we often use A&AI as our starting point, but conceptually, this model can have more than is currently or makes sense to have in A&AI. So, POMBA needs to evolve independently of AAI, even though it has common ancestry. As we are having discussions about the AAI Schema Services with a view to future dynamic schema updates, e.g. via SDC modelling, we will need to be aware of the downstream impacts of such a change. I think this needs to be added to the use cases for AAI Schema Services (and GraphGraph?) behaviours. | |
17 | AAI-BBS for Dublin | OPEN | Started collating proposed changes in wiki AAI-BBS Proposals for Dublin Release Investigating code proposal 1: In AAI, when the "orchestration-status", "service-instance-location-id", "input-params" for a Service Instance changes, notify ExtAPI so it does not need to poll AAI Is it already covered by spike? If so, how to consume the events? If not, how to make AAI publish new events to DMaaP? | ||
18 | Orchestration Scenarios for VNFs | OPEN | Comments on Orchestration Scenarios related to AAI: Viswanath Kumar Skand Priya / kspviswa said:
and Fernando Oliveira replied: For the first question: I think that A&AI needs to maintain the VNF instance ↔ VNFM instance and the NS instance ↔ NFVO instance relationship for subsequent life cycle operations, i.e. a scale or heal operation. The path would be something like Event (VNF Instance, Busy) → DCAE (policy for VNF instance) → Policy Evaluation (VNF instance, Scale-out) → SO (VNF instance, Scale-out) → A&AI (find VNFM instance for the VNF instance) → SO (VNF instance, VNFM instance, Scale-out) → SOL003 Adapter (VNFM instance, VNF instance, Scale-out) → VNFM instance (VNF Instance, Scale-out). As I understand, ESR has "esr-vnfm-list", which has an "esr-vnfm", which has "esr-system-info-list", which has "esr-system-info", which has a "relationship-list" that can contain relationships to "generic-vnf" and other AAI objects. The "generic-vnf" object also contains "self-link", "ipv4OamAddress", "ipv4OamGatewayAddress", etc, which links the AAI object back to its "source-of-truth" external-system. Is there some new data, new schema or new API that is required on top of this? Fernando Oliveira; Apologies for my lack of knowledge, but a few comments:
Bo Lv can comment more on the current ESR capabilities, but I believe there are only 3 kinds of systems so far: EMS, VNFM and third-party SDNC. ESR could be extended to handle VNFO as another kind of system. Fernando Oliveira : I created JIRA stories:
for various parts of the scenario. | ||
19 | Modeling team R4 discussion, including extra AAI attributes in a model-driven way | IN PROGRESS | 15th Nov 2018 | Modelling team having Service Instance thoughts by Chesla Wechsler, which will affect AAI schema. Also referred from comments on ONAP R4+ Service Modeling Discussion Calls 9)“vhn-portal-url”?“Bandwidth”,"QoS","SLA",etc, attribtutes that not all the services need but still need to be stored in certain service instance: stored as a schemaless field on the service-instance vertex (Chesla will follow up) (my concerns: according to the call, is that ok if we set a "global-type of service" and a "customized-type of service", then mapped it with internal descriptor, and A&AI's model only stores global type in service instance's schema, but stores the customer-faced attributes of service in a schemaless way? Chesla Wechsler Kevin Scaggs Andy Mayer) See also Modeling 2018-11-13 The service-instance already uses a "metadata" relationship, which can store an arbitrary list of key-value pairs, but perhaps AAI should extend the use of the "properties" element, which is also an arbitrary list of name-value pairs or the "extra-properties" element, which is also an arbitrary list of name-value pairs. 15th Nov: Having seen Chesla's presentation, it should be called "Model-driven schema" rather than "schemaless" behaviour, since the idea is that the changes are controlled by SDC modelling. Seems aligned to the eventual goal in AAI Schema Service Use Case Proposals and AAI Schema Service. | |
20 | Jackson Replacement | ON HOLD
| 1st Nov 2018 | Security subcommittee has recommended teams move away from jackson, and will be presenting alternatives and asking for an assessment from each project. Our team will need to do an analysis - this would not be trivial, especially given how many of our repos are impacted. As of now, this would be a very high LOE for the team, we need to understand what the recommendation from the SECCOM is before we can provide better details on what the LOE would be. Updated: Using Google gson vs FasterXML Jackson 10th Oct: Present to Seccom meeting 15th Oct: Present to PTL meeting 31st Oct: Debatable whether the cost of swapping Cassandra and changing code is worth the benefit of removing Jackson from the vulnerabilities list. On-Hold until James Forsyth consults with other PTLs: PTL 2018-11-05 10th Dec: Mentioned as S3P in vF2F meeting: ONAP Project Developers Event, Dec 10 - 12, 2018, (Virtual Webinars) | |
21 | AAI too slow for Holmes | IN PROGRESS | 1st Nov 2018 | Guangrong Fu mentioned AAI in Baseline Measurements based on Testing Results:
The problem for caching is how to know when to update the cached data. Even though the access time may be fast for Holmes, the risk is using out-of-date data, so the correlations will be wrong anyway. Also, duplicating the AAI data outside of AAI is probably a bad architectural decision. Making AAI faster for these use cases would be better. Has there been a performance analysis of where the time is spent? Could it help to use ElasticSearch (e.g. as in sparky)? Should Holmes have a batch interface to get more AAI data in fewer calls? Or a better correlation API that results in fewer calls? 31st Oct: https://lists.onap.org/g/onap-discuss/topic/27805753 1st Nov:
Would the AAI Cacher - AAI-1337Getting issue details... STATUS help to improve performance? | |
22 | 2 Types of logging in A&AI WS | ON HOLD | 1st Nov 2018 | There are 2 types of logging in the services
Is that correct? Shouldn't there be just 1 type? 1st Nov: After Casablanca release investigate logging guidelines and figure out what library to use in order to unify logging within A&AI 26th Nov: See also ONAP Application Logging Specification - Post Dublin 29th Nov: how does this fit with - LOG-877Getting issue details... STATUS ? | |
23 | Disable unused web services (see also Helm chart requested values) | IN PROGRESS | 20th Dec 2018 | Could we disable unused (i.e. not integrated) A&AI web services, so that the deployment is faster and the resource footprint is smaller? e.g. Champ (any other ws?) Motivation: Decrease the resource footprint for A&AI (ONAP) deployments Idea: we could support 2 different deployments 1. full (normal) deployment and 2. barebones deployment. The point of the "barebone" deployment would be to deploy only the essential services necessary for proper functioning of A&AI (leaving out services like cacher, sparky, graphadmin, having 1 cassandra node instead of 3 or 5 etc). In order to reduce hardware/cloud costs (mainly the memory footprint) it could be beneficial to support a minimalistic A&AI deployment. 1st Nov: Venkata Harish Kajur Former user (Deleted) - investigate how to disable/enable charts in A&AI so we can create a core group of pods which handle the use-cases and than extended group will all the services. Consider a group of unused/unintegrated services (like Champ). Consider other possible groups (like GUI?)
| |
24 | AAI Champ | IN PROGRESS | 1st Nov 2018 |
| |
25 | Dublin 5G Use Case | IN PROGRESS | 25th Oct 2018 | Dublin AAI changes in support of 5g use cases. Link for presentation: 5G - PNF Plug and Play (Casablanca carry-over items)
| |
26 | AAI Tutorial updates | IN PROGRESS | 29th Nov | Following updates for AAI Developer Environment Setup - Casablanca need to update the link in step 14: Tutorial: Making and Testing a Schema Change in A&AI Harish will update the documentation - the section Tutorial: Making and Testing a Schema Change in A&AI Need to distinguish on the wiki tutorials for:
James Forsyth will check if we still need to support HEAT deployment for Dublin | |
27 | Schema Service | IN PROGRESS | 29th Nov 2018 | Discuss about the Schema Microservice 11th Oct: Suggested Use Case Proposals for Dynamic AAI Schema Changes based on CCVPN usecase experience 1st Nov: William Reehil Robby Maharajh Venkata Harish Kajur will review requirement updates and research the open questions so that a final draft can be prepared for implementation 8th Nov: Added AAI Schema Service Use Case Proposals for discussion and planning 15th Nov: Reviewed the Requirements section in AAI Schema Service again. | |
28 | AAI GraphGraph | IN PROGRESS | 29th Nov 2018 | William Reehil wrote introduction to (proposal for?) A&AI GraphGraph
Looks like an API to reflect on the schema from an instance in the database. Is there some overlap with the AAI Schema Services? Is this leveraging AAI Schema Services e.g. as client? proxy? facade? implementation detail? Is it an alternative to AAI Schema Services? Update: Keong Lim I do see an overlap with the schema service mainly for the retrieval of the data, but GraphgGraph will offer more on top of that(UI, NLP), during the POC code for GraphGaph there was no schema service. Implementation details can be discussed on our call for how exactly to leverage the schema service, the poc code, and where the core logic resides for this functionality. 15th Nov: Reviewed A&AI GraphGraph and agreed that there is now overlap with AAI Schema Service that should be addressed in updating the POC version. | |
29 | AAI HAProxy and 2-way-TLS | IN PROGRESS | 29th Nov | Technical solution to either decommission the proxy or make design changes to AAF to enable client side certificates. After VF2F we will know if this is a requirement in Dublin. We discuss after this date. question raised: MSB - would client authentication be supported? | |
30 | ESR | DONE | 6th Dec 2018 | ESR in Dublin will be bug fix and non-functional requirements. Should ESR use AAF for RBAC in Dublin? Clients of ESR are MultiVIM and VF-C External system passwords are not encrypted in AAI During registration of cloud-region, ESR calls multi cloud, but when user is deleting data from the port ESR does not call multi cloud but should (feature request?) James Forsyth will establish a weekly AAI/ESR call - 9 PM EST 10 AM China time for several weeks to get the project on track for Dublin. James Forsyth will create a session next Thursday 19.12. 10 AM China time for Bo Lv | |
31 | AAF Resiliency | IN PROGRESS | 6th Dec 2018 | Yang Xu wrote in AAF Component:
This looks like some problem for resiliency and healthchecks. Is there a JIRA case for this already? An earlier case mentioned the "fix" in passing - AAI-1762Getting issue details... STATUS . Also in - AAI-1772Getting issue details... STATUS Helen Chen, is there a scenario for this in Casablanca Release Resiliency Testing Status? Jonathan Gathman, Mike Elliott, how can this remedy be automated? See also - AAF-599Getting issue details... STATUS
| |
32 | named-query replacements | James Forsyth | IN PROGRESS | 6th Dec 2018 | - AAI-1989Getting issue details... STATUS - APPC-1245Getting issue details... STATUS - POLICY-1278Getting issue details... STATUS - VID-355Getting issue details... STATUS Need to replace custom queries currently in use by these systems (and others?) in Dublin toward the retirement of the named-query API in Dublin |
33 | Helm chart 'requested' values for Small + Large AAI deployments | James Forsyth | IN PROGRESS | 6th Dec 2018 | - AAI-1940Getting issue details... STATUS
|
34 | Answering wiki questions | Team | IN PROGRESS | 13th Dec 2018 | Recently saw a few unanswered AAI questions in https://lf-onap.atlassian.net/wiki/display/DW/questions/all?&filter=recent Probably need a regular reminder to monitor this corner of the wiki to be responsive to users. Pavel: Or maybe something like an A&AI mailing list? (just brainstorming here) Pavel: RESOLUTION MISSING! |
35 | Inconsistency in Janus Graph | IN PROGRESS | 13th Dec 2018 | Invite Andrew Muller CT Paterson or other Amdocs people to talk about A&AI consistency (https://docs.janusgraph.org/latest/eventual-consistency.html) and how to resolve the issue (for example 48529533). Ideally on 20th Dec 2018 on the dev meeting. The idea in Beijing was that since we are going to use Champ, we would get rid of inconsistency problems we have currently in Janus - clarify it this is true, if not figure out an approach how to deal with this issue. Update 13th Dec : Agenda will probably be discussed on the AAI weekly on Wednesday | |
36 | Janus Graph 0.2.2 or 0.3.1 | James Forsyth | IN PROGRESS | 13th Dec 2018 | Discuss a move to janusgraph 0.2.2 or 0.3.1 in Dublin. Benefits ? Breaking change + migration needed ? Analysis needed on benefits of 0.3.1 until 10th Jan 2019 (decision should be made on the meeting on 10th). If no significant benefit in 0.3.X present and migration would be needed then upgrade only within 0.2.X version (assuming upgrading within 0.2.X is not a breaking change) |
37 | Spring Boot Upgrade | IN PROGRESS | 13th Dec 2018 | https://spring.io/blog/2018/11/29/spring-boot-1-5-18-available-now - support for 1.5.x spring boot will end in Aug 2019 so we should consider upgrading. We should update to spring 2.X.X. On the meeting on the 10th Jan discuss if anyone has anything against it, if not then James Forsyth creates a JIRA item for Dublin for upgrading the spring in all A&AI repositories. | |
38 |