Meeting Dates
...
07/12/2017 - DMaaP Project Kick-Off
Attendees
Habib Madani
- Dominic Lunanuova
Agenda
- Introduction
DMaaP Release Planning Review
- Define what role each member will play ( Contributor, Committer )
Action items
- Send Doodle invite for Weekly invite
Owner: Varun
Status: Done - Schedule Weekly Meeting
Owner: Varun
Status: Done
...
07/21/2017 - DMaaP Weekly Meeting
Agenda
- Review Release Planning
- Habib's proposal – Do we have time to evaluate Habib's proposal and make architectural changes and deliver by R1
- Collaborate with DCAE team
08/23/2017
Attendees
...
...
Habib Madani
...
08/30/2017
Agenda
- Review DMaaP walk through slide
- Review Action Items
- Review M4 Code Freeze Checklist
Action Items
- Deploy DMaaP using OOM ( Kubernetes)
Owner: Raghava
Status: In Progress. Verify Scripts
Due Date: Sep 14th - Push Message Router code to new DMaaP repos
Owner: Varun
Status: In Progress
Due Date : ASAP - Push Bus Controller and dcapi code to new DMaaP repos
Owner: Dominic
Status: In Progress
Dute Date : ASAP - Set up Dev to Dev integration Environment ( Helen Chen is the contact from integration team )
Owner: Varun
Status: In Progress
Due Date: Sep 14th - Set up Jenkins Job, Release artifacts to Nexus and set up Sonar for Message Router
Owner: Varun
Status: In progess
Due Date : ASAP - Set up Jenkins Job, Release artifacts to Nexus and set up Sonar for Bus Controller and dcapi
Owner: Dominic
Status: In progess
Due Date : ASAP - Set up Continuous System and Integration Testing using Robot Framework
Owner : Sunil
Status:
Due Date: Sep 14th
- Deploy DMaaP using OOM ( Kubernetes)
...
Starting 7/23/2018.. until 8/10, meet daily to discuss Datarouter tasks for Casablanca.
8/13 till ? - meet on mon & wed (wed also overlaps with weekly DMaaP meeting open to ONAP community.)
Schedule for Sept: keep mon & wed regular schedule (fridays as needed)
Schedule for Oct: keep mon & wed (expect observer participation from AT&T DR Dev and ECOMP teams)
Participants:
Ram Koya, AT&T – DmaaP PTL. Very familiar with ONAP procedures.
Anne Kopp, AT&T – DmaaP Scrum Master. Expert in Jira.
Sunil Unnava, AT&T – supporting DMaaP MR. Expert in Kubernetes. Very familiar with ONAP delivery methodology.
Dominic Lunanuova, AT&T – supporting DMaaP Buscontroller. Member of original AT&T DR dev team. Very familiar with all DMaaP components, and ONAP delivery methodology.
Michela Bevilacqua, Ericsson – ONAP technical coord and ONAP delegate
Oskar Malm, Ericsson – Radio ONAP Principal Arch and delegate
Leah Cullen, Ericsson – Program Mgr
Rebecca Lantz, Ericsson – ONAP delegate focus on PNF integration
Mark Scott, Ericsson – performance mgt and modeling, and ONAP delegate
Emmet Cox, Ericsson – representative for dev team, “Team Fortress”. (I have to confess I didn’t know this was already a thing: https://wiki.teamfortress.com/wiki/Main_Page)
Tom O’Neil – scrum master
Brian Sweeney, Ericsson – program support
Team Fortress emails: Conor Ward <conor.ward@ericsson.com>; Emmett Cox <emmett.cox@ericsson.com>; Fiachra Corcoran <fiachra.corcoran@ericsson.com>; Helena Lydon <helena.lydon@ericsson.com>; Kyle Stewart <kyle.stewart@ericsson.com>; Mariusz Sobucki <mariusz.sobucki@ericsson.com>; Paul Dennehy P <paul.p.dennehy@ericsson.com>; Ronan Keogh <ronan.keogh@ericsson.com>; Tom O'Neill <tom.o.neill@ericsson.com>
Standing agenda:
- Blocking issues
...
Ref
...
Status
...
Description/Notes
...
1
...
Closed
...
Need Jira and Gerrit ids for Team Fortress. Some members of Fortress have LF ids. No problems yet.
...
2
...
Closed
...
Need SSL certs from AAF team. How to initiate the process for acquisition? Waiting for instructions from AAF (common to all teams).
- Demo planned for 7/24. Meeting info distributed to our team.
- 7/24: New documentation/video link: http://onap.readthedocs.io/en/latest/submodules/aaf/authz.git/docs/sections/development/
- need to initiate the cert request process.
8/2: Work ongoing, mail sent to to request ONAP certificate - 8/8: Dom to investigate generation of cert for DR until Jonathan returns. That failed due to lack of perms.
- 8/8: Ram requested assistance from Sai for Ronan's certificate request
- 8/15: still no response from Jonathan
- 8/17: AAF provisioning completed by Jonathan. Need to download certs now. Dom will download the certs until Fortress has access to Windriver.
- 8/22: Dom provided certs, but Fortress needs some overview.
- 8/24: replaced self-signed certs and are into testing
...
Can Ram request Committer privilege for a designated Team Fortress member?
7/25: Ram says first start Contributing, and then send requesting email to all existing DMaaP Committers. Process at: Committer Promotion Request
7/31: Ok with status quo even if Fortress contributors are not approved.
...
Can't see Sonar results until project CLM jobs are working.
8/2: nexus-iq links fixed, awaiting docker image push before we run CLM job
8/7: java release job still failing so fixes aren't visible yet
8/8: New Sonar report generated
...
Open Questions
...
Ref
...
Status
...
Description/Notes
...
1
...
Closed
...
Any additional documentation that can help?
Dom has identified some internal AT&T deployment docs but needs to scrub them before releasing.
Dom distributed 2 docs on installation and configuration
...
2
...
Closed
...
Identification of committers for relevant ONAP projects. Need to identify committer per gerrit repo:
dmaap/datarouter: Ram Koya
integration: Gary Wu
ci-management: Jessica Wagontall
oom: Borislav Glozman, Michael O'Brien, Mike Elliot
...
Closed
...
Need to review ONAP milestones – highlight high risk items.
7/25: might need a level of discretion / exemption required for what we submit on next milestone this week(M3). Ram: Is there a way to limit the functional commitment of DR to just what is needed for RAN Use Cases? e.g. existing code base, single Node, existing pub/sub API, simplest pub and sub routing path, no heat template for Itegration-Stable
Review RAN Use Case and identify DR features which will be exercised, then focus all efforts on this. Leah will follow up with RAN team to identify Use Cases.
7/26: reviewed RAN Use Case and confirmed DR functionality is adequate. And may have clarified even more function to exclude from code coverage.
...
4
...
Closed
...
Tie into CI process. JJB templates. What is repo? What are good examples?
7/24: Sunil shared existing JJB templates
7/25: Dom provided overview of CSIT process
...
5
...
Closed
...
Automated testing framework. Show some examples/repos.
- Integration test team using ONAP test framework (aka CSIT). See item 4. Closed.
- Unit test (aka Code Coverage). See item 12 details
- OOM healthcheck. ...leave till OOM work. Part of DMAAP-107. healthcheck is NOT in the Helm chart - Sunil to send link with info
8/10: Sunil shared email from Roger Maitland (Subject: DMaaP Message Router Health Check). Very good explanation that should be captured on the wiki someplace.
8/17: understand what is needed. need to get containers up first.
8/22: containers working. focus on sub testing first
8/29: target healthcheck for next week. initial oom delivery is waiting OOM team review and merge.
9/5: healthcheck included in oom delivery - waiting to be merged
9/10: target for merge is today
...
6
...
Closed
...
Different from AAI DR?
Yes, aai/datarouter seems to be a service that listens to some Message Router topic for certain events for the purpose of discovery.
...
7
...
Closed
...
Introduce ONAP required documentation. Architecture, APIs, etc.
7/25: need to assign JIRA tickets
7/26: Brian needs to enumerate what documentation is really needed
7/27: Conner provided a list of documentation requirements which will be added to Jira
...
7/24: Ronan asks:
We have started looking into Jenkins Job Builder.
- We have started with the Using Standard Jenkins Job (JJB) Templates page – is this the best place to start? A: Yes, but also look to existing project templates as working examples.
- Do you have any sample template files that we could start with? A: Yes, Sunil to share. Sent.
- Will we need access to run jobs in https://jenkins.onap.org/ ? A: No. The commit of the template into the ci-management/jjb repo is sufficient to run the job.
...
Be sure not to introduce any OpenSource security vulnerabilities. DR code has not been scanned yet. Expectation is that mySQL driver has some known vulnerability. Other projects have switched to PostgreSQL, or may need to claim an exception. Strategy: get a version committed ASAP so scan gets done.
7/26: get a container up and running and committed so we get a scan. Using 3/23 scan results, found few violations - Dom to send that report. Also mysql issues were NOT severe (red).
7/27: known vulnerability: new version of com.thoughtworks.xstream. Also, mySQL has a license violation, so may need to use mariadb
8/2: See Section 4, in blocking issues
8/7: code changes in progress, but still can't see because of blocking issue #4
8/8: DMAAP-557 has fixes
8/9: complicated by Jetty 9 API changes. But mySQL and xstream changes are finished.
8/17: improved results but remaining changes: xstream 1.4.10,, jetty 9.4.12.RC2 (Sunil used 9.3.8)
8/22: 1 violation left: License not approved: CDDL-1.1 or GPL-2.0, CDDL-1.1 or GPL-2.0-CPE. In javax.websocket : javax.websocket-client-api : 1.0. See onap-discuss email thread.
8/27: Warning from jetty logs stating that the RC2 version is unstable and shouldn't be used in production. Possibly need to downgrade??
8/27: Agreed that all dmaap components should stay in sync so use 9.3.8RC0
8:28: Initiated run-clm on last merge (which uses 9.3.8.RC0). Results: https://jenkins.onap.org/job/dmaap-datarouter-maven-clm-master/2/ Summary: 0 Critical, 24 Severe. Many of the Severe license issues are due to missing license declarations.
8/28: Dom: Tried 9.3.8.RC0 for buscontroller. It did clear the old violations, but introduced 2 new Severe license issues. So, I switched back to 9.4.12RC0 which only has 1 Critical security violation for a configuration option that doesn't apply to us. I requested a "not applicable" designation from help@onap.
8/30: Dom escalated to Steve Winslow for guidance on procedure. NOTE: related to point 30
8/31: Winslow cleared 9.3.8.RC0 license exceptions.
9/5: team will work on remaining violations
9/14: down to 8 severe. will pursue with Steve Winslow
9/19: add dom4j explanation (restricted to unit test) on vulnerability page
9/24: no critical. 2 severe security only used for unit test. 5 severe license waiver requested to Steve Winslow.
10/8 Check M4 checklist before closing this item.
10/10: email from Pawe? Pawlak: consider dom4j 2.1.1
10/15: dom4j fixed! and documented on onap-seccom dist list
...
Do we need to amend the M1 checklist to include DR? Or do we want to get a bit more confidence?
Is there risk for NOT amending the checklist? Anne notes that Functionality Freeze is 7/26 anyway, so maybe that is the place to re-introduce DR.
Anne to review with Ram.
7/30: Moot point now!
...
Fiachra requests help with code testing error resolution. Dom to meet to review who to contact next.
7/26: Fiachra demonstrated unit testing progress against DR in docker containers. Many tests (of provisioning API) are failing with 403. Attempts to manually configure for proper authorization were not successful. Also, test code seems to be missing a critical tests.properties file. Fiachra took a good stab at creating this, but uncertain if we are missing any assumptions. Dom to investigate further, and request help from prior DR contributors.
7/27: Progress! only 11 of 96 tests failing. Still concerned that code coverage metric seems not to reflect properly after these tests are run (i.e. under 5%). Will continue on current path but may need to adjust methodology.
7/30: existing tests are functional (tests the API entry points), and assume server is running in a sep container, so doesn't match the code coverage methodology. Will be a major work item. Sunil shared MR pom settings:
<jacoco.version>0.7.7.201606060606</jacoco.version> <sonar-jacoco-listeners.version>3.2</sonar-jacoco-listeners.version> <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin> <!-- Default Sonar configuration --> <sonar.jacoco.reportPath>target/code-coverage/jacoco-ut.exec</sonar.jacoco.reportPath> <sonar.jacoco.itReportPath>target/code-coverage/jacoco-it.exec</sonar.jacoco.itReportPath> <!-- Note: This list should match jacoco-maven-plugin's exclusion list below --> <sonar.exclusions>**/gen/**,**/generated-sources/**,**/yang-gen**,**/pax/**</sonar.exclusions> <sitePath>/content/sites/site/org/onap/dmaap/messagerouter/msgrtr/${project.artifactId}/${project.version}</sitePath>
7/31: will need to mock api for unit test coverage
8/2 : No change in status, mocking API the process to increase code coverage
8/7: getting tips from test team, expects 50% to be achievable.
8/8: Target is 8/15
8/15: Sonar is not picking up code coverage (missing pom entry). Prov at 8% (Feed Servlet 95%. Subscribe Servlet 70%.) Node not started.
8/17: Prov 12%. Sunil: Be sure to add Sonar properties and surefire plugin to pom.xml
8/21: Prov 19% Sonar is still not picking up coverage - concentrating on increasing code coverage instead of resolving this
8/22: Overall > 30%. Still messing around with pom.
8/27: Results showing up in Sonar! Still just over 30% coverage. Continue covering under DMAAP-101 (Kyle)
...
Desire to sync with Anne about JIRA management.
7/30: need to wait for assignment of new PM
08/01: I created the next two sprints (Leah/Ericsson) so we should be able to populate DMAAP 2018-11 now, also contacted RAM about getting admin access to update the quick filters.
08/02: New sprint created - need to ensure that we also handle Casablanca Release Platform Maturity requirements
...
Keep this line in Copyright section of code?
ECOMP is a trademark and service mark of AT&T Intellectual Property.
A: Yes, please.
...
Problem with jenkins job creating docker container.
7/31: successfully building images locally. turn on boolean for push during jenkins build
8/2 : troubleshooting ongoing - expect resolution shortly
8/7: image on nexus3! seeing both SNAPSHOT and finished image, but Sunil says that is correct.
...
OOM updates. App team makes delivery to oom project. No special approval, but nice to let them know.
NOTE: port reservation coordination recorded on wiki page....a good item to discuss with oom team
8/2: 2 ports reserved, a good starting place, may need a third?
...
Acquire original DR Prov DDL scripts.
8/3: provided by Dom
...
Backlog grooming. Fortress to review list and bring up anything needing discussion.
8/9: cover in JIRA discussion
...
Ronan:
- Q on DMAAP-51 re docker tagging. necessary? A: Sunil: not used yet. But suggestion for creating separate jjb file for docker. see email in onap-discuss from a few weeks ago
- See released version in Nexus3? A: Sunil advises to see if they are released by 8/9. in POM use version-SNAPSHOT. When satisfied, enter LF Ticket to release docker image. If not, then enter LF ticket.
8/9: Sunil: jar in staging until the end, then released. To release, send email to LF w/ CC to Ram for approval. Be sure image works with integration tests because tests will be repeated every day.
8/31: images on staging nexus. so lets complete some CSIT before updating manifest.
8/31: tested successfully with Buscontroller CSIT
...
CSIT testing: publish feed request gets redirected and auth header gets stripped.
Workarounds: publish directly to Node. Sleep before publish to allow for provisioning propagation delay.
...
Fortress demo of DR to RAN collector possible for this week.
8/9: probably next week. NOTE: Dom unavail on 8/20 and 8/21.
813: no reply from client group....so close till they are ready
...
Nobody on Dmaap team has JIRA Admin access. Gildas is on vac. Mail sent to Ram for privileges.
8/13: will provide details for filters. Ram will get that implemented in Jira. Sunil now has admin access
8/15: Rachel has admin access too
...
CSIT tests failing.
/setup.sh: line 16: /usr/local/bin/docker-compose: Permission deniedSunil suggests Conor send request to helpdesk@onap.org for inquiry.
8/13: Jessica will investigate, but it cleared up in the meantime.
...
M3 Readiness includes API review with Arch team. Sunil opinion: Arch will request review if needed. But DR is not really new so no email necessary.
8/15: Ram: "I don’t think we need to review as its an existing component."
...
If DR is not supporting heat deployment, do we need a waiver of some sort? Sunil: bring up in next TSC review for official decision. Look for advice for Ram on wed.
8/15: Ram: "Should we check with deployment team if they still insist us to create heat-style deployment?" Sunil: Project is Gildas, Integration is ???
Target next thurs TSC mtg where they review M3 checklist (last thursday of release cycle)
8/17: Heat deployments contact is Marco Platania (ooo till 9/3).
8/22: Ram to inquire at Arch sub-committee mtg today
8/23: Ram has sent email to Gildas and Brian Freeman.
Answer: We are good with OOM deployment only. If time permits, we’ll create HEAT.
...
https://wiki.onap.org/display/DW/CII+Badging+Program 98% for Beijing
8/17: Ram requests a User Story for all 3 projects to get Silver badge.
JIRA created.
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
...
DR API Authorization model doesn't work well with Kubernetes. i.e. client POD names aren't known in advance, and may change, so they aren't good for DR authorized lists. For Casablanca, Fortress will introduce a config param to disable the host authorization checks. Also, will create a backlog item to re-think the API authorization model as it relates to K8S.
8/17: disabling auth checks by IP. Added story: DMAAP-597
...
Reviewed feed provisioning and publishing sequence with Team Frontrunner (RAN File Collector). Initially, will assume static pre-provisioned single feed that Frontrunner can assume will exist in ONAP. Later phase would support dynamic provisioning of feed via Buscontroller, but we need all the pieces (AAF, certs, DR) in place to demonstrate how to do this, so we don't want it to block Frontrunner.
Henrik to provide Name, Description, publisher credentials for the static provisioning.
8/27: Fiachre gave Henrik the info on provisioning.
...
Emmett asks “Release Planning Platform Maturity Table” due date?
Release Planning Template was due 6/28/2018 but that seems to be a different definition.
There are Platform Maturity Requirements (S3P) with Casablanca specifics but no date.
8/22: see
https://wiki.onap.org/display/DW/DMaaP+R3+-+M1+Release+Planning#DMaaPR3-M1ReleasePlanning-PlatformMaturity update by RC0 (10/10)
8/31: discussed questions from Emmett's email. In general, we establish metrics and goals, and self-report back. There may be some details which could be problematic if taken literally. e.g. "single logging system" or "80% code coverage"
...
Sunil warns to keep an eye on "High" Security Violations for M4. May effect all projects.
8/23: In TSC call, Gildas confirmed "Severe" and "Critical" will need to be addressed.
8/24: Fortress needs access to reports so there is no delay, esp in addressing the orange violations. Need to request from Ram.
If can't fix a license issue, need to document why in project page. (Other projects have similar concerns) questions to Steve Winslow. (
Steve will do manual scans using Fossology.
9/5: Ram's advice: focus on Critical and then we'll deal with Severe
9/19: Steve will review waivers for license issues
9/27: Refer to 10
...
Requirement to include port number in "links" attribute of json payload/response?
eg:
"links": {
"feed": "https://dmaap-dr-prov/feed/1",
"log": "https://dmaap-dr-prov/sublog/1",
"self": "https://dmaap-dr-prov/subs/1"
}
Curl will fail without the required port so maybe we should include it?
8/27: keep an eye on this when we integrate with Buscontroller and end client pubs
8/29: another consideration, OOM deployments are short on node ports
9/10: For Casablanca, keep 8443 in place. Revisit in Dublin
...
Is there an ONAP container repo on public docker hub? Decide whether ONAP delivery or public delivery is best for DR client code samples.
8/29: will build docker container as part of project and deliver to ONAP Nexus
...
For Ram....are there logging consolidation tools requirements in Casablanca (e.g. filebeat and kabana )? as it relates to Platform Maturity.
9/5: Ram: probably kabana, but will confirm.
10/8 Dom sent email to Ram. Still waiting on reply.
10/10: commitment was for Level 1 which requires log4j or logback.xml or EELF. Commitment is satisfied.
...
Outside developers are making contributions to code base to address Sonar issues. Ram is merging. e.g.
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
9/5: be careful on merges. Ask Ameresh Kumar to add developer for review, and ask how he is testing.
9/14: new commits from Ameresh are getting reviewed by Fortress and merged by Ram
...
What is the sequence for releasing a docker container? Sunil described it. Tom 
was going to reply with notes so we all know the steps. see DMAAP-764 for versioning of docker containers.
Independent Versioning and Release Process.
9/24: finally its understood! details coming...
9/26: Release Version of Component
needs step 5 for oom manifest...
...
CII Badging. Questions about TLS certificates.
- Certificate verification by TLS?
- What version of TLS?
How do we answer these?
9/19: Tony Hansen is working this same issue for DCAE team and will share ONAP general answers.
9/26: Conor is working this and making progress
10/1: up to 60%
10/10: up to 78%. Remaining: medium and critical vulnerabilities (see item 40)
10/17: any update to new target for Casablanca? (dom)
...
Dom to review code to remove IP addresses in source
review completed.
...
DR not coming up in OOM deployments. Under investigation.
Rebuilt environment, and deployment succeeded.
...
Critical Sonar bug: src/.../datarouter/provisioning/utils/DB.java ("Use try-with-resources or close this "Connection" in a "finally" clause"). Not as easy as it seems. Requires major refactoring late in the release. Q for Ram: What is process for getting an exception/waiver?
10/8 No exception for sonar bug. Track it for dublin release ?? Ask Ram is this ok.
10/17: Are Sonar vulnerabilities required for CII passing?
...
Sonar issues:
9/26: down to 32 sonar vulnerabilities, more coming...
10/1: 0 sonar vulnerabilities.
...
No issue with Data Router in External Labs Heat jenkins jobs.
According to this page - https://jenkins.onap.org/view/External%20Labs/ The last job run for a Heat deployment was on Sept 5th and we were removed from Heat healthcheck on Sept 19th
...
Dublin (R4) planning with Michela. Suggested friday 10/12. Dom to send invite.
10/12: meetings started.
...
current problem in oom deployment for dmaap (possibly due to change to common postgresql chart https://gerrit.onap.org/r/#/c/67941/)
10/11: Jasmine provided fix
...
healthchecks failed just before RC0, but are now resolved. was a jenkins environment issue.
10/17: happened to fail on 10/16, but passes today. Sunill: Jenkins suffering from temporary issues
- PM (Jira items, ONAP Release schedule)
Brian has a deck of high level tasks
- Other?
...
| Page Tree | ||
|---|---|---|
|