Attendees

Luke Parker , Prudence Au , Michael O'Brien , Sanjay, Lorraine A. Welch

Agenda

Meetings every 2 weeks now - next meeting the 19th
Need to accomidate Luke Parker - who is up at 3 am right now in Australia - will be 1 in the morning until April - then 2am
- ideas: Better to be up at 1-3 than 8am - keep at 11 for now
For NA - stays at 11 - but I will verify if we are on UTC
DST is coming this weekend - we should agree on the meeting time - Luke does not go DST for 3 more weeks
Michael: status no progress on logging related jiras involving dublin scope (code/tech) last week
Passed M2
Discussed in TSC and PTL meets - issue related to manifest vs k8s values.yaml overrides for casablanca and CMR releases -
- TSC-86 - Getting issue details... STATUS
- TSC 2019-02-28
- OOM-1560 - Getting issue details... STATUS
In architecture review for M3 - (Addressed comments) Logging Dublin M3 Architecture Review
Working more for OOM lately - Created RKE script to replace rancher - pending work is to test HA - https://gerrit.onap.org/r/#/c/79067/ for OOM-1670 - Getting issue details... STATUS
Assisting with ONS April demo prep - 2 or more ONAP installations working together - one as an edge cloud
Focused on onap devops - primarily for 2 customers deploying ONAP
5G training next week
April 3-5 ONS work
- k8s infrastructure and
- Logging will be a part of the demo - we will show the kibana logs during a vFW operation in an i-frame for the master and edge ONAP deploys
PTL position is open - let me know if anyone would like to take it.
Also quickly discussed idea of rolling LOG into OOM - should revisit
Pending committer for pomba - Committer Request for Trevor Tait and Pierre Rioux - Committer Request for [Logging-Analytics] - will send out reviews once stats are completed - TSC review will be needed
TODO I need to check MSB issues on onap-discuss -
Spondon - import the dashboard - fix the one on the wiki - Lorraine A. Welch export to json first ONAP Culprit Locator
Lorraine A. Welch question about changing the index for the db (Grafana for example) - @Timestamp timestamp for example - TODO - merge the 2 indexes -
Q) Sanjay: (opentracking related - picking up errors on stdout (err, out)
- how:
- (also review from 2 weeks ago - Lorraine A. Welch System.out standardout - review in terms of not using a hardcoded file appender - in terms of syslogs
- see https://12factor.net/logs)
- log retention on pod failure - those not mapped to PVs - including the current EmptyDir link between the sidecar - etherial PV - logs only end up in the elk stack
- Look into PV/PVC structure for log retention instead of the default emptydir
  - https://git.onap.org/logging-analytics/tree/reference/logging-kubernetes/logdemonode/charts/logdemonode/templates/deployment.yaml#n114
  - see PV/PVC example . ~~logs can go to the /dockerdata~~nfs)
  - https://git.onap.org/oom/tree/kubernetes/portal/charts/portal-mariadb/templates/deployment.yaml#n84
  - Topic: File retention
  - Q: rotation schedule, size of retention - what role does the PV on the ELK stack play compared to the individual source pod PVs
  - @Luke's example - discuss with Mike Elliott - review a pending upstream contribution
  - https://git.onap.org/logging-analytics/tree/reference/provider/helm/logback/chart/resources/logback.xml#n10
  - variables set in
  - https://git.onap.org/logging-analytics/tree/reference/provider/helm/logback/chart/values.yaml#n14'
  - Review any issues with multiple providers in SDNC and APPC
- Lorraine A. Welch check https://www.loomsystems.com/blog/single-post/2017/01/30/a-comparison-of-fluentd-vs-logstash-log-collector

Attendees

Items

PRI	Title	Responsible	Status OPEN DONE	In Dublin	Last Worked on	Start	Notes
	Security Vulnerability template	Michael O'Brien	Ongoing	IN		20190122
	M1 template	Michael O'Brien Prudence Au	DONE	IN	20190124	20190122
	ONS NA 2019 April Talk proposal	Michael O'Brien	DONE	IN		20190122	pending 20190208 decision LOG-947 - Getting issue details... STATUS
	Use manifest generation over raw oom values.yaml docker image tag names	Michael O'Brien	DONE	IN	20190124	20190117	pending documentation in RTD Team, In the TSC it was decided to treat the diff between oom and the manifest by always running the manifest generated yaml in your deployments – you will not need to do this for master work – just for Casablanca and RC0-2 work Working out the details in https://jira.onap.org/browse/LOG-929 https://onap.readthedocs.io/en/casablanca/submodules/integration.git/docs/index.html?highlight=manifest /michael
	S3P Logging compliance TSC/PTL	Michael O'Brien	IN PROGRESS	IN	20190115	20190114	PTL 2019-01-14 Log Streaming Compliance and API discussed at last PTL meet with djhunt
	El-Alto 1.4 logging spec change - plan only todo merge with Dave's below	Michael O'Brien Dave Williamson	IN PROGRESS	IN	20190122
	Dublin Scope Planning	Michael O'Brien	DONE	IN	20190124		LOG-707 - Getting issue details... STATUS Logging Dublin M1 Release Planning
	RTD documentation	Michael O'Brien	OPEN	IN	20180129		Attending Thu 1130-1230 meets
	restart log4j format and files example	Michael O'Brien	IN PROGRESS	IN	201901115	20190108	https://gerrit.onap.org/r/#/c/62405/ for LOG-630 - Getting issue details... STATUS and LOG-178 - Getting issue details... STATUS Log Streaming Compliance and API#DeploymentProfiles
	Work with portal/sdk library	Michael O'Brien	IN PROGRESS	IN	20190129	20190115	Update: 20190129 - Existing eclipse environ for the RI being retrofitted At the pom stage bringing in the jar via portal/sdk in use by aai, dmaap, sdk, vid (vid link into so maybe?) <groupId>org.onap.portal.sdk</groupId> epic LOG-600 - Getting issue details... STATUS Jira PORTAL-348 - Getting issue details... STATUS review investigation in Logging Developer Guide Log Streaming Compliance and API#ExistingLibraryResearch\ Luke Parker discussion need to use the portal library in an initiating project for tx processing working likely with the SO team - via the work we are doing for them in https://gerrit.onap.org/r/#/c/69947/ (check the original spec - ODL specific - check appc/sdnc use of ccsdk)
	New Committers		OPEN		20190115		We have room for 2-5 committers and will be reviewing the list Logging Enhancements Project Proposal#KeyProjectFacts add your details to Logging Committer Promotion Requests 20190129 status - waiting on contributor documentation from each contributor
	OPNFV/ONAP Paris		DONE		20190108		https://ddfplugfest19.sched.com/ Tue-Thu Clover Gambia on prior https://zoom.us/j/115579117 - 7 hours ahead https://ddfplugfest19.sched.com/event/K1Gy/opnfv-clover-utilizing-cloud-native-technologies-for-nfv
	Security badging	Michael O'Brien	IN PROGRESS	IN	20190129		Need to restart this
	Security Vulnerabilities	Michael O'Brien Prudence Au	IN PROGRESS	IN	20190129		lower - but for M4
	s3p Secure https endpoints LOG + POMBA	for djhunt	OPEN	IN			Discussion on whether we need to lock down the nodeport exposed ports Can key off POMBA work already done todo: get s3p page
	Format compliance - working with AAI team + perf	Michael O'Brien	IN PROGRESS	IN	20190115	20181101	20190115 - casablanca cherry pick in queue logstack 5 to 3 and 1 https://gerrit.onap.org/r/#/c/75702/ (+) 20190109 from aai team https://lf-onap.atlassian.net/wiki/display/DW/2019-01-17+AAI+Developers+Meeting+Open+Agenda "hector has discovered that the stress test jar (liveness probe?) in aai-cassandra is hammering the cpu/ram/hd on the vm that aai is on - this breaks the etcd cluster (not the latency/network issues we suspected that may cause pod rescheduling) " #6 on 2018-12-20 AAI Developers Meeting around LOG-376 - Getting issue details... STATUS Discussion with @Sanjay Agraharam and [~pau2882] on checking how cassandra is running on the vm and if debug levels are on should be verified use labelling to split aai-cs and ls - no DaemonSet Michael O'Brien to reduce core count for ls to 1 from 3 LOG-915 - Getting issue details... STATUS edited 2019-01-10 AAI Developers Meeting for the 10th
	AAI team - 2 types of logging AOP/non-AOP		OPEN	IN		20181101	#22 on 2018-12-20 AAI Developers Meeting
	Logging requests from Vendors		OPEN				LOG-877 - Getting issue details... STATUS LOG-876 - Getting issue details... STATUS #15,19 and 37 on SP priorities for Dublin
	LOG Streaming compliance		IN PROGRESS	IN			Log Streaming Compliance and API LOG-487 - Getting issue details... STATUS LOG-487 - Getting issue details... STATUS LOG-852 - Getting issue details... STATUS and PTL 2019-01-14
	opentracing via https://opentracing.io/			IN (planning/POC for sure)	20190123		@Sanjay discuss integration - out of band processing - LOG-104 - Getting issue details... STATUS see zipkin arch https://zipkin.io/pages/architecture.html possibly tie both as a client of es ? Tie in to ONS NA 2019 April demo booth for LF https://lists.onap.org/g/onap-discuss/message/15066?p=,,,20,0,0,0::Created,,opentracing,20,2,0,22460823
	discussion - remove				20190108		discuss tick/tock logging spec behaviour - cassablanca implemented in dublin, dublin implemented in elalto Capture any changes to the Casablanca spec to be implemented in Dublin Active Logging Specifications - in the Dublin spec to be implemented in E* ONAP Application Logging Specification - Post Dublin verified but take care to keep the baseline spec - keep separation for any changes for elalto
	Log Checker	Cheuk Yiu Horace Ip Lee Breslau		OUT	20190109		MIke to review with Horace
	Search Guard	Michael O'Brien	OPEN	Maybe	20180109		LOG-494 - Getting issue details... STATUS
	spec changes for Dublin	Dave Williamson		IN (planning)	20190109	20190109	Dublin spec changes for Elalto environment name release name check mail for reply Michael O'Brien Prudence Au proposal of renaming the log file name itself for the release ie: 3.0.0-ONAP - will discuss later for next week
	Cluster logging behaviour S3P	Dave Williamson Michael O'Brien Prudence Au		IN			server name in clustered environments - I will add the details and the Jira right after this meet
	LOG ELK stack indexing/dashboards with Prometheus below		OPEN	IN	20190123
	Casablanca 3.0.1 work until 10th Jan Including POMBA	Michael O'Brien	DONE		20190122	20190113	LOG-913 - Getting issue details... STATUS revert Jira for data-router off TSC-92 - pending merge of https://gerrit.onap.org/r/#/c/75999/
	LOG openlab tenant devops cluster creation/testing	Michael O'Brien	Done pending vFW	IN			We have 2 clusters a 1+4 and 1+13 used for testing deployments and running the vFW Logging DevOps Infrastructure
	Wiki edits, RTD review	Michael O'Brien	OPEN	IN			Requiring Updates, Merges or Marked Deprecated
	Metric Streaming and Prometheus	Michael O'Brien Vaibhav Chopra	IN PROGRESS	IN	20181207		LOG-911 - Getting issue details... STATUS - experimental chart on http://secure.solar:30000/graph LOG-773 - Getting issue details... STATUS LOG-861 - Getting issue details... STATUS work with Vaibhav Chopra OOM-1504 - Getting issue details... STATUS @Sanjay - note the prom chart assumes a k8s environment - what about bare metal
	Finish SO filebeat additions	Michael O'Brien	IN PROGRESS	IN	20181207		SO-1110 - Getting issue details... STATUS https://gerrit.onap.org/r/#/c/69947/
	Finish LOG common charts	Michael O'Brien	OPEN	OUT to El-alto	20190123	20181207	James MacNider - bring in Prianka's common eLK charts and use them in Clamp, LOG, SDC, POMBA https://gerrit.onap.org/r/#/c/64767/ OOM-1276 - Getting issue details... STATUS rever to El-alto under LOG-936 - Getting issue details... STATUS
	Team Members Thank you and review		OPEN	IN			Shane Daniel to be removed - moved on to AI work - thanks for everything done for the logging and oom projects Lee Breslau Luke Parker Michael O'Brien Dave Williamson @Sanjay Agraharam Lorraine A. Welch James MacNider Prudence Au Anthony Manuel Cheuk Yiu Horace Ip
del	Review last 4 weeks since LOG Meeting Minutes 2018-12-05		IN PROGRESS				TSC-79 - Getting issue details... STATUS https://lists.onap.org/g/onap-discuss/topic/onap_helpdesk_65866/28800220?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,28800220 https://lf-onap.atlassian.net/wiki/display/DW/Cloud+Native+Deployment#CloudNativeDeployment-NexusProxy
	TSC/PTL meet actions		IN PROGRESS				TSC 2018-12-20 TSC 2019-01-03 PTL 2019-01-07
	OOM transfer chart ownership to teams LOG is part of poc	Michael O'Brien Mike Elliott	IN PROGRESS	IN	20190107		Starting - will have a training session - will send out any meetings to onap-discuss We may have the same symlink repo folder like we do for doc Last discussed TSC 20180109
	OOM Deployment priority base platform includes LOG	Michael O'Brien Mike Elliott	IN PROGRESS	OUT todo review with Mike Elliott	20190123	20181207	Q2) priority of system level containers like the ELK stack - OOM has a common services JIRA - DMaaP, AAF - TODO get JIRA - make sure log is in this! There is a cd.sh retrofit that sequences the pods in order for deployment stability - this will be phased out when tiered deployment comes in LOG-898 - Getting issue details... STATUS LOG-326 - Getting issue details... STATUS https://gerrit.onap.org/r/#/c/75422/ via ONAP Development#WorkingwithJSONPath DCAEGEN2-1067 - Getting issue details... STATUS
	k8s manifest or oom values.yaml for docker tags - truth	Michael O'Brien	IN PROGRESS		20190123		TSC-86 - Getting issue details... STATUS TODO: paste TSC review - manifest is truth RTD doc link to run the script to get the yaml override
	Nexus3 routing slowdown	Michael O'Brien	DONE		20181222		TSC-79 - Getting issue details... STATUS 20181217-22 LOG-898 - Getting issue details... STATUS
	LOG compliance diagram/exercise	Michael O'Brien @Sanjay Agraharam	IN PROGRESS		20181205		Log Streaming Compliance and API part of prometheus work now Sanjay - diagram FB must be split between using AOP and AOP+spec compliant - only this one should be green

	Ongoing
	CI/CD pipeline TSC poc	Michael O'Brien	IN PROGRESS		20180107		TSC-25 - Getting issue details... STATUS LOG is part of this poc https://lists.onap.org/g/onap-discuss/topic/cd_task_force_tsc_25_meetings/29001640?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,20,29001640

Notes