2021-09-28 DCAE Meeting Notes

S.NO

 Time (est)

 Topics

 Requester/Assignee

 Notes/Links

START RECORDING

PARTICIPANT LIST

Istanbul Release topics

@Vijay Kumar

• Release Planning: Istanbul

• R9 M1 - DCAEGEN2-2757 - DONE

• R9 M2 - DCAEGEN2-2783 - DONE

• R9 M3 - DCAEGEN2-2841 - 08/27  09/02 - DONE

• R9 M4 - DCAEGEN2-2896   09/16 - DONE*

• R9 RC - 10/14

• Istanbul Release Requirements

• Istanbul Impact View per Component

• 0: Integration Weather Board for Istanbul Release

• Architecture

  • https://lf-onap.atlassian.net/wiki/display/DW/ARC+DCAE+Component+Description+-+Istanbul-R9

  • https://lf-onap.atlassian.net/wiki/display/DW/DCAE+R9+M2+Architecture+Review

• Security updates - Istanbul DCAE  - DCAEGEN2-2768: SECCOM - DCAE Vulnerability updates for IstanbulClosed

• Istanbul Integration Blocking points

Review Outstanding gerrit submission and bootstrap updates

@Vijay Kumar

RC0 Tasks

• Pairwise testing  - DCAE Pair Wise Testing for Istanbul Release + include E2E slicing usecase test reference

• Documentation 

  • Component owner expected to add specific installation-helm.rst (similar to https://gerrit.onap.org/r/c/dcaegen2/+/124070) to explain the properties/override specifric to application

• Release branching 

  • @Vijay Kumar  will target branching for all repository by Oct 1, 2021 

    • CI change

    • Repo branching (except dcaegen2 (doc repo)  deferred to next week)

    • Update repo with nexus/raw reference to new path on master

Istanbul Container version - See below - 20210908DCAEMeetingNotes-IstanbulArtifactsTracker

Review Outstanding bugs 

Istanbul Integration Blocking points

Jakarata - Helm migration 

@Vijay Kumar 

Discuss DCAE transformation planned items for J release - DCAEGEN2-2773

Priorities

• Switch components to latest SDK version

• OOM charts updates to disable Cloudify/handlers (CBS may be exception depending on above item)

• Integration testsuite migration

• Helm flow support through MOD 

• DMaap Provisioning optimization  - @Ajay Deep Singh  

  • Pub/sub credential handling alignment - OKAY to proceed 

  • Topic/Feed cleanup on undeploy -

    • Topic cleanup to be split into separate Jira and assessed later ; Feed cleanup OKAY to proceed.

       Impacted by AAF removal, Strimzi Operator introduction

@Former user (Deleted)  will share example of envsubst issue for CBS SDK migration for DFC - deferred to next week

3gpp OpenAPI Rel17 schema issue (Istanbul) - 

DCAEGEN2-2893 DCAEGEN2-2894 DCAEGEN2-2895

@Vijay Kumar 

As the default SA91 openAPI schema files are having issues, the default deployment for VESOpenAPI manager for Istanbul will be switched to SA88-Rel16 schemas (matches to default VESCollector container loaded schemas). The instructions for both VESCollector and VESOpenAPI manager RTD will be updated to reflect the same and include steps to override to SA-91-Rel16 if required. This will refer to the steps documented under oom/external-schema-repo-generator repo.

1. Submit updates to OOM repository to revert configmap in dcaegen2-services to SA88 

2. Provide reference to oom/external-schema-repo-generator repo documentation for SA91 related configmap generation. - https://gerrit.onap.org/r/c/oom/utils/+/123794

3. Update DCAE VESCollector and VESOpenAPI manager document to reflect SA88 as default for Istanbul and include optional steps to load SA91 (based on #2) 

Other ONAP features/changes with DCAE impacts for Jakarta

@Vijay Kumar 

General discussion

• Servicemesh support - DCAE component impact on below 

1.  

   1. Allow http support (early J release) - DCAE components that rely only on HTTPS will be impacted

   2. Components dependent on AAF cert 

   3. Secure topic/feed dependency - DFC, PM-Mapper, PNF-Reg, SliceAnalysis, BBS-Ep

• Impact from DMAAP-MR → Strimzi/Kafka

  • Pub/sub API change - may need to integrate with new SDK provided by DMAAP team; all DCAE MS will have impact to align with new SDK.

    • SDK client lib need to have backward compatibility with MR (controlled via ENV/deployment property)

  • Mapping service - Alternate solution proposed by @Fiachra Corcoran /DMAAP team could be candidate solution; avoid technology dependency and can be handled generic across all MS without much impact

  • Provisioning API will change - DCAE helm template job MR provisioning template/scripts to be revisited under Strimzi/Operator

Topics/Demos' for future meetings

<Topics can be added below by presenters as they are ready>

@Vijay Kumar

NA

@Tony Hansen

NA

@Jack Lucas

NA

@Former user (Deleted)

NA

@Former user (Deleted)

NA

@Former user (Deleted)

NA

COMPONENTS (ISTANBUL RLS VERSION)

REVISED from H RLS

SNAPSHOT

Notes

onap/org.onap.ccsdk.dashboard.ccsdk-app-os:1.4.3

YES

1.4.3 (H maintenance) - released

05/04/2021 - 1.4.3 released
05/03/21  - 1.4.3 Dashboard user/pg insert error
03/27/2021 - 1.4.2 released
03/23/2021 - 1.4.2 revert spring/jackson version

onap/org.onap.dcaegen2.analytics.tca-gen2.dcae-analytics-tca-web:1.3.1

YES

1.3.1
1.2.2 (H maintenance)

09/07/21 - 1.3.1 (master) released (includes Update DCAE-SDK to load config and policies from mounted files + Vulnerabilities fixes + Add AaiSerializer to prevent the duplication entry + Fix AAI HTTPS connection issues)
08/06/21 - 1.2.2 released
08/02/21 - 1.2.2 (H Mainteancne) AAI connection error fix
07/13/21 - 1.3.1 - cbs client sdk 1.8.6
05/20/21 - 1.3.0 released (includes bug fixes + vul updates)
4/13/21 - 1.3.0 switch to integration-java11:8.0.0
2/18/21 - 1.2.2 - vulnerability changes (https://jira.onap.org/browse/DCAEGEN2-2654)

onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:1.6.1

YES

1.6.1 

09/09/21 - 1.6.1 released

08/24/21 - 1.6.1 SDK version bump from 1.5.0 to 1.8.7 in order to enable config file support
--7/30/21 - 1.6.1 Fix vulnerabilities (top up spring-boot version to 2.4.8) + Released
7/21/21 - 1.6.0 release + vulnerability updates

onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:1.9.1

YES

1.9.1

9/2/21 - 1.9.1 released
8/25/21 - 1.9.1 Update DCAE SDK version from 1.8.2 to 1.8.7
7/9/21 - 1.9.0 released
7/7/21 - 1.9.0 remove checker-framework dependency + sdk version 1.8.4 (config/policy source cbs)

onap/org.onap.dcaegen2.collectors.restconfcollector:1.2.5

YES

1.2.5

05/04/2021 - 1.2.5 released; blueprint update pending
03/22/2021 - 1.2.5 Disable access to external entities in XML parsing

onap/org.onap.dcaegen2.collectors.snmptrap:2.0.5

YES

2.0.5

07/22/2021 - 2.0.5 CBS client 2.2.1 upgrade
02/26/2021 - 2.0.4 release submitted
02/17/2021 - 2.0.4 py39 support + revise pysnmp version

onap/org.onap.dcaegen2.collectors.ves.vescollector:1.10.1

YES

1.10.1

9/2/21 - 1.10.1 - released
       Update Cbs-Client to version 1.8.7
      - read configuration from file by default
       Fix ambiguities in pom.xml for spring-boot-maven-plugin and docker base image
08/24/21 - 1.10.1 - update CBS-Client from 1.8.0 to 1.8.7 in order to enable config file support

onap/org.onap.dcaegen2.deployments.cm-container:4.6.1

YES

4.6.1

08/30/2021 - 4.6.1 released
08/25/2021 - 4.6.1 Upgrade clamp-plugin to 1.1.1 (Fix dcaepolicyplugin dependency)
06/10/2021 - 4.6.0 released
06/07/2021 - 4.6.0  dmaap-1.5.1 plugin inclusion

onap/org.onap.dcaegen2.deployments.consul-loader-container:1.1.1

YES

1.1.1

08/23/21 - 1.1.1 released
08/23/21 - introduce delete_key.sh to remove without error
02/8/21 - Support for yaml uplaod and consul key removal

onap/org.onap.dcaegen2.deployments.dcae-k8s-cleanup-container:1.0.0

NO

1.0.0

onap/org.onap.dcaegen2.deployments.healthcheck-container:2.2.0

YES

2.2.0

06/25 - 2.2.0 released
06/10 - Switch to node:14.17.0-alpine3.13 (remove python2)

onap/org.onap.dcaegen2.deployments.k8s-bootstrap-container:3.3.5

YES

3.3.5 

09/14/21 - 3.3.5 release (misc blueprint revision for istanbul)

09/08/21 - 3.3.4 released; OOM merge pending (new 3.3.5 to include misc blueprint updates)
08/23/2021 - 3.3.4 PM Mapper updated to 1.7.1 - Fix granularityPeriod issue
08/16/2021 : 3.3.3 released (includes PRH 1.7.0 -  AAI variable syntax in URL config + AAI v23 api) + SNMP 2.0.5
08/02/2021  3.3.2 released
08/02/2021 - 3.3.2 PRH 1.6.1 update
07/29/2021 - 3.3.1 PM-mapper blueprint revision 1.7.0 + released
07/27/2021 - 3.3.1 Heartbeat DCAE heartbeat container updated to 2.3.1 - Switched to CBS client lib 2.2.1, dockerfile update, OJSI SQL vulnerability fix
07/22/21 - 3.3.0 DFC 1.6.0 (integration base image + vulnerability removal)
06/10/21 - 3.2.0 released
6/8/21 - 3.2.0 - update dmapp plugin version (dfc/pm-mapper/kpi-ms)
06/07/21 - .3.2.0 - Remove DCAE MS deployment via bootstrap
May/21 - 3.1.0 (blueprint updates for PRH/PM-mapper/restconf/tcagen2/pmsh/heartbeat)

onap/org.onap.dcaegen2.deployments.multisite-init-container:1.0.0

NO

1.0.0

3/11 - Further change may be required; can be done post M4
2/20 - Possible impact on CBS nodeport removal for HTTP

onap/org.onap.dcaegen2.deployments.redis-cluster-container:1.0.0

NO

1.0.0

onap/org.onap.dcaegen2.deployments.tca-cdap-container:1.2.2

NO

1.2.2

2/11 - 1.2.2 updated into OOM/chart
1/13 - 1.2.2 introduced for consul dependency removal (DCAEGEN2-2021)

onap/org.onap.dcaegen2.deployments.tls-init-container:2.1.0

NO

2.1.0

2/11 - 2.1.0 (dynamic cert)
<2/11 - 1.0.3 (El-alato)

onap/org.onap.dcaegen2.platform.configbinding:2.5.4

YES

2.5.4

09/14/21 - OOM change submitted under https://gerrit.onap.org/r/c/oom/+/124119
5/4/21 - 2.5.4 released; OOM update pending
3/29/21 - 2.5.4 change merged
2/24/21 - 2.5.4 for base image change for integration 8.0.0 (py39)

onap/org.onap.dcaegen2.platform.deployment-handler:4.4.1

NO

4.4.1

2/24/21 - Consul removal in DH will be deferred
9/23 - 4.4.1 - default tenant_id fix
9/14 - 4.4.0 released
8/19 - DH change for deployment query (cloudify)/dashboard dependency
2/20 - Possibl impact on Policy changes

onap/org.onap.dcaegen2.platform.inventory-api:3.5.2

NO

3.5.2

3.5.2 - seccom vulnerability update
3.5.0 (Guilin) - exception 7/13 - 3.5.1 for classnotfound error
7/3 - 3.5.0 for java11 upgrade
2/19 - Rls submitted
2/18 - 3.4.1 pending rls; includes non-root + jks pwd sourcing change

onap/org.onap.dcaegen2.platform.policy-handler:5.1.2

NO

5.1.2

03/05/21 - 5.1.2 released; OOM update pending
03/02/21 - 5.1.2 (bugfix for DCAEGEN2-2652;startup error);requirements updated
02/26/21 - 5.1.1 released (https://jira.onap.org/browse/DCAEGEN2-2652)
01/24/21 - 5.1.1 python 3.9 update
8/26 - 5.1.1 py3.8 support
2/20 - New updates based on policy update flow support

onap/org.onap.dcaegen2.platform.servicechange-handler:1.4.0

NO

1.4.0 

7/17 - 1.4.0 for java11
8/6 - startup issue (DCAEGEN2-1718)
8/6- Version updated for re-release

onap/org.onap.dcaegen2.services.components.bbs-event-processor:2.1.1

NO

2.1.1

09/09/2021 - 2.1.1 released

04/12/2021 - 2.1.1 integration-java11:8.0.0 update
12/14/2020 - 2.1.0 - released

onap/org.onap.dcaegen2.services.heartbeat:2.3.1

YES

2.3.1

07/27/21 - 2.3.1 release 2.3.1 container
- Switch to CBS client lib 2.2.1
- Dockerfile update to remove volume mapping
- OJSI SQL security vulnerability fix
- Misc code cleanup
05/27/21 - 2.2.0 released
05/25/21 - Tested 2.2.0-SNAPSHOT-20210518T144249Z; ready for release
05/13/21 - 2.2.0 merged; pending test and release
04/13/21 - 2.2.0 - py39 support + integration-python:8.0.0
9/14 - 2.1.1 released
7/27 - 2.1.1 python 3.8 update
Test updates primarily (2.1.0 - ElAlto)

onap/org.onap.dcaegen2.services.mapper.vesadapter.universalvesadaptor:1.3.0

YES

1.3.0

08/26/21 - 1.3.0  switch to nexus3.onap.org:10001/onap/integration-java11:9.0.0
08/10/21 - 1.3.0 - CBS client sdk update to 1.8.7 (from 1.1.6) + docker non-root + onap/integration-java11:9.0.0
7/15/21 - 1.3.0 istanbul vul updates
2/12/21 - 1.2.0 released
2/5/21 - 1.2.0 - vulnerability fixes

onap/org.onap.dcaegen2.services.pm-mapper:1.7.2

YES

1.7.2 

09/09/21 - 1.7.2 released

08/26/21-  1.7.2 introduced for prev sonar change revision
08/25/21 - 1.7.1 (should be 1.7.2) sonar issues fix  (version bump missed); can be rolled with subsequent updated
08/19/21 - 1.7.1 released (add pm-mapper local dev tools  + granularityPeriod)
07/29/21 - 1.7.0 released
07/23/21 - 1.7.0 - vulnerability updates
04/16/21 - 1.6.0 released
04/15/21 - 1.6.0 integrated with sdk/dmaap client

onap/org.onap.dcaegen2.services.prh.prh-app-server:1.7.1

YES

1.7.1

08/31/21 - 1.7.1 released
08/24/21 - 1.7.1 Update DCAE SDK version from 1.6.0 to 1.8.7
08/11/21 - 1.7.0 released
08/10/21 - 1.7.0 - switch AAI variable syntax from config + v11 to v23 apis
--7/30/21 - 1.6.1 Fix vulnerabilities (top up spring-boot version to 2.4.8) + Released
--08/x - 1.6.1 released fixed vulnerabilities

onap/org.onap.dcaegen2.services.son-handler:2.1.4

YES

2.1.4

09/09/21 - 2.1.4 released

09/08/21 - 2.1.4 release pending

08/27/21 - 2.1.4 - CPS integration
08/24/21 - 2.1.4 - update FM spec to VES 7.2 support
7/21/21 - 2.1.4 - Istanbul vulnerability updates
06/21/2021 - 2.1.4 - CBS SDK update to 1.8.6

onap/org.onap.dcaegen2.platform.mod.onboardingapi:2.12.5

YES

2.12.5

02/15/2021 - component spec schema fix to include required parameters for config_map support
01/29/2021 - component-spec-schema updated to support: kafka stream subscribes and publishes, config maps as volumes

onap/org.onap.dcaegen2.platform.mod.distributorapi:1.1.0

NO

1.1.0

9/14 - 1.1.0 released
8/19 - 1.1.0 python 3.8 support
3/30 - 1.0.1 - nonroot change
1.0.0-SNAPSHOT-20200226T234626Z

onap/org.onap.dcaegen2.platform.mod.designtool-web:1.0.2

NO

1.0.2

3/30 - 1.0.2 - nonroot change
1.0.0-SNAPSHOT-20200217T165545Z

onap/org.onap.dcaegen2.platform.mod.genprocessor-http:1.0.2

NO

1.0.2

7/13 - 1.0.2 - Guilin sec vulnerability updates
3/30 - 1.0.1 - nonroot change
1.0.0-20200217T215940Z

onap/org.onap.dcaegen2.platform.mod.genprocessor-job:1.0.2

NO

1.0.2

7/8 - vulnerability update
3/30 - 1.0.1 - nonroot change
1.0.0-20200217T215940Z

onap/org.onap.dcaegen2.platform.mod.mod-registry:1.0.0

NO

1.0.0

1.0.0-SNAPSHOT-20200217T165545Z

onap/org.onap.dcaegen2.platform.mod.runtime-web:1.2.3

NO

1.2.3

2/26/21 - 1.2.3  waiting on merge in OOM
2/24/21 - 1.2.3 released
2/24/21 - 1.2.3 new bp-gen 1.7.3 version integration (default updates)

onap/org.onap.dcaegen2.services.datalakefeeder:1.1.1

YES

1.1.1

08/02/2021 -1.1.1 fixed elasticsearch entry + Released
05/11/2021 - 1.1.1 DL feeder released (base image + non-root)
04/19/2021 - 1.1.1 - DL feeder  integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)

onap/org.onap.dcaegen2.services.datalakeadminui:1.1.1

YES

1.1.1 

09/03/21 - 1.1.1 released
04/12/21 - 1.1.1 - DES integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)

onap/org.onap.dcaegen2.services.datalake.dbinit:1.0.0

NO

No image pushed

2/12 - 1.0.0 introduced (snapshot)

onap/org.onap.dcaegen2.services.pmsh:1.3.2

YES

1.3.2

09/10/21 - 1.3.2 merged

08/25/21 - 1.3.2 Subscription API DB changes with IPV4, IPV6 updates
06/21/21 - 1.4.0 - Update filter - User can edit/update the nfFilter to include/exclude selected NFs - REVERTED
05/04/21 - 1.3.1 released
04/20/21 - 1.3.1 update config-key for PMSH monitoring policy
04/12/21 - 1.3.0 integration-python:8.0.0 updates

onap/org.onap.dcaegen2.platform.adapter.acumos:1.0.6

YES

1.0.6 

09/13/2021 - 1.0.6 release file submitted; verified in Highstreet lab
6/04/2021 - [1.0.6] - Adapt DCAE adapter to work with Acumos Demeter's metadata structure change
5/5/21 - 1.0.5 support behind proxy

onap/org.onap.dcaegen2.services.components.slice-analysis-ms:1.0.6

YES

1.0.6 

09/09/21 - 1.0.6 released

09/08/2021 - 1.0.6 container release pending

08/30/21 - 1.0.6 merged
08/27/21 - 1.0.6 CPS integration
6/30/21 - 1.0.6 - vulnerability updates
4/13/21 - 1.0.5 - Sonar coverage improvement

onap/org.onap.dcaegen2.services.datalake.exposure.service:1.1.1

NO

1.1.1 (Istanbul)

5/10/21 - 1.1.1 released (non root + base image change)
04/13/21 - 1.1.1 merged
04/12/21 - 1.1.1 - DES integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)
4/8/21 - 1.1.1 - non root db + swtich mariadb to pg

onap/org.onap.dcaegen2.platform.ves-openapi-manager:1.0.1

NA

1.0.1

2/24 - 1.0.1 released 
02/19 - Introduced for H release

onap/org.onap.dcaegen2.deployments.dcae-services-policy-sync:1.0.1

NA

1.0.1

04/23/2021 - 1.0.1 released
04/22/2021 - 1.0.1 - include onap pem certificate + file umask fix
02/25/2021 - 1.0.0 released

onap/org.onap.dcaegen2.services.components.kpi-ms:1.0.1

NA

1.0.1 

09/10/2021 - 1.0.1 released

09/08/2021 - 1.0.1 container release pending

08/27/2021 - 1.0.1 - move certain config to policy section
02/25/2021 - 1.0.0 released

onap/org.onap.dcaegen2.platform.mod.auth-service:1.0.0

NA

1.0.1 (Not released/POC)

04/30 - 1.0.1 : int base image 8.0.0

onap/org.onap.dcaegen2.platform.mod.catalog-service:1.0.0

NA

1.0.2 (Not released/POC)

04/30 - 1.0.1 : int base image 8.0.0

onap/org.onap.dcaegen2.platform.mod.ui:1.0.0

NA

1.0.0 (Not released/POC)