2022-04-26 DCAE Meeting Notes

S.NO

 Time (est)

 Topics

 Requester/Assignee

 Notes/Links

Notes to moderator

• Start recording

• Participants List

Jakarata Release Updates

@Vijay Kumar 

Completed

• Release Planning: Jakarta

• R10 M1 -DCAEGEN2-2979 -  Rescheduled to 12/09/2021 

• ARC - ONAPARC-724 (Scheduled for 01/25/2022)

• R10 M2 - DCAEGEN2-3009  (01/27/2022) (02/03/2022)

• DCAE R10 Jakarta M2 Release Planning   

• Jakarta Impact View per Component

• R10 M3 - DCAEGEN2-3070  (03/03/2022 03/10/2022)

• R10 M4 - DCAEGEN2-3108  Feature Freeze (03/24/2022 → 03/31/2022)

• DCAE R10 Vulnerability - SECCOM Report (DCAEGEN2-3006)

Ongoing

• R10 RC - DCAEGEN2-3130 (04/28/2022)

  • Documentation/RTD updates

  • Closure on OOM patches (supporting gating/test issues)

  • Pairwise and integration test - DCAE Pair Wise Testing for Jakarta Release

  • Slicing Usecase - E2E Network Slicing Testing

  • IBN/CCVPN Test  - Integration Tests of IBN with CCVPN Use Case for Jakarta

• Only bug-fixes can be merged for Jakata release

• All other patch can be reviewed but NOT merged until J release branching work is completed

Open J issues

@Vijay Kumar 

Open

DCAEGEN2-3145 - Required for J release; fix in progress. New container to be verified/released and patched into OOM by 04/28 - @Niranjana Y 

DCAEGEN2-3141 - Pending OOM merge (https://gerrit.onap.org/r/c/oom/+/128043 )

Review Jakarta Stretch - goal EPIC/Stories

@Vijay Kumar 

Review commitments/stretch-goal items from DCAE R10 Jakarta M2 Release Planning   

1. ML MS (REQ-1074) -  @Niranjana Y

   1. Latest patch submission pending review - https://gerrit.onap.org/r/c/dcaegen2/services/+/127490; can be merged once all GR and BR codecomplaince are met.  Will need detailed doc for deployment & integration in slice usecase documented in onap wiki

Jakarata - Helm migration 

@Vijay Kumar 

Discuss DCAE transformation planned items for J release - DCAEGEN2-2773

All changes scoped under this EPIC are delivered 

DCAE J release versioning and branching

@Vijay Kumar 

Review Jakarta versions and open OOM updates

1. https://gerrit.onap.org/r/c/oom/+/128043  (ibn/ccpvn - slice) - WIP

   1. New version of sliceanlaysis containt will be amended into this patch including both bug fixes (DCAEGEN2-3142, DCAEGEN2-3141)

see "Jakarta Artifacts Tracker" below

Branching for J release - Will be done during week of Apr 25, 2022 

• All active repositories with exception of "dcaegen2" - doc repo has been branched and CI jobs inplace for J release

• "dcaegen2" will be branched by 04/27 

Any new patch for Jakarta MUST be cherrypicked to/from "master" branch to Jakarta branch

DCAEGEN2-2935 - Support configuration update through Configmap for DCAE service

(Kohn Release)

@Vijay Kumar @Jack Lucas 

Use CBS_CLIENT_CONFIG_PATH env to point to the file required to be sourced by SDK (e.g. /app-config-input/application_config.yaml)

Option#1 To update file mounted post secret replacement to be editable by application ID
Option#2 Put credential into separate file
Option#3 Expose Secret as ENV and let application handle substitution (removing envsubs from dcaegen2 common template)
Option#4 Global approach under OOM for all MS to handle secret which can supported via configmap (via operator)

Option#3 is preferred; dynamic DR provisioning solution may need to enhanced to create secrets (and let SDK do the substitutions)

Kohn Release assessment

1. EPIC DCAEGEN2-3119 created for DCAE Helm Transformation - Post migration and deferred activities

   1. DCAEGEN2-2774 - DCAE MS SVC template enhancement

   2. DCAEGEN2-3087 - helm generator should use "log.path" rather than "logDirectory"  (Kohn)

   3. DFC test case (related to config update) disabled currently; will be added back in Kohn release (INT-2074)

2. Add DCAE MS optional component under daily/weekly 

3. Verify/cleanup DL-Feeder charts on consul config

4. DCAEGEN2-3120 (IBN/CCVPN)- Enhance sliceanalysis MS to use DCAE SDK dmaap-client lib  -  @Former user (Deleted)

Other usecase/function impacts?

• DMAAP Strimzi migration (REQ-1207: DMaaP Enhancements for K releaseDone)

Topics/Demos' for future meetings

<Topics can be added below by presenters as they are ready>

TBD - OpenSSF Best Practice Goals for DCAE and Secure programming consideration - @Tony Hansen 

@Vijay Kumar

NA

@Tony Hansen

NA

@Jack Lucas

NA

@Former user (Deleted)

NA

@Former user (Deleted)

NA

@Niranjana Y 

NA

COMPONENTS (JAKARTA RLS VERSION)

REVISED from Istanbul RLS

SNAPSHOT

Notes/ChangeLog

onap/org.onap.ccsdk.dashboard.ccsdk-app-os:1.4.4

NO

1.4.4

07/21/2021 - 1.4.4 released (Support for special char in generated PG password
which caused issues on Daily/gating runs)
05/04/2021 - 1.4.3 released
05/03/21  - 1.4.3 Dashboard user/pg insert error

onap/org.onap.dcaegen2.analytics.tca-gen2.dcae-analytics-tca-web:1.3.2

YES

1.3.2

3/18/22 - 1.3.2 released
2/21/22 - 1.3.2 vulnerability updates
09/07/21 - 1.3.1 (master) released (includes Update DCAE-SDK to load config and policies from mounted files + Vulnerabilities fixes + Add AaiSerializer to prevent the duplication entry + Fix AAI HTTPS connection issues)

onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:1.7.1

YES

1.7.1

03/19/21 - 1.7.1 released to fix null pointer issues; merged into OOM

02/15/21 - 1.7.0 released
02/14/21 - 1.7.0 vulnerability updates
09/09/21 - 1.6.1 released
08/24/21 - 1.6.1 SDK version bump from 1.5.0 to 1.8.7 in order to enable config file support

onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:1.10.0

YES

1.10.0

2/24/21 - 1.10.0 released
2/23/21 - 1.10.0 vulnerability updates
2/21/21 - 1.10.0 (currently 1.9.2) scramsha-512 support added

onap/org.onap.dcaegen2.collectors.restconfcollector:1.3.2

YES

1.3.2 (jakarta)
1.2.7 (istanbul)

2/28/22 - 1.3.2 image release submission
## [1.3.2] - 2022/02/23
         - [DCAEGEN2-3043] - Vulnerability updates for J release (gson, logback-core)

## [1.3.1] - 2022/01/20
         - [DCAEGEN2-2857] - RESTConf - Switch CBS client library to 1.8.7 or higher
1/21/22 - 1.2.7. image released
1/18/22 - 1.3.0 base image update + 1.2.7 - log4j upgrade to 2.17.1

onap/org.onap.dcaegen2.collectors.snmptrap:2.0.6

YES

2.0.6

2/11/2022 - 2.0.6 released 02/09 (OOM update pending)
10/27/2021 - 2.0.6 - SNMP Trap collector - STDOUT complaince
07/22/2021 - 2.0.5 CBS client 2.2.1 upgrade
02/26/2021 - 2.0.4 release submitted
02/17/2021 - 2.0.4 py39 support + revise pysnmp version

onap/org.onap.dcaegen2.collectors.ves.vescollector:1.11.0

YES

1.11.0

2/14/22 - 1.11.0 vulnerability updates
1/28/22 - 1.11.0 switch VEScollector base image to integration onap image
1/20/22 - 1.10.3 released
1/19/22 - 1.10.3 log4j issue; upgrade to 2.17.1
12/18/2021 -1.10.2 released
12/14/2021 - 1.10.2 Remediation for Log4Shell vulnerability

onap/org.onap.dcaegen2.deployments.cm-container:4.6.1

NO

4.6.1

08/30/2021 - 4.6.1 released
08/25/2021 - 4.6.1 Upgrade clamp-plugin to 1.1.1 (Fix dcaepolicyplugin dependency)
06/10/2021 - 4.6.0 released
06/07/2021 - 4.6.0  dmaap-1.5.1 plugin inclusion

onap/org.onap.dcaegen2.deployments.consul-loader-container:1.1.1

NO

1.1.1

08/23/21 - 1.1.1 released
08/23/21 - introduce delete_key.sh to remove without error
02/8/21 - Support for yaml uplaod and consul key removal

onap/org.onap.dcaegen2.deployments.dcae-k8s-cleanup-container:1.0.0

NO

1.0.0

onap/org.onap.dcaegen2.deployments.healthcheck-container:2.4.0

YES

2.4.0

12/12/21 - 2.4.0 release (dynamic Add health checks for DCAE microservices deployed after
the initial installation of dcaegen2-services)
11/18/2021 - 2.3.0 released (enhanced logging + base image to latest node.js LTS release)

onap/org.onap.dcaegen2.deployments.k8s-bootstrap-container:3.3.6

YES

3.3.6

02/18/22 - 3.3.6 released
02/15/22 - 3.3.6 Fix for inventory upload issue
09/14/21 - 3.3.5 release (misc blueprint revision for istanbul)
09/08/21 - 3.3.4 released; OOM merge pending (new 3.3.5 to include misc blueprint updates)

onap/org.onap.dcaegen2.deployments.tls-init-container:2.1.0

NO

2.1.0

2/11 - 2.1.0 (dynamic cert)
<2/11 - 1.0.3 (El-alato)

onap/org.onap.dcaegen2.platform.configbinding:2.5.4

NO

2.5.4

09/14/21 - OOM change submitted under https://gerrit.onap.org/r/c/oom/+/124119
5/4/21 - 2.5.4 released; OOM update pending
3/29/21 - 2.5.4 change merged

onap/org.onap.dcaegen2.platform.deployment-handler:4.4.1

NO

4.4.1

2/24/21 - Consul removal in DH will be deferred
9/23 - 4.4.1 - default tenant_id fix

onap/org.onap.dcaegen2.platform.inventory-api:3.5.2

NO

3.5.2

3.5.2 - seccom vulnerability update
3.5.0 (Guilin) - exception 7/13 - 3.5.1 for classnotfound error

onap/org.onap.dcaegen2.platform.policy-handler:5.1.3

NO

5.1.3

0x/x//21 - 5.1.3 docker file issue with volumen mapping and required new container to be rebuild
03/05/21 - 5.1.2 released; OOM update pending

onap/org.onap.dcaegen2.platform.servicechange-handler:1.4.0

NO

1.4.0 

7/17 - 1.4.0 for java11
8/6 - startup issue (DCAEGEN2-1718)
8/6- Version updated for re-release

onap/org.onap.dcaegen2.services.components.bbs-event-processor:2.1.1

NO

2.1.1

09/09/2021 - 2.1.1 released
04/12/2021 - 2.1.1 integration-java11:8.0.0 update

onap/org.onap.dcaegen2.services.heartbeat:2.3.1

NO

2.3.1

07/27/21 - 2.3.1 release 2.3.1 container
- Switch to CBS client lib 2.2.1
- Dockerfile update to remove volume mapping
- OJSI SQL security vulnerability fix
- Misc code cleanup
05/27/21 - 2.2.0 released

onap/org.onap.dcaegen2.services.mapper.vesadapter.universalvesadaptor:1.4.0

YES

1.4.0

03/29/22 - 1.4.0 released
03/29/22 - 1.4.0 introduced to fix CBS/consul lookup

02/28/22 - 1.3.3 release submitted
## [1.3.3] - 2022/02/23
         - [DCAEGEN2-3048] - Vulnerability fixes for J release (xstream,jackson-core,jackson-databind,gson)
1/21/22 - 1.3.2 released
1/18/22 - 1.3.2 log4j fix - upgrade to 2.17.1

onap/org.onap.dcaegen2.services.pm-mapper:1.8.0

YES

1.8.0

2/15/22 - 1.8.0 released
2/14/22 - Allow supports unauthenticated topic and connection without TLS
- Make cert paths field optional in configuration.
- Allow to skip ssl context load.
- Make PublisherConfig and SubscriberConfig fields optional.
- Remove Auth Header when AAF credentials are empty
1/24/22 - 1.8.0 CBS SDK integration
09/09/21 - 1.7.2 released

onap/org.onap.dcaegen2.services.prh.prh-app-server:1.8.0

YES

1.8.0

2/15/22 - 1.8.0 released
02/14/22 - 1.8.0 vulnerability updates
08/31/21 - 1.7.1 released
08/24/21 - 1.7.1 Update DCAE SDK version from 1.6.0 to 1.8.7
08/11/21 - 1.7.0 released
08/10/21 - 1.7.0 - switch AAI variable syntax from config + v11 to v23 apis

onap/org.onap.dcaegen2.services.son-handler:2.1.6

YES

2.1.6

3/18/22 - 2.1.6 released
2/14/2022 - 2.1.6 Vulnerability updates
2/10/22 - 2.1.6 Fix bug in triggering control loop for PCI collision/confusion - by replacing Config DB with CPS
10/22/21 - 2.1.5 fix for ves fault layout for collision detection (DCAEGEN2-2943)
09/09/21 - 2.1.4 released

onap/org.onap.dcaegen2.platform.mod.onboardingapi:2.13.0

YES

2.13.0

11/12/21 - 2.13.0 image released
09/23/21 - v3 spec integartion (helm support)
02/15/2021 - component spec schema fix to include required parameters for config_map support
01/29/2021 - component-spec-schema updated to support: kafka stream subscribes and publishes, config maps as volumes

onap/org.onap.dcaegen2.platform.mod.distributorapi:1.1.0

NO

1.1.0

9/14 - 1.1.0 released
8/19 - 1.1.0 python 3.8 support

onap/org.onap.dcaegen2.platform.mod.designtool-web:1.0.2

NO

1.0.2

3/30 - 1.0.2 - nonroot change
1.0.0-SNAPSHOT-20200217T165545Z

onap/org.onap.dcaegen2.platform.mod.genprocessor-http:1.0.2

NO

1.0.2

7/13 - 1.0.2 - Guilin sec vulnerability updates
3/30 - 1.0.1 - nonroot change
1.0.0-20200217T215940Z

onap/org.onap.dcaegen2.platform.mod.genprocessor-job:1.0.2

NO

1.0.2

7/8 - vulnerability update
3/30 - 1.0.1 - nonroot change
1.0.0-20200217T215940Z

onap/org.onap.dcaegen2.platform.mod.mod-registry:1.0.0

NO

1.0.0

1.0.0-SNAPSHOT-20200217T165545Z

onap/org.onap.dcaegen2.platform.mod.runtime-web:1.3.1

YES

1.3.2
1.2.4 (Istanbul)

3/4/22 - 1.3.2 release submitted
## [1.3.2] - 2022-01-04
- [DCAEGEN2-3028] Change version format to MMddHHmm (removed YY)
- [DCAEGEN2-3008] Update helmgen-core from snapshot to released version
- [DCAEGEN2-3088] Set spring-boot-maven-plugin to 2.6.2
- [DCAEGEN2-3052] Update helmgen-core to 1.0.3 (includes vulnerability fixes for J)
2/18/22 - 1.2.4 released
2/15/22 - 1.2.4 NFE exception fix on Istanbul branch

onap/org.onap.dcaegen2.services.datalakefeeder:1.1.1

NO

1.1.1 (Istanbul)

08/02/2021 -1.1.1 fixed elasticsearch entry + Released
05/11/2021 - 1.1.1 DL feeder released (base image + non-root)
04/19/2021 - 1.1.1 - DL feeder  integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)
04/12/21 - 1.1.1 - DES integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)
2/24/2021 - 1.1.1 for non-root db access
7/29 - java11 upgrade; suggested 1.1.0 version
4/6  - 1.0.2 being released
3/21 - 1.0.2 for non-root

onap/org.onap.dcaegen2.services.datalakeadminui:1.1.1

NO

1.1.1  (Istanbul)

09/03/21 - 1.1.1 released
04/12/21 - 1.1.1 - DES integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)
2/24/2021 - 1.11 docker build error
9/11 - Std o/p log
4/6  - 1.0.2 being released
3/21 - 1.0.2 for non-root

onap/org.onap.dcaegen2.services.pmsh:2.2.2

YES

2.2.2

03/22/21 - 2.2.2 released
03/21/22 - 2.2.2 ## [2.2.2]
### Changed
* Bug fix for Create MG (DCAEGEN2-3118)
3/14/22 - 2.2.1 released
  * Swagger Indentation Error (DCAEGEN2-3103)
2/8/22 - 2.0.0 released
Nov-DEC/21 - 2.0.0 several changes (refer Changelog.md)

onap/org.onap.dcaegen2.platform.adapter.acumos:1.0.7

YES

1.0.7

02/24/22 - 1.0.7 released
12/15/2021 - 1.0.7 new v3 schema support for helm flow integration
09/13/2021 - 1.0.6 release file submitted; verified in Highstreet lab

onap/org.onap.dcaegen2.services.components.slice-analysis-ms:1.1.1

YES

1.1.1 (CCVPN/IBN support)
1.0.7 

04/24/22 - 1.1.1 released         - [DCAEGEN2-3142](DCAEGEN2-3142: Filter data from AAI to avoid possible exceptions, remove null parameters in policy payload and add logsClosed) - Filter data from AAI to avoid possible exceptions, remove null parameters in policy payload and add logs

03/22/22 - 1.1.0 released
[SLICEANALYSIS] Release image 1.1.0

    - Jakarta changes in slice-analysis-ms for IBN Cloud leased line update and CCVPN closed-loop.
3/16/22 - 1.1.0 CCPVN/IBM update under review
3/16/22 - 1.0.7 released
[SLICEANALYSIS] Release image 1.0.7

     - Use onap/integration-java11 image
     - Switch CBS client library to 1.8.7
     - Fix null pointer exception while fetching slice-config
     - Remove security vulnerabilities
     - Calculate slice utilization data
     - Migrate SliceAnalysis MS to use unauthenticated topic
     - Update onset message test data
2/10/22 - 1.0.7 vulnerability updates
12/16/21 - 1.0.7 use onap/integration-java11 image

onap/org.onap.dcaegen2.services.datalake.exposure.service:1.1.1

NO

1.1.1 (Istanbul)

5/10/21 - 1.1.1 released (non root + base image change)
04/13/21 - 1.1.1 merged
04/12/21 - 1.1.1 - DES integration-java11:8.0.0 update + non rootdb (switch mariadb to pg)

onap/org.onap.dcaegen2.platform.ves-openapi-manager:1.1.0

YES

1.1.0

2/24 - 1.0.1 released 
02/19 - Introduced for H release

onap/org.onap.dcaegen2.deployments.dcae-services-policy-sync:1.0.1

NO

1.0.1

04/23/2021 - 1.0.1 released
04/22/2021 - 1.0.1 - include onap pem certificate + file umask fix
02/25/2021 - 1.0.0 released

onap/org.onap.dcaegen2.services.components.kpi-ms:1.0.4

YES

1.0.4

3/18/22 - 1.0.4 released
3/16/22 - 1.0.4 release in gerrit
2/10/22 - 1.0.4 vulnerability fixes

onap/org.onap.dcaegen2.platform.mod.auth-service:1.0.0

POC

1.0.1 (Not released

04/30 - 1.0.1 : int base image 8.0.0

onap/org.onap.dcaegen2.platform.mod.catalog-service:1.0.0

POC

1.0.2 (Not released

04/30 - 1.0.1 : int base image 8.0.0

onap/org.onap.dcaegen2.platform.mod.ui:1.0.0

POC

1.0.0 (Not released)

helmgenerator-core (java library)

YES

1.0.2

10/13/21 - [DCAEGEN2-2936] Convert streams_publishes and streams_subscribes json strings under applicationConfig to map
10/04/21 - *  [DCAEGEN2-2911] Refactor the code to make it more testable
    - Converted some static methods to instance methods
*  [DCAEGEN2-2911] Distributor throws an error if distribution fails
*  [DCAEGEN2-2917] Add helm repo registry step when initializing the helm client
    - Add "helm repo add ..."
    - Add "helm deployment update" before linting

Service  SDK : 1.8.8

Yes

1.8.8

02/09/21 - 1.8.8 released
02/08/21 - 1.8.8 - Spring-Boot version to 2.5.9
08/02/21 - 1.8.7 released
07/30/31 - 1.8.7 Make CBS-Client config and policy file paths configurable by environment variables