Page Comparison

Wiki to track the design requirements for Helm generator to support 

SUB-TASKS/USECASES

1. Build helm chart generator taking following inputs (values.yaml) and templates to generate chart package and perform lint for consolidated charts.

Helm Chart directory structure 

templates:

configmap.yaml

deployment.yaml

service.yaml

secret.yaml

Chart.yaml

requirements.yaml

values.yaml

(Include dependent charts along with main chart so that the validation can be done)

                Helm lint (checks for syntax) – Initial validation can be just done with lint.

                Validation needs to be configured (default – enabled and can be disabled) 

            a. Run it as standalone tool -  so that it can be run with the given input and generate helm chart package.

                Note: This is for testing purpose only until the integration with Catalog Service is done.

                           tool parameters are Input template directory and output as tar -

2. Separate Values.yaml into separate templates and verify dynamic values.yaml generation (and parameter substitution

                Configuration/parameters required common across MS ( highlighted in yellow in ppt)

Configuration/parameters to be templatized and values sourced from ComponentSpec ( highlighed in Red )

Note : Optional configuration/parameters to be templatized and included based on flags/properties from ComponentSpec (Covered under 6 to 10)

The generator must consolidate these separate base values.template and created required values.yaml

<Need to add from ppt>

                TEST: Generated charts must be validated in ONAP lab K8S environment

Refer Sample Chart Yaml mapping from component spec and Sample Values yaml mapping from component spec in the requirements document.

stream_publish/stream_subscribe handle as string; change VES spec and test 

readiness mapping  Include PORT/initialdelaySeconds into auxillary.healthcheck on component spec schema as optional parameter for mapping into readiness check

Image Removed

globalNodeportPrefix - make default from base template. (It should be already supported)

3. Identify Component-spec schema changes for ENV setting mapping  (refer REQ #1 below) 

List is not supported in current common/service template

4. Identify Component-spec schema changes for Service mapping (and nodeport) (refer REQ #7 below) 

5. Build helm chart generator taking as inputs template directory and template list file to be used for chart generation and perform lint for consolidated charts. - No longer applicable; based on design 17 Jun 2021 

Note: Use base/default template if corresponding template not found on specified template directory

6. Support MAPPING requirement – ENV SETTING (refer REQ #1 for details) - 

Schema Validation to be integrated - tracked separately under task #24

7. Support MAPPING requirement – CMPv2 Certificates (refer REQ #3 for details) 

Include support for base + addon chart structure (certificate.yaml will be optional) 

Add validation if enabled in spec and certificate.yaml not provided - tool must error - switch to use use_external_tls/flag  

8. Support MAPPING requirement – Postgres (refer REQ #5 for details) 

9. Support MAPPING requirement – Policy Sidecar (refer REQ #4  for details)

10. Support SERVICE MAPPING based on spec file (refer REQ #7 for details) 

Schema Validation to be integrated - tracked separately under task #24

11. Support MAPPING requirement – ConfigMap support (refer REQ #2 for details) 

12. Support MAPPING requirement – DMAAP Secure Topic/Feed (refer REQ #6 for details)

13. Create user guide for the tool detailing all command-line options/override

14. Submit code to ONAP; ensure compliance to ONAP coding standard and test coverage requirement (atleast 80%) and verify the library build/pushed to ONAP:nexus

15. Integrate tool into MOD/Runtime or MOD2/CatalogService

16. Verify E2E for ONAP DCAE MS spec file (TCA w/policy, PM-Mapper, VES, and validate corresponding charts generated in ONAP lab if components can be successfully deployed

17. Add distribution support in tool. Additional configuration support needed for below parameters either in tool property file or CMD line options

DistributionEnabled 

DistributionURL:endpoint 

DistributionUsername

DistributionPwd

DistributionFormat - tgz or as directory

ONAP - https://chartmuseum.com/docs/

18. Provide REST interface to support HELM generation

19. Input Spec validation  (duplicate to #24)

20. Comments should be cascaded from template and when new yaml is generator and also preserve order

21. Secret mapping requirement TBD and pv mapping  

      - CMPv2 + application_env

       Option 1: Change spec schema to include secret name as additional input and use that for helm values.yaml setting

       Option 2: Have tool set the secret name as <component-name>-cmpv2-keystore-password   -  Current chosen option  

22. Support chart generation as directory (instead of tgz) via configuration 

23. Support dependency download (helm dep up) from repo/access based on deployment configration 

24. Add schema validator module; keep the schema definiton overriddable (either through CI or api parameter); this should be configurable  - default enabled (added on 11 Jun 2021  based on svc mapping test)  

           - default schema to be included in tool (optional override support) 

           - validation NPE for schema missing (pick up from resource folder) 

           -  Update gerrit with latest spec schema (initialdelayseconds = integer)

           - Add wrapper class to consolidate the error - Dhrumin (moved to task #30

25. Testing with ONAP spec files on ONAP lab with componen-spec (ves, pm-mapper, dfc, tcagen2, hv-ves, prh) 

               Running the tool requires following

                     - helmchartgenerator jar file

                     - component spec file

                     - Directory holding addon templates (such as certificates) under addons and base charts and templates (such as common, dcaegen2-services-common, readinessCheck, repositoryGenerator) under base directory

                     - Optional specSpecma json file ( if not provided, it will not validate the schema as of 7/16/21)

       For example, 

            java -jar ./helmchartgenerator-1.3.5-SNAPSHOT.jar vescollector-componentspec-v3-helm.json ./blueprint ./output schemas/specSchema.json

            java -jar helmchartgenerator-1.3.5-SNAPSHOT.jar ./input/specs/hv-ves-collector.componentspec-new.json ./input/blueprint ./output ./schemas/specSchema.json

       helm install command

           helm install --set global.dockerHubRepository=nexus3.onap.org:10001 dev-dcae-ves-collector -n onap ./dcae-ves-collector-1.8.0.tgz --set global.pullPolicy=IfNotPresent

• Modify spec to align with new schema
  • https://git.onap.org/dcaegen2/collectors/hv-ves/tree/dpo/spec/hv-ves-collector.componentspec.json (pending test)
  • https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json
  • https://git.onap.org/dcaegen2/analytics/tca-gen2/tree/dcae-analytics/dpo/tcagen2_spec.json  
  • https://git.onap.org/dcaegen2/services/pm-mapper/tree/dpo/spec/pmmapper-component-spec.json (pending test)
  • https://git.onap.org/dcaegen2/collectors/datafile/tree/datafile-app-server/dpo/spec/datafile-component-spec.json (pending test)
  • https://git.onap.org/dcaegen2/services/prh/tree/dpo/spec/prh.componentspec.json (pending test)
• Generate charts and deploy and check if component comes up clean
• Environment for test to be setup/checked (onap5) 
• Verify spec can be run through bp-gen and deployed (backward compatibility) 
  • New spec schema not backward compatible; bp-gen and onboardingAPI needs to be updated to reflect schema + onboardingAPI?
  • New spec schame will be added under v3 directory

        26. Submit updates on each spec json update to gerrit (to be verified if updated spec breaks bp-gen creation - incase new fields introduced are not ignored)

27. Support for readinesscheck for script based execution (e.g PMMapper/hv-ves); port/endpoint flagged as mandatory fields which are required only for HTTP  - added 30 Jun 2021

                    Use docker_healthcheck_script

                     V3 spec schema change needed to include   initialDelaySeconds: 5  

                     Map to helm charts as below   

readiness:
  type: exec   
  initialDelaySeconds: 5  # set from spec docker_healthcheck_script.Properties.initialDelaySeconds
  periodSeconds: 15     # set from spec docker_healthcheck_script.Properties.interval
  timeoutSeconds: 2     # set from spec docker_healthcheck_script.Properties.timeout
  command:
  - /opt/ves-hv-collector/healthcheck.sh  # set from spec docker_healthcheck_script.Properties.script

28. Dynamic secret creation and mapping to env variable - support from Helm tool - added 30 Jun 2021 

29.  Postgres should be enabled/disabled based on requirement.yaml and take values from values.yaml - added 08 Jul 2021 

             Require base template change (requirement.yaml) & values.yaml setting to true when PG is enabled 

30. Spec schame validator (current library- https://github.com/everit-org/json-schema) validation error not verbose. Evaluate internal validator class or below library used for VEScollector  - added on 12 Jul 2021 

     <groupId>com.networknt</groupId>
      <artifactId>json-schema-validator</artifactId>
      <version>1.0.49</version>

31.  OnboardingAPI changes to use V3 spec files for supporting helm spec files - added 12 Jul 2021  

32. Syncup latest oom/templates under repo:test/blueprint   (DUPLICATE)

33. Include policyRelease: "onap" for policysidecar config and remove dcaePolicySyncImage

34. Support for helm generator cli input based on flags (instead of order)

REQUIREMENTS

1. ENV SETTING SUPPORT

...