ONAP "Networking" Options (>=Kohn)

Owned by Andreas Geißler
Last updated: Jul 20, 2022
29 min read

Communication patterns

  • Intra-Component communication (e.g. between so-bpmn-infra and so-sdnc-adapter)

  • Inter-Component communication (e.g. between onap-cli and so)

  • External communication (e.g. user → sdc-ui)

Assumptions (to be agreed)

  • AAF will be removed

    • → No Container port encryption

  • Services must not use NodePorts 

    • → external communication only via Ingress

  • Ingress is the default for external communication

    • Istio IngressGateway

    • Nginx Ingress ?

    • Rules for URLs (<comp-api>.<base-url>)

      • Background: wildcard-certificate usually covers just 1 level e.g. a.simpledemo.onap.org, not b.a.simpledemo.org

      • current Ingress settings (see HOSTS):

        Current Ingress APIs

        NAME GATEWAYS HOSTS AGE onap-aaf-cm-service ["onap-aaf-cm-gateway"] ["aafcm.simpledemo.onap.org"] 8h onap-aaf-fs-service ["onap-aaf-fs-gateway"] ["aaffs.simpledemo.onap.org"] 8h onap-aaf-gui-service ["onap-aaf-gui-gateway"] ["aafgui.simpledemo.onap.org"] 8h onap-aaf-locate-service ["onap-aaf-locate-gateway"] ["aaflocate.simpledemo.onap.org"] 8h onap-aaf-oauth-service ["onap-aaf-oauth-gateway"] ["aafoauth.simpledemo.onap.org"] 8h onap-aaf-service-service ["onap-aaf-service-gateway"] ["aafservice.simpledemo.onap.org"] 8h onap-aai-babel-service ["onap-aai-babel-gateway"] ["aaibabel.simpledemo.onap.org"] 8h onap-aai-service ["onap-aai-gateway"] ["aai.api.simpledemo.onap.org"] 8h onap-aai-sparky-be-service ["onap-aai-sparky-be-gateway"] ["aaisparkybe.simpledemo.onap.org"] 8h onap-cds-blueprints-processor-service ["onap-cds-blueprints-processor-gateway"] ["blueprintsprocessorhttp.simpledemo.onap.org"] 8h onap-cds-ui-service ["onap-cds-ui-gateway"] ["cdsui.simpledemo.onap.org"] 8h onap-cli-service ["onap-cli-gateway"] ["cli.api.simpledemo.onap.org","cli2.api.simpledemo.onap.org"] 8h onap-consul-service ["onap-consul-gateway"] ["consul.api.simpledemo.onap.org"] 8h onap-cps-core-service ["onap-cps-core-gateway"] ["cps-core.simpledemo.onap.org"] 8h onap-cps-temporal-service ["onap-cps-temporal-gateway"] ["cps-temporal.simpledemo.onap.org"] 8h onap-dcaemod-distributor-api-service ["onap-dcaemod-distributor-api-gateway"] ["dcaemod.simpledemo.onap.org"] 8h onap-dcaemod-genprocessor-service ["onap-dcaemod-genprocessor-gateway"] ["dcaemod.simpledemo.onap.org"] 8h onap-dcaemod-onboarding-api-service ["onap-dcaemod-onboarding-api-gateway"] ["dcaemod.simpledemo.onap.org"] 8h onap-dmaap-bc-service ["onap-dmaap-bc-gateway"] ["dmaapbc.simpledemo.onap.org"] 8h onap-dmaap-dr-node-service ["onap-dmaap-dr-node-gateway"] ["dmaapdrnode.simpledemo.onap.org"] 8h onap-dmaap-dr-prov-service ["onap-dmaap-dr-prov-gateway"] ["dmaapdrprov.simpledemo.onap.org"] 8h onap-msb-consul-service ["onap-msb-consul-gateway"] ["msbconsul.simpledemo.onap.org"] 8h onap-msb-discovery-service ["onap-msb-discovery-gateway"] ["msb.api.discovery.simpledemo.onap.org"] 8h onap-msb-eag-service ["onap-msb-eag-gateway"] ["msbeag.simpledemo.onap.org"] 8h onap-msb-iag-service ["onap-msb-iag-gateway"] ["msbiag.simpledemo.onap.org"] 8h onap-nbi-service ["onap-nbi-gateway"] ["nbi.api.simpledemo.onap.org"] 8h onap-ncmp-dmi-plugin-service ["onap-ncmp-dmi-plugin-gateway"] ["ncmp-dmi-plugin.simpledemo.onap.org"] 8h onap-oof-has-api-service ["onap-oof-has-api-gateway"] ["oof-has-api.onap.simpledemo.onap.org"] 8h onap-oof-service ["onap-oof-gateway"] ["oofosdf.simpledemo.onap.org"] 8h onap-policy-gui-service ["onap-policy-gui-gateway"] ["policygui.api.simpledemo.onap.org"] 8h onap-robot-service ["onap-robot-gateway"] ["robot.api.simpledemo.onap.org"] 8h onap-sdc-be-service ["onap-sdc-be-gateway"] ["sdc.api.be.simpledemo.onap.org"] 8h onap-sdc-fe-service ["onap-sdc-fe-gateway"] ["sdc.api.fe.simpledemo.onap.org"] 8h onap-sdc-wfd-be-service ["onap-sdc-wfd-be-gateway"] ["sdcwfdbe.simpledemo.onap.org"] 8h onap-sdc-wfd-fe-service ["onap-sdc-wfd-fe-gateway"] ["sdcwfdfe.simpledemo.onap.org"] 8h onap-sdnc-dgbuilder-service ["onap-sdnc-dgbuilder-gateway"] ["sdnc-dgbuilder.simpledemo.onap.org","sdnc-web-service.simpledemo.onap.org"] 8h onap-sdnc-service ["onap-sdnc-gateway"] ["sdnc.api.simpledemo.onap.org"] 8h onap-so-admin-cockpit-service ["onap-so-admin-cockpit-gateway"] ["soadmincockpit.simpledemo.onap.org"] 7h47m onap-so-etsi-nfvo-ns-lcm-service ["onap-so-etsi-nfvo-ns-lcm-gateway"] ["soetsinfvonslcm.simpledemo.onap.org"] 7h47m onap-so-etsi-sol003-adapter-service ["onap-so-etsi-sol003-adapter-gateway"] ["soetsisol003adapter.simpledemo.onap.org"] 7h47m onap-so-service ["onap-so-gateway"] ["so.api.simpledemo.onap.org"] 7h47m onap-uui-server-service ["onap-uui-server-gateway"] ["uuiserver.simpledemo.onap.org"] 7h44m onap-uui-service ["onap-uui-gateway"] ["uui.api.simpledemo.onap.org"] 7h44m onap-vnfsdk-service ["onap-vnfsdk-gateway"] ["refrepo.simpledemo.onap.org"] 7h44m



      • → should we make a common rule for Ingress URLs, e.g. 

        • don't use sub-urls (e.g. aai.api), but use dash (e.g. aai-api)

        • use "-api" for apis, use "-ui" for UIs

        • use common way of naming: <component>-<application>-<api|ui>

        • Possible result:



        • Proposal for Ingress API Names

          NAME GATEWAYS HOSTS AGE onap-aaf-cm-service ["onap-aaf-cm-gateway"] ["aaf-cm-api.simpledemo.onap.org"] 8h onap-aaf-fs-service ["onap-aaf-fs-gateway"] ["aaf-fs-api.simpledemo.onap.org"] 8h onap-aaf-gui-service ["onap-aaf-gui-gateway"] ["aaf-ui.simpledemo.onap.org"] 8h onap-aaf-locate-service ["onap-aaf-locate-gateway"] ["aaf-locate-api.simpledemo.onap.org"] 8h onap-aaf-oauth-service ["onap-aaf-oauth-gateway"] ["aaf-oauth-api.simpledemo.onap.org"] 8h onap-aaf-service-service ["onap-aaf-service-gateway"] ["aaf-service-api.simpledemo.onap.org"] 8h onap-aai-babel-service ["onap-aai-babel-gateway"] ["aai-babel-api.simpledemo.onap.org"] 8h onap-aai-service ["onap-aai-gateway"] ["aai-api.simpledemo.onap.org"] 8h onap-aai-sparky-be-service ["onap-aai-sparky-be-gateway"] ["aai-sparkybe-api.simpledemo.onap.org"] 8h onap-cds-blueprints-processor-service ["onap-cds-blueprints-processor-gateway"] ["cds-blueprintsprocessor-api.simpledemo.onap.org"] 8h onap-cds-ui-service ["onap-cds-ui-gateway"] ["cds-ui.simpledemo.onap.org"] 8h onap-cli-service ["onap-cli-gateway"] ["cli-api.simpledemo.onap.org","cli2-api.simpledemo.onap.org"] 8h onap-consul-service ["onap-consul-gateway"] ["consul-api.simpledemo.onap.org"] 8h onap-cps-core-service ["onap-cps-core-gateway"] ["cps-core-api.simpledemo.onap.org"] 8h onap-cps-temporal-service ["onap-cps-temporal-gateway"] ["cps-temporal-api.simpledemo.onap.org"] 8h onap-dcaemod-distributor-api-service ["onap-dcaemod-distributor-api-gateway"] ["dcaemod-distributor-api.simpledemo.onap.org"] 8h onap-dcaemod-genprocessor-service ["onap-dcaemod-genprocessor-gateway"] ["dcaemod-genprocessor-api.simpledemo.onap.org"] 8h onap-dcaemod-onboarding-api-service ["onap-dcaemod-onboarding-api-gateway"] ["dcaemod-onboarding-api.simpledemo.onap.org"] 8h onap-dmaap-bc-service ["onap-dmaap-bc-gateway"] ["dmaap-bc-api.simpledemo.onap.org"] 8h onap-dmaap-dr-node-service ["onap-dmaap-dr-node-gateway"] ["dmaap-drnode-api.simpledemo.onap.org"] 8h onap-dmaap-dr-prov-service ["onap-dmaap-dr-prov-gateway"] ["dmaap-drprov-api.simpledemo.onap.org"] 8h onap-msb-consul-service ["onap-msb-consul-gateway"] ["msb-consul-api.simpledemo.onap.org"] 8h onap-msb-discovery-service ["onap-msb-discovery-gateway"] ["msb-api.discovery.simpledemo.onap.org"] 8h onap-msb-eag-service ["onap-msb-eag-gateway"] ["msb-eag-api.simpledemo.onap.org"] 8h onap-msb-iag-service ["onap-msb-iag-gateway"] ["msb-iag-api.simpledemo.onap.org"] 8h onap-nbi-service ["onap-nbi-gateway"] ["nbi-api.simpledemo.onap.org"] 8h onap-ncmp-dmi-plugin-service ["onap-ncmp-dmi-plugin-gateway"] ["cps-ncmpdmiplugin-api.simpledemo.onap.org"] 8h onap-oof-has-api-service ["onap-oof-has-api-gateway"] ["oof-has-api.onap.simpledemo.onap.org"] 8h onap-oof-service ["onap-oof-gateway"] ["oof-osdf-api.simpledemo.onap.org"] 8h onap-policy-gui-service ["onap-policy-gui-gateway"] ["policy-ui.api.simpledemo.onap.org"] 8h onap-robot-service ["onap-robot-gateway"] ["robot-api.simpledemo.onap.org"] 8h onap-sdc-be-service ["onap-sdc-be-gateway"] ["sdc-be-api.simpledemo.onap.org"] 8h onap-sdc-fe-service ["onap-sdc-fe-gateway"] ["sdc-fe-api.simpledemo.onap.org"] 8h onap-sdc-wfd-be-service ["onap-sdc-wfd-be-gateway"] ["sdc-wfdbe-api.simpledemo.onap.org"] 8h onap-sdc-wfd-fe-service ["onap-sdc-wfd-fe-gateway"] ["sdc-wfdfe-ui.simpledemo.onap.org"] 8h onap-sdnc-dgbuilder-service ["onap-sdnc-dgbuilder-gateway"] ["sdnc-dgbuilder-api.simpledemo.onap.org","sdnc-webservice-api.simpledemo.onap.org"] 8h onap-sdnc-service ["onap-sdnc-gateway"] ["sdnc-api.simpledemo.onap.org"] 8h onap-so-admin-cockpit-service ["onap-so-admin-cockpit-gateway"] ["so-admincockpit-ui.simpledemo.onap.org"] 7h47m onap-so-etsi-nfvo-ns-lcm-service ["onap-so-etsi-nfvo-ns-lcm-gateway"] ["so-etsinfvonslcm-api.simpledemo.onap.org"] 7h47m onap-so-etsi-sol003-adapter-service ["onap-so-etsi-sol003-adapter-gateway"] ["so-etsisol003adapter-api.simpledemo.onap.org"] 7h47m onap-so-service ["onap-so-gateway"] ["so-api.simpledemo.onap.org"] 7h47m onap-uui-server-service ["onap-uui-server-gateway"] ["uui-server-api.simpledemo.onap.org"] 7h44m onap-uui-service ["onap-uui-gateway"] ["uui-ui.simpledemo.onap.org"] 7h44m onap-vnfsdk-service ["onap-vnfsdk-gateway"] ["vnfsdk-refrepo-api.simpledemo.onap.org"] 7h44m



  • Inter-component communication can be 

    • directly (as today)

    • via Ingress (Seshu's proposal) ?

  • Communication encryption can be done:

    • on Ingress level (adding certificate to Gateway)

    • on SM (e.g. Istio sidecars)

    • on Kernel Level (using eBPF via Cilium)

ONAP Setups (supported by OOM)

Default Secure ONAP setup

  • Discussed and agreed with SECCOM Meeting (19.07.22)

  • External communication:

    • Components expose (external) interfaces to Ingress 

    • Encryption on Ingress (optional)

  • Internal communication: 

    • Service Mesh enabled

    • No TLS port encryption on pods

    • Direct encrypted inter-component communication (via sidecars)

Solution using Istio (ONAP components deployed on one k8s cluster):

 



Solution using Istio (ONAP components deployed on different k8s clusters):




Alternative future solution using eBPF via Cilium:

https://cilium.io/blog/2020/11/10/ebpf-future-of-networking/
https://ebpf.io/

Also supported in Istio (Merbridge): https://istio.io/latest/blog/2022/merbridge/



Alternative (insecure options)

Option 1 (no ONAP internal Encryption)

  • External communication:

    • Components expose (external) interfaces to Ingress 

    • Encryption on Ingress (optional)

  • Internal communication: 

    • No service Mesh

    • No TLS port encryption on pods

    • Direct unencrypted inter-component communication

Option 2 (inter-component encryption)

  • External communication:

    • Components expose (external) interfaces to Ingress 

    • Encryption on Ingress (optional)

  • Internal communication: 

    • No service Mesh

    • No TLS port encryption on pods

    • Inter-component communication via Ingress (encrypted)