External Kafka Access via Ingress

https://github.com/strimzi/strimzi-kafka-operator/blob/main/documentation/api/io.strimzi.api.kafka.model.listener.arraylistener.GenericKafkaListener.adoc

Current Setup - no Ingress (Kohn):

External Access via Nodeports
- onap-strimzi-kafka-external-bootstrap (30493)
- onap-strimzi-kafka-0 (30490)
- onap-strimzi-kafka-1 (30491)
- onap-strimzi-kafka-2 (30492)
TLS termination on Kafka Pods

External Access to Kafka (DT implementation) in Jakarta/Kohn

External Access via Ingress (Traefik)
- new TCP "EntryPoints" in Traefik Gateway for bootstrap and brokers
- Update Pod "clienttls" ports (9093) to use "advertizedHost" and "advertizedPort"
- NodePorts not used...
- IngressRouteTCP entry to "internal" bootstrap service
  - Use "tls passthough"
- IngressRouteTCP entries to external broker ports

Proposal for London (External Access via Ingress)

External Access via Ingress (istio-ingress)
- new TLS ports on Ingress Gateway for bootstrap and brokers
Disable TLS on "external" broker ports
Disable all Nodeports in Service definitions

Need to check:
https://github.com/istio/istio/issues/20076

Test steps on an existing ServiceMesh cluster

Add custom ports to istio-ingressgateway service
(https://www.dangtrinh.com/2019/09/how-to-open-custom-port-on-istio.html)
Modify onap-strimzi-kafka pods and services to disable TLS and set advertizedHosts
Add Ingress Gateway/VCs for onap_strimzi to istio-ingressgateway
Create External Kafka User (optional)
Test the external client access to Kafka

Add custom ports to istio-ingressgateway service

Export existing service definition

Add Custom ports

kubectl -n istio-ingress get service istio-ingressgateway -o yaml > istio_ingressgateway.yaml

Check existing Nodeports (The range of valid ports is 30000-32767) and choose 4 free ports (e.g. 30900, 30901,30902, 30910)

kubectl get svc -A |grep Load
kubectl get svc -A |grep NodePort

Edit istio_ingressgateway.yaml and add new ports

  - port: 9010
    nodePort: 30910
    targetPort: 9010
    name: kafka-bootstrap
    protocol: TCP
  - port: 9000
    nodePort: 30900
    targetPort: 9000
    name: kafka-0
    protocol: TCP
  - port: 9001
    nodePort: 30901
    targetPort: 9001
    name: kafka-1
    protocol: TCP
  - port: 9002
    nodePort: 30902
    targetPort: 9002
    name: kafka-2
    protocol: TCP

Apply changes:

Modify onap-strimzi-kafka pods and services to disable TLS and set advertizedHosts

Login to the K8S Control Node and set the helm environment

Modify pods

Modify the onap-strimzi config

- Apply the changes to onap-strimzi

Add Ingress Gateway/VCs for onap_strimzi to istio-ingressgateway

Create a file (e.g. kafka-ingress.yaml) Istio Ingress Gateway/VirtualService entries for the kafka-bootstrap-api and the brokers

GW/VC

Apply the file

Add Kafka User for external Access

Create kafka-user.yaml

tls-user.yaml

Apply kafka-user.yaml

Create user

List kafka users

Check/List new user

List strimzi secrets

List user secrets

Get the user password

For each KafkaUser resource with scram-sha-512 auth, there will be a corresponding secret:

Get the user secret

Test the external client access to Kafka

Add hostnames to DNS (or /etc/hosts) by using the IP Address of the istio-ingressgateway LB

Install KafkaCat

Get the Metadata (use an existing Kafka User, here "external-strimzi-kafka-user"):

- Get Topic Data (use an existing Kafka User, here "external-strimzi-kafka-user"):

Tasks required for London:

Add Ingress-Gateway "custom port" configuration in OOM Documents
Extend _ingress.tpl to accept
- external ports (here 9010,9000,...)
- specific settings...
Modify onap-strimzi charts
- Add ingress configuration
- Update strimzi-kafka configuration to disable TLS in SM case