External Kafka Access via Ingress
- Andreas Geißler
https://strimzi.io/docs/operators/latest/configuring.html
https://strimzi.io/docs/operators/latest/configuring.html#proc-accessing-kafka-using-ingress-str
https://strimzi.io/blog/2019/04/23/accessing-kafka-part-2/
Current Setup - no Ingress (Kohn):
External Access via Nodeports
onap-strimzi-kafka-external-bootstrap (30493)
onap-strimzi-kafka-0 (30490)
onap-strimzi-kafka-1 (30491)
onap-strimzi-kafka-2 (30492)
TLS termination on Kafka Pods
External Access to Kafka (DT implementation) in Jakarta/Kohn
External Access via Ingress (Traefik)
new TCP "EntryPoints" in Traefik Gateway for bootstrap and brokers
Update Pod "clienttls" ports (9093) to use "advertizedHost" and "advertizedPort"
NodePorts not used...
IngressRouteTCP entry to "internal" bootstrap service
Use "tls passthough"
IngressRouteTCP entries to external broker ports
Proposal for London (External Access via Ingress)
External Access via Ingress (istio-ingress)
new TLS ports on Ingress Gateway for bootstrap and brokers
Disable TLS on "external" broker ports
Disable all Nodeports in Service definitions
Need to check:
https://github.com/istio/istio/issues/20076
Test steps on an existing ServiceMesh cluster
Add custom ports to istio-ingressgateway service
(https://www.dangtrinh.com/2019/09/how-to-open-custom-port-on-istio.html)Modify onap-strimzi-kafka pods and services to disable TLS and set advertizedHosts
Add Ingress Gateway/VCs for onap_strimzi to istio-ingressgateway
Create External Kafka User (optional)
Test the external client access to Kafka
Add custom ports to istio-ingressgateway service
Export existing service definition
Add Custom ports
kubectl -n istio-ingress get service istio-ingressgateway -o yaml > istio_ingressgateway.yamlCheck existing Nodeports (The range of valid ports is 30000-32767) and choose 4 free ports (e.g. 30900, 30901,30902, 30910)
kubectl get svc -A |grep Load
kubectl get svc -A |grep NodePort Edit istio_ingressgateway.yaml and add new ports
- port: 9010
nodePort: 30910
targetPort: 9010
name: kafka-bootstrap
protocol: TCP
- port: 9000
nodePort: 30900
targetPort: 9000
name: kafka-0
protocol: TCP
- port: 9001
nodePort: 30901
targetPort: 9001
name: kafka-1
protocol: TCP
- port: 9002
nodePort: 30902
targetPort: 9002
name: kafka-2
protocol: TCP Apply changes:
Modify onap-strimzi-kafka pods and services to disable TLS and set advertizedHosts
Login to the K8S Control Node and set the helm environment
Modify pods
Modify the onap-strimzi config
Apply the changes to onap-strimzi
Add Ingress Gateway/VCs for onap_strimzi to istio-ingressgateway
Create a file (e.g. kafka-ingress.yaml) Istio Ingress Gateway/VirtualService entries for the kafka-bootstrap-api and the brokers
GW/VC
Apply the file
Add Kafka User for external Access
Create kafka-user.yaml
tls-user.yaml
Apply kafka-user.yaml
Create user
List kafka users
Check/List new user
List strimzi secrets
List user secrets
Get the user password
For each KafkaUser resource with scram-sha-512 auth, there will be a corresponding secret:
Get the user secret
Test the external client access to Kafka
Add hostnames to DNS (or /etc/hosts) by using the IP Address of the istio-ingressgateway LB
Install KafkaCat
Get the Metadata (use an existing Kafka User, here "external-strimzi-kafka-user"):
Get Topic Data (use an existing Kafka User, here "external-strimzi-kafka-user"):
Tasks required for London:
Add Ingress-Gateway "custom port" configuration in OOM Documents
Extend _ingress.tpl to accept
external ports (here 9010,9000,...)
specific settings...
Modify onap-strimzi charts
Add ingress configuration
Update strimzi-kafka configuration to disable TLS in SM case