Page Comparison

Table of Contents

Introduction

In Casablanca release, MSB project is integrating Istio Service Mesh with ONAP to manage ONAP microservices. Istio Service Mesh is a dedicated infrastructure layer to connect, manage and secure microservices, which brings the below benefits:

• Stability and Reliability: Reliable communication with retries and circuit breaker 
• Security: Secured communication with TLS 
• Performance: Latency aware load balancing with warm cache
• Observability: Metrics measurement and distributed tracing without instrumenting application 
• Manageability: Routing rule and rate limiting enforcement 
• Testability: Fault injection to test resilience of the services

Installation

Currently, the installation scripts are in Github, they will be moved to ONAP Gerrit once the requested repo is created.

Download installation scripts with git clone:

git clone https://github.com/zhaohuabing/istio-install-scripts.git

Kubernetes Master

We need Kubernetes1.9 or newer to enable automatic sidecar injection, so we don't have to modify every individual ONAP kubernetes yaml deployment files to add the sidecar container, which would be inconvenient.

Istio leverages the webhook feature of Kubernetes to automatically inject an Envoy sidecar to each Pod. Kubernetes API server will call the Istio sidecar injection webhook when it receives a request to create a Pod resource, the webhook adds an Envoy sidecar container to the Pod, then the modified Pod resource is stored into etcd.    

Webhook and other needed features have already been configured in the install scripts.

Create the Kubernetes master by running this script:

cd istio-install-scripts
./1_install_k8s_master.sh

This script will create a Kubernetes master node with Kubeadm and install calico network plugin. Some other needed tools such as Docker, Kubectl and Helm will also be installed.

From the output of the script, you should see a command on how to join a node to the created Kubernets cluster. Note that this is an example, the token and cert-hash of your installation will be different, please copy & paste the command to somewhere, we will need it later.

You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.12.5.104:6443 --token 1x62yf.60ys5p2iw13tx2t8 --discovery-token-ca-cert-hash sha256:f06628c7cee002b262e69f3f9efadf47bdec125e19606ebff743a3e514a8383b

Kubernetes worker Node

...

Table of Contents

Introduction

In Casablanca release, MSB project is integrating Istio Service Mesh with ONAP to manage ONAP microservices. Istio Service Mesh is a dedicated infrastructure layer to connect, manage and secure microservices, which brings the below benefits:

• Stability and Reliability: Reliable communication with retries and circuit breaker 
• Security: Secured communication with TLS 
• Performance: Latency aware load balancing with warm cache
• Observability: Metrics measurement and distributed tracing without instrumenting application 
• Manageability: Routing rule and rate limiting enforcement 
• Testability: Fault injection to test resilience of the services

Installation

Download installation scripts from ONAP Gerrit:

git clone https://gerrit.onap.org/r/msb/service-mesh

Kubernetes Master

We need Kubernetes1.9 or newer to enable automatic sidecar injection, so we don't have to modify every individual ONAP kubernetes yaml deployment files to add the sidecar container, which would be inconvenient.

Istio leverages the webhook feature of Kubernetes to automatically inject an Envoy sidecar to each Pod. Kubernetes API server will call the Istio sidecar injection webhook when it receives a request to create a Pod resource, the webhook adds an Envoy sidecar container to the Pod, then the modified Pod resource is stored into etcd.    

Webhook and other needed features have already been configured in the install scripts to enable Istio sidecar injection.

Create the Kubernetes master by running this script:

./2_installcd service-mesh/install/
./1_install_k8s_minionmaster.sh

You can now join this machines by running "kubeadmin join" command as root:

sudo kubeadm join 10.12.5.104:6443 --token 1x62yf.60ys5p2iw13tx2t8 --discovery-token-ca-cert-hash sha256:f06628c7cee002b262e69f3f9efadf47bdec125e19606ebff743a3e514a8383b

Please note that this is just an example, please refer to the output of the "kubeamin init" when creating the k8s master for the exact command to use in your k8s cluster.

If you would like to get kubectl talk to your k8s master, you need to copy the administrator kubeconfig file from your master to your workstation like this:

...

This script will create a Kubernetes master node with Kubeadm and install calico network plugin. Some other needed tools such as Docker, Kubectl and Helm will be installed as well.

From the output of the script, you should see a command on how to join a node to the created Kubernets cluster. Note that this is an example, the token and cert-hash of your installation will be different, please copy & paste the command to somewhere, we will need it later.

scp root@<master ip>:/etc/kubernetes/admin.conf .
kubectl --kubeconfig ./admin.conf get nodes

or you can manually copy the content of this file to ~/.kube/conf if scp can't be used due to security reason.

Istio Control Plane

Install Istio by running this script:

./ 3_install_istio.sh

This script installs the followings Istio components:

• Install Istioctl command line tool in the /usr/bin directory
• Install Istio control plane components, including Pilot, Citadel, Mixer
• Install addons including servicegraph, Promeheus, Grafana, jaeger

Confirm Istio was installed:

kubectl get svc -n istio-system
NAME      You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.12.5.104:6443 --token 1x62yf.60ys5p2iw13tx2t8 --discovery-token-ca-cert-hash sha256:f06628c7cee002b262e69f3f9efadf47bdec125e19606ebff743a3e514a8383b

Kubernetes worker Node

Log in the worker node machine, run this script to create a kubernetes worker node:

./2_install_k8s_minion.sh

You can now join this machines by running "kubeadmin join" command as root:

sudo kubeadm join 10.12.5.104:6443 --token 1x62yf.60ys5p2iw13tx2t8 --discovery-token-ca-cert-hash sha256:f06628c7cee002b262e69f3f9efadf47bdec125e19606ebff743a3e514a8383b

Please note that this is just an example, please refer to the output of the "kubeamin init" when creating the k8s master for the exact command to use in your k8s cluster.

If you would like to get kubectl talk to your k8s master, you need to copy the administrator kubeconfig file from your master to your workstation like this:

scp root@<master ip>:/etc/kubernetes/admin.conf .
kubectl --kubeconfig ./admin.conf get nodes

or you can manually copy the content of this file to ~/.kube/conf if scp can't be used due to security reason.

Istio Control Plane

Install Istio by running this script:

./ 3_install_istio.sh

This script installs the followings Istio components:

• Install Istioctl command line tool in the /usr/bin directory
• Install Istio control plane components, including Pilot, Citadel, Mixer
• Install addons including servicegraph, Promeheus, Grafana, jaeger

Confirm Istio was installed:

kubectl get svc -n istio-system
NAME                       TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                               AGE
grafana                    NodePort       10.109.190.71    <none>        3000:30300/TCP                                                        20m
istio-citadel              ClusterIP      10.106.185.181   <none>        8060/TCP,9093/TCP         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                       20m
istio-egressgateway        ClusterIP      10.102.224.133   <none>        80/TCP,443/TCP              AGE grafana                    NodePort       10.109.190.71    <none>        3000:30300/TCP  20m
istio-ingressgateway       LoadBalancer   10.100.168.32    <pending>          80:31380/TCP,443:31390/TCP,31400:31400/TCP                             20m
istio-citadelpilot                ClusterIP      10.106101.18564.181153    <none>        806015003/TCP,15005/TCP,15007/TCP,15010/TCP,15011/TCP,8080/TCP,9093/TCP                                          TCP   20m
istio-policy       20m istio-egressgateway        ClusterIP      10.102104.22411.133162    <none>        809091/TCP,15004/TCP,4439093/TCP                                                        20m
istio-ingressgatewaysidecar-injector     ClusterIP  LoadBalancer    10.100.168229.3240    <pending>     80:31380/TCP,443:31390/TCP,31400:31400/TCP<none>              443/TCP              20m istio-pilot                ClusterIP      10.101.64.153    <none>        15003/TCP,15005/TCP,15007/TCP,15010/TCP,15011/TCP,8080/TCP,9093/TCP   20m istio-policy          20m
istio-statsd-prom-bridge    ClusterIP      10.104107.1127.16291     <none>        90919102/TCP,15004/TCP,9093/TCP9125/UDP                                                     20m
istio-sidecar-injectortelemetry            ClusterIP      10.100101.229153.40114    <none>        4439091/TCP,15004/TCP,9093/TCP,42422/TCP                                 20m
prometheus                 ClusterIP      10.103.0.205      20m
istio-statsd-prom-bridge<none>   ClusterIP      10.107.27.919090/TCP     <none>        9102/TCP,9125/UDP                                                 20m
servicegraph   20m istio-telemetry           NodePort ClusterIP      10.101106.15349.114168    <none>        90918088:30088/TCP,15004/TCP,9093/TCP,42422/TCP                                 20m prometheus                 ClusterIP      10.103.0.20520m
tracing     <none>        9090/TCP       LoadBalancer   10.100.158.236   <pending>     80:30188/TCP                                            20m servicegraph             20m
 NodePort  zipkin     10.106.49.168    <none>        8088:30088/TCP    NodePort       10.96.164.255    <none>        9411:30411/TCP                                 20m tracing                    LoadBalancer   10.100.158.236   <pending>     80:30188/TCP                  20m

Sidecar Injection

In the transition phase, the Istio sidecar injector policy is configured as "disabled" when installing Istio. So the sidecar injector will not inject the sidecar into pods by default. Add the `sidecar.istio.io/inject annotation` with value `true` to the pod template spec to enable injection.

Example:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: {{ include "common.fullname" . }}
  namespace: {{ include "common.namespace" . }}
  labels:
    app: {{ include "common.name" . }}
    chart: {{ .Chart.Name          }}-{{ .Chart.Version | replace "+" "_" }}
    release: {{ .Release.Name }}
    heritage:  20m
zipkin {{ .Release.Service }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
     NodePort app: multicloud-vio
    10.96.164.255template:
   <none> metadata:
      9411labels:30411/TCP
        app: {{      include "common.name" . }}
        release: {{ .Release.Name }}
      name: {{ include "common.name" . }}
      annotations:
        20m

Sidecar Injection

...

sidecar.istio.io/inject: "{{.Values.istioSidecar}}"

Note: when all ONAP projects are ready for Istio integration, the Istio sidecar injector policy could be configured as "enabled", then the annotation in the pod will not be necessary any more.

...

In Casablanca, MSB project is working with VF-C and MultiCloud to verify Istio integration, so we are focusing on these three projects right now. More projects will engage later.as pilot projects, we would like to roll out it to the other ONAP projects after verifying the integration and Istio features.

helm install local/msb -n msb --namespace onap
helm install local/vfc -n vfc --namespace onap
helm install local/multicloud -n multicloud -n multicloud --namespace onap--namespace onap

Note that you can also install other ONAP projects with helm install if they are needed. But Istio sidecar will not be injected to their Pods by default.

Confirm that ONAP microservices have been started

...

You can open the MSB portal http://Node_IP:30280/iui/microservices/default.html in the browser to see all the registered services.

Image RemovedImage Added

Explore

...

Istio Features

Distributed Tracing

First, let's generate some traffics in the application, access the following URLs with curl command or open them in the browser

...

Then open your browser at http://tracing_node_ip:tracing_node_port/, you should see something similar to the following:

Note

• Tracing_node_port can be found by 'kubctl get svc -n istio-system'.
• ONAP microservices need to propagate the appropriate HTTP headers so that when the proxies send span information, the spans can be correlated correctly into a single trace.

Service Graph

Istio provides a Servicegraph service which generates and visualizes graph representations of the services in the mesh.

Open your browser at httpat http://node_ip:30088/dotviz or httpdotviz or http://node_ip:30088/force/forcegraph.html, you should see the service graph:

 Image Modified

Metrics Visualization

Istio automatically gathers telemetry for services in a mesh. A Prometheus adapter is plugged into Mixer to serve the generated metric data. A Grafana addon is pre-configured with a Prometheus data source and has an Istio dashboard installed for the metric visualization.

Open your browser at http://node_ip:30300, you should see the Grafana Istio dashboard:

Image Added