Versions Compared

Old Version 25

changes.mady.by.user Huabing Zhao

Saved on

compared with

New Version 26

changes.mady.by.user Huabing Zhao

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Introduction

In Casablanca release, MSB project is integrating Istio Service Mesh with ONAP to manage ONAP microservices. Istio Service Mesh is a dedicated infrastructure layer to connect, manage and secure microservices, which brings the below benefits:

...

Table of Contents

Introduction

In Casablanca release, MSB project is integrating Istio Service Mesh with ONAP to manage ONAP microservices. Istio Service Mesh is a dedicated infrastructure layer to connect, manage and secure microservices, which brings the below benefits:

  • Stability and Reliability: Reliable communication with retries and circuit breaker 
  • Security: Secured communication with TLS 
  • Performance: Latency aware load balancing with warm cache
  • Observability: Metrics measurement and distributed tracing without instrumenting application 
  • Manageability: Routing rule and rate limiting enforcement 
  • Testability: Fault injection to test resilience of the services

...

This script will create a Kubernetes master node with Kubeadm and install calico network plugin. Some other needed tools such as Docker, Kubectl and Helm will also be installed.

From the output of the script, you should see a command on how to join a node to the created Kubernets cluster. Note that this is an example, the token and cert-hash of your installation will be different, please copy & paste the command to somewhere, we will need it later.

Code Block
languagebash
themeRDark
You can now join any number of machines by running the following on each node
as root:

  kubeadm join 10.12.5.104:6443 --token 1x62yf.60ys5p2iw13tx2t8 --discovery-token-ca-cert-hash sha256:f06628c7cee002b262e69f3f9efadf47bdec125e19606ebff743a3e514a8383b

Kubernetes worker Node

Log in the worker node machine, run this script to create a kubernetes worker node:

Code Block
languagebash
themeRDark
./2_install_k8s_minion.sh

...

Note: when all ONAP projects are ready for Istio integration, the Istio sidecar injector policy could be configured as "enabled", the annotation in the pod will not be necessary any more.Enable Istio sidecar injection webhookbe configured as "enabled", the annotation in the pod will not be necessary any more.

Enable Istio sidecar injection webhook.

Code Block
languagebash
themeRDark
kubectl create namespace onap
kubectl label namespace onap istio-injection=enabled

Confirm that auto sidecar injection has been enabled on onap namespace.

kubectl get namespace -L istio-injection NAME
Code Block
Code Block
languagebash
themeRDark
languagebash
themeRDark
kubectl create namespace onap
kubectl label namespace onap istio-injection=enabled

Confirm that auto sidecar injection has been enabled on onap namespace.

themeRDark
kubectl get namespace -L istio-injection
NAME           STATUS    AGE       ISTIO-INJECTION
default        Active    20m
istio-system   Active    10m
kube-public    Active    20m
kube-system    Active    20m
onap           Active    8s STATUS    AGE   enabled    ISTIO-INJECTION default

Start a local helm repository server and add it to helm repository list:

Code Block
languagebash
themeRDark
helm serve Active&
helm repo add 20m
istio-system   Active    10m
kube-public    Active    20m
kube-system    Active    20m
onap   local http://127.0.0.1:8879

Download OOM Gerrit repository and build the helm charts. 

Code Block
languagebash
themeRDark
git clone -b beijing http://gerrit.onap.org/r/oom
cd oom/kubernetes
make all

Confirm that ONAP charts have been successfully created.

Code Block
languagebash
themeRDark
helm search onap
NAME        Active    8s        enabledCHART VERSION   APP VERSION     DESCRIPTION

Start a local helm repository server and add it to helm repository list:

Code Block
languagebash
themeRDark
helm serve &
helm repo add local http://127.0.0.1:8879

Download OOM Gerrit repository and build the helm charts. 

Code Block
languagebash
themeRDark
git clone -b beijing http://gerrit.onap.org/r/oom
cd oom/kubernetes
make all

Confirm that ONAP charts have been successfully created.

Code Block
languagebash
themeRDark
helm search onap
NAME  local/onap              2.0.0           beijing         Open Network Automation Platform (ONAP)
local/aaf               2.0.0                  CHART VERSION   APP VERSION    ONAP DESCRIPTIONApplication Authorization Framework
local/onapaai               2.0.0           beijing            Open Network Automation Platform (ONAP)ONAP Active and Available Inventory
local/aafclamp               2.0.0                           ONAP ApplicationClamp
Authorization Framework
local/aaicli               2.0.0                           ONAP ActiveCommand andLine AvailableInterface
Inventory
local/clampconsul             2.0.0                           ONAP Clamp
local/cli  ONAP Consul Agent
local/dcaegen2          2.0.0                           ONAP CommandDCAE LineGen2
Interface
local/consuldmaap             2.0.0                           ONAP ConsulDMaaP Agentcomponents
local/dcaegen2esr               2.0.0                           ONAP DCAEExternal System Gen2Register
local/dmaaplog               2.0.0                           ONAP DMaaPLogging componentsElasticStack
local/esrmsb               2.0.0                           ONAP ExternalMicroServices SystemBus
Register
local/log       multicloud        2.0.0                           ONAP Loggingmulticloud ElasticStackbroker
local/msbnbi               2.0.0                           ONAP Northbound Interface
local/oof    ONAP MicroServices Bus local/multicloud        2.0.0                           ONAP multicloudOptimization brokerFramework
local/nbi   policy            2.0.0                           ONAP Policy NorthboundAdministration InterfacePoint
local/oof   portal            2.0.0                           ONAP OptimizationWeb FrameworkPortal
local/policypostgres            2.0.0                           ONAP PolicyPostgres AdministrationServer
Point
local/portalrobot             2.0.0                           A helm Chart for kubernetes-ONAP Web PortalRobot
local/postgressdnc-prom          2.0.0                           ONAP PostgresSDNC ServerPolicy local/robot         Driven Ownership Management
local/sniro-emulator    2.0.0                           ONAP Mock Sniro Emulator
local/so A helm Chart for kubernetes-ONAP Robot local/sdnc-prom         2.0.0                           ONAP SDNC Policy Driven Ownership Management
local/sniro-emulatorService Orchestrator
local/uui               2.0.0                           ONAP Mock Sniro Emulatoruui
local/so vfc               2.0.0                           ONAP Virtual Service OrchestratorFunction Controller (VF-C)
local/uuivid               2.0.0                           ONAP Virtual uuiInfrastructure Deployment
local/vfc   vnfsdk            2.0.0                           ONAP Virtual Function Controller (VF-C)
local/vid               2.0.0                           ONAP Virtual Infrastructure Deployment
local/vnfsdk            2.0.0                           ONAP VNF SDK

Install local/onap chart.  Local/onap chart will do some initialization setup which is needed for onap components, such as creating service accounts. 

Code Block
languagebash
themeRDark
cd oom/kubernetes
helm install local/onap -n common --namespace onap -f onap/resources/environments/disable-allcharts.yaml

In Casablanca, MSB project is working with VF-C and MultiCloud to verify Istio integration, so we are focusing on these three projects right now. More projects will engage later.

Code Block
languagebash
themeRDark
helm install local/msb -n msb --namespace onap
helm install local/vfc -n vfc --namespace onap
helm install local/multicloud -n multicloud --namespace onap

Confirm that ONAP microservices have been started

Code Block
languagebash
themeRDark
kubectl get all -n onap
NAME READY STATUS RESTARTS AGE
pod/msb-kube2msb-77ccb675dd-rhfn7 1/1 Running 0 3h
pod/msb-msb-consul-646987f5cf-qms5v 2/2 Running 0 3h
pod/msb-msb-discovery-7647f6476f-cl6xw 3/3 Running 0 3h
pod/msb-msb-eag-d678c65d6-fmfn6 3/3 Running 0 3h
pod/msb-msb-iag-647d5f998c-dc766 3/3 Running 0 3h
pod/multicloud-multicloud-5679bd9876-tzxzw 2/2 Running 0 1h
pod/multicloud-multicloud-ocata-774579596-f7smf 3/3 Running 0 1h
pod/multicloud-multicloud-vio-8c7dbc8d5-lfcw6 3/3 Running 0 1h
pod/multicloud-multicloud-windriver-85b595675d-5vx45 3/3 Running 0 1h
pod/vfc-vfc-catalog-79764dfd8f-rkx6f 2/2 Running 1VNF SDK

Install local/onap chart.  Local/onap chart will do some initialization setup which is needed for onap components, such as creating service accounts. 

Code Block
languagebash
themeRDark
cd oom/kubernetes
helm install local/onap -n common --namespace onap -f onap/resources/environments/disable-allcharts.yaml

In Casablanca, MSB project is working with VF-C and MultiCloud to verify Istio integration, so we are focusing on these three projects right now. More projects will engage later.

Code Block
languagebash
themeRDark
helm install local/msb -n msb --namespace onap
helm install local/vfc -n vfc --namespace onap
helm install local/multicloud -n multicloud --namespace onap

Confirm that ONAP microservices have been started

Code Block
languagebash
themeRDark
kubectl get all -n onap
NAME READY STATUS RESTARTS AGE
pod/msb-kube2msb-77ccb675dd-rhfn7 1/1 Running 0 3h
pod/msb-msb-consul-646987f5cf-qms5v 2/2 Running 0 3h
pod/msb-msb-discovery-7647f6476f-cl6xw 3/3 Running 0 3h
pod/msb-msb-eag-d678c65d6-fmfn6 3/3 Running 0 3h
pod/msb-msb-iag-647d5f998c-dc766 3/3 Running 0 3h
pod/multicloud-multicloud-5679bd9876-tzxzw 2/2 Running 0 1h
pod/multicloud-multicloud-ocata-774579596-f7smf 3/3 Running 0 1h
pod/multicloud-multicloud-vio-8c7dbc8d5-lfcw6 3/3 Running 0 1h
pod/multicloud-multicloud-windriver-85b595675d-5vx45 3/3 Running 0 1h
pod/vfc-vfc-catalog-79764dfd8f-rkx6f 2/2 Running 1 2d
pod/vfc-vfc-ems-driver-75bc68b946-6r6r6 1/1 Running 1 2d
pod/vfc-vfc-generic-vnfm-driver-69bf778bfd-pscjn 2/2 Running 0 2d
pod/vfc-vfc-huawei-vnfm-driver-8574569f4c-8jwc4 2/2 Running 1 2d
pod/vfc-vfc-juju-vnfm-driver-6dfd876bb8-bh7dq 2/2 Running 0 2d
pod/vfc-vfc-multivim-proxy-58c7bd47dc-7qdtd 1/1 Running 0 2d
pod/vfc-vfc-nokia-v2vnfm-driver-7b77c469bd-krfrw 1/1 Running 0 2d
pod/vfc-vfc-nokia-vnfm-driver-98fbdb5b5-p9zqw 2/2 Running 0 2d
pod/vfc-vfc-emsnslcm-driver74956bb876-75bc68b946-6r6r6v9kbt 12/12 Running 10 2d
pod/vfc-vfc-generic-vnfm-driverresmgr-69bf778bfd57dc4c98b5-pscjndzp7f 2/2 Running 0 2d
pod/vfc-vfc-huaweivnflcm-vnfm-driver-8574569f4c-8jwc46f9dc7df44-hncf4 2/2 Running 1 2d
pod/vfc-vfc-jujuvnfmgr-vnfm-driver-6dfd876bb8-bh7dq5585c688c6-7qrnp 2/2 Running 0 2d
pod/vfc-vfc-multivimvnfres-proxy54bc985599-58c7bd47dc-7qdtd9zkqn 12/12 Running 0 2d
pod/vfc-vfc-nokiaworkflow-v2vnfm-driver-7b77c469bd-krfrw6db56f95b9-np8tg 1/1 Running 01 2d
pod/vfc-vfc-nokiaworkflow-vnfmengine-driver7fb49fd974-98fbdb5b5-p9zqwkcb8q 21/21 Running 01 2d
pod/vfc-vfc-nslcm-74956bb876-v9kbt 2/2zte-sdnc-driver-585d449797-87nhp 1/1 Running 0 2d
pod/vfc-vfc-zte-resmgrvnfm-driver-57dc4c98b559d4756fbc-dzp7frpn9v 2/2 Running 0 2d
pod/vfc-vfc-vnflcm-6f9dc7df44-hncf4 2/2 Running 1 2d
pod/vfc-vfc-vnfmgr-5585c688c6-7qrnp 2/2 Running 0 2d
pod/vfc-vfc-vnfres-54bc985599-9zkqn 2/2 Running 0 2d
pod/vfc-vfc-workflow-6db56f95b9-np8tg 1/1 Running 1 2d
pod/vfc-vfc-workflow-engine-7fb49fd974-kcb8q 1/1 Running 1 2d
pod/vfc-vfc-zte-sdnc-driver-585d449797-87nhp 1/1 Running 0 2d
pod/vfc-vfc-zte-vnfm-driver-59d4756fbc-rpn9v 2/2 Running 0 2d

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/msb-consul NodePort 10.96.255.198 <none> 8500:30285/TCP 3h
service/msb-discovery NodePort 10.105.163.81 <none> 10081:30281/TCP 3h
service/msb-eag NodePort 10.100.221.66 <none> 80:30282/TCP,443:30284/TCP 3h
service/msb-iag NodePort 10.96.179.117 <none> 80:30280/TCP,443:30283/TCP 3h
service/multicloud NodePort 10.102.72.237 <none> 9001:30291/TCP 1h
service/multicloud-ocata NodePort 10.99.131.129 <none> 9006:30293/TCP 1h
service/multicloud-vio NodePort 10.111.175.58 <none> 9004:30292/TCP 1h
service/multicloud-windriver NodePort 10.110.92.61 <none> 9005:30294/TCP 1h
service/vfc-catalog ClusterIP 10.99.98.115 <none> 8806

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/msb-consul NodePort 10.96.255.198 <none> 8500:30285/TCP 3h
service/msb-discovery NodePort 10.105.163.81 <none> 10081:30281/TCP 3h
service/msb-eag NodePort 10.100.221.66 <none> 80:30282/TCP,443:30284/TCP 3h
service/msb-iag NodePort 10.96.179.117 <none> 80:30280/TCP,443:30283/TCP 3h
service/multicloud NodePort 10.102.72.237 <none> 9001:30291/TCP 1h
service/multicloud-ocata NodePort 10.99.131.129 <none> 9006:30293/TCP 1h
service/multicloud-vio NodePort 10.111.175.58 <none> 9004:30292/TCP 1h
service/multicloud-windriver NodePort 10.110.92.61 <none> 9005:30294/TCP 1h
service/vfc-catalog ClusterIP 10.99.98.115 <none> 8806/TCP 2d
service/vfc-ems-driver ClusterIP 10.96.189.14 <none> 8206/TCP 2d
service/vfc-generic-vnfm-driver ClusterIP 10.109.48.184 <none> 8484/TCP 2d
service/vfc-huawei-vnfm-driver ClusterIP 10.104.208.38 <none> 8482/TCP,8483/TCP 2d
service/vfc-juju-vnfm-driver ClusterIP 10.96.182.14 <none> 8483/TCP 2d
service/vfc-multivim-proxy ClusterIP 10.107.106.216 <none> 8481/TCP 2d
service/vfc-nokia-v2vnfm-driver ClusterIP 10.107.12.32 <none> 8089/TCP 2d
service/vfc-nokia-vnfm-driver ClusterIP 10.102.179.150 <none> 8486/TCP 2d
service/vfc-nslcm ClusterIP 10.106.43.164 <none> 8403/TCP 2d
service/vfc-resmgr ClusterIP 10.98.174.184 <none> 8480/TCP 2d
service/vfc-ems-drivervnflcm ClusterIP 10.96108.189132.14123 <none> 82068801/TCP 2d
service/vfc-generic-vnfm-drivervnfmgr ClusterIP 10.109108.4859.184102 <none> 84848803/TCP 2d
service/vfc-huawei-vnfm-drivervnfres ClusterIP 10.104111.20885.38161 <none> 84828802/TCP,8483/TCP 2d
service/vfc-juju-vnfm-driverworkflow ClusterIP 10.9697.182184.14206 <none> 848310550/TCP 2d
service/vfc-multivimworkflow-proxyengine ClusterIP 10.107109.106175.21661 <none> 84818080/TCP 2d2
service/vfc-nokiazte-v2vnfmsdnc-driver ClusterIP 10.107103.1294.32142 <none> 80898411/TCP 2d
service/vfc-nokiazte-vnfm-driver ClusterIP 10.102108.179146.150237 <none> 8486/TCP 2d
service/vfc-nslcm ClusterIP 10.106.43.164 <none> 8403/TCP 2d
service/vfc-resmgr ClusterIP 10.98.174.184 <none> 8480/TCP 2d
service/vfc-vnflcm ClusterIP 10.108.132.123 <none> 8801/TCP 2d
service/vfc-vnfmgr ClusterIP 10.108.59.102 <none> 8803/TCP 2d
service/vfc-vnfres ClusterIP 10.111.85.161 <none> 8802/TCP 2d
service/vfc-workflow ClusterIP 10.97.184.206 <none> 10550/TCP 2d
service/vfc-workflow-engine ClusterIP 10.109.175.61 <none> 8080/TCP 2
service/vfc-zte-sdnc-driver ClusterIP 10.103.94.142 <none> 8411/TCP 2d
service/vfc-zte-vnfm-driver ClusterIP 10.108.146.237 <none> 8410/TCP 2d

You can open the msb portal http://Node_IP:30280/iui/microservices/default.html in the browser to see all the registered services.

...

 8410/TCP 2d

You can open the MSB portal http://Node_IP:30280/iui/microservices/default.html in the browser to see all the registered services.

Image Added

Explore Istio features

Distributed Tracing

First, let's generate some traffics in the application, access the following URLs with curl command or open them in the browser

http://node_ip:30280/api/multicloud/v0/swagger.json

http://node_ip:30280/api/multicloud-vio/v0/swagger.json

http://node_ip:30280/api/multicloud-ocata/v0/swagger.json

Then open your browser at http://tracing_node_ip:tracing_node_port/, you should see something similar to the following:

Image Added

Note

  • Tracing_node_port can be found by 'kubctl get svc -n istio-system'.
  • ONAP microservices need to propagate the appropriate HTTP headers so that when the proxies send span information, the spans can be correlated correctly into a single trace.

Service Graph

Istio provides a Servicegraph service which generates and visualizes graph representations of the services in the mesh.

Open your browser at http://node_ip:30088/dotviz or http://node_ip:30088/force/forcegraph.html, you should see the service graph:

 Image Added

node  [n??d]  ??X
????
n. ?????[?] ??
n. (Node)???(?)??
????
node: ??
lymph node: ???
Atrioventricular node: ???