Description: - Connect multiple microservices on multiple logical clouds deployed across multiple clusters
Diagram
NOTE - For this scenario, the default mesh wide policy must be set to "PERMISSIVE" on both the clusters. It will not work if the default Mesh Policy is "STRICT"
Important Info - cert-chain.pem is Envoy’s cert that needs to be presented to the other side. key.pem is Envoy’s private key paired with Envoy’s cert in cert-chain.pem. root-cert.pem is the root cert to verify the peer’s cert. In this example, we only have one Citadel in a cluster, so all Envoys have the same root-cert.pem.
Add Inbound service 01
POST - traffic intent for the inbound service (service hosted behind the cluster)
POST
URL: /v2/projects/{project-name}/composite-apps/blue-app/{version}/traffic-intent-set/inbound-intent/
POST BODY:
{
"metadata": {
"name": "<>" // unique name for each intent
"description": "connectivity intent for stateless micro-service to stateless micro-service communication"
"userdata1": <>,
"userdata2": <>
}
"spec": { // update the memory allocation for each field as per OpenAPI standards
"application": "<app1>",
"servicename": "httpbin" //actual name of the client service - {istioobject - serviceEntry of client's cluster}
"externName": ""
"protocol": "HTTP",
"headless": "false", // default is false. Option "True" will make sure all the instances of the headless service will have access to the client service
"mutualTLS": "MUTUAL", // default is simple. Option MUTUAL will enforce mtls {istioobject - destinationRule}
"port" : "80", // port on which service is exposed as through servicemesh, not the port it is actually running on
"serviceMesh": "istio", // get it from cluster record
"sidecar-proxy": "yes", // The features (mTLS, LB, Circuit breaking) are not available to services without istio-proxy. Only inbound routing is possible.
traffic-management-info : {
// Traffic configuration - Loadbalancing is applicable per service. The traffic to this service is distrbuted amongst the pods under it.
"loadbalancingType": "ConsistenHash", // "Simple" and "consistentHash" are the two modes - {istioobject - destinationRule}
"loadBalancerMode": "httpCookie" // Modes for consistentHash - "httpHeaderName", "httpCookie", "useSourceIP", "minimumRingSize", Modes for simple - "LEAST_CONN", "ROUND_ROBIN", "RANDOM", "PASSTHROUGH" // choices of the mode must be explicit - {istioobject - destinationRule}
"httpCookie": "user1" // Name of the cookie to maitain stick sessions - {istioobject - destinationRule}
// Circuit Breaking
"maxConnections": 10 //connection pool for tcp and http traffic - {istioobject - destinationRule}
"concurrenthttp2Requests": 1000 // concurent http2 requests which can be allowed - {istioobject - destinationRule}
"httpRequestPerConnection": 100 //number of http requests per connection. Valid only for http traffic - {istioobject - destinationRule}
"consecutiveErrors": 8 // Default is 5. Number of consecutive error before the host is removed from load balancing pool - {istioobject - destinationRule}
"baseEjectionTime" : 15 // Default is 5, time for which the host will be removed from load balancing pool when it returns error for no of times more than "consecutiveErrors" limit - {istioobject - destinationRule}
"intervalSweep": 5m, //time limit before the removed hosts are added back to the load balancing pool. - {istioobject - destinationRule}
}
// credentials for mTLS.
"Servicecertificate" : "" // Present actual certificate here.
"ServicePrivateKey" : "" // Present actual private key here.
"caCertificate" : "" // present the trusted certificate to verify the client connection, Required only when mtls mode is MUTUAL
// Access Control
namespaces: [] // Workloads from this namespaces can access the inbound service - {istioobject - authorizationPolicy}
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "inbound service created"
}
Add Clients to inbound service 01
POST - traffic intent to add clients for accessing a specific inbound service - NOTE - Clients will have the mTLS mode same as the inbound service
POST
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients
POST BODY:
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "connectivity intent add client communication"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec: {
"clientServiceName": "sleep", // Name of the client service
"headless": "false", // default is false. Option "True" will generate the required configs for all the instances of headless service
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Client created"
}
Add Security detail for a client of inbound service 01
WARNING - This task requires mutual TLS enabled because the following examples use principal and namespace in the policies
GET
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients/sleep/security/security-intent
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "Security intent"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec:{
serviceAccountAccess : {[ "cluster.local/ns/default/sa/sleep": ["GET": "/status"]} // {istioobject - authorizationPolicy, will be applied for the inbound service}
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Security Rule created"
}
Add another Security detail for a client of inbound service 01
WARNING - This task requires mutual TLS enabled because the following examples use principal and namespace in the policies
GET
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients/sleep/security/security-intent
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "Security intent"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec:{
serviceAccountAccess : {"cluster.local/ns/default/sa/sleep" : ["GET": "/headers"]} // {istioobject - authorizationPolicy, will be applied for the inbound service}
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Security Rule created"
}
Add Inbound service 02
POST
POST
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-intent-set/us-to-us-intents/
POST BODY:
{
"metadata": {
"name": "<httpbin>" // unique name for each intent
"description": "connectivity intent for stateless micro-service to stateless micro-service communication"
"userdata1": <>,
"userdata2": <>
}
"spec": {
"application": "<app1>",
"servicename": "productpage" //actual name of the client service
"protocol": "HTTP",
"headless": "false", // default is false. Option "True" will make sure all the instances of the headless service will have access to the client service
"mutualTLS": "MUTUAL", // default is simple. Option MUTUAL will enforce mtls
"port" : "80", // port on which service is exposed as through servicemesh, not the port it is actually running on
"serviceMesh": "istio", // get it from cluster record
"istio-proxy": "yes", // The features (mTLS, LB, Circuit breaking) are no avaialble to services without istio-proxy. Only inbound routing is possible.
// Traffic configuration - Loadbalancing is applicable per service. The traffic to this service is distrbuted amongst the pods under it.
"loadbalancingType": "ConsistenHash", // "Simple" and "consistentHash" are the two modes
"loadBalancerMode": "httpCookie" // Modes for consistentHash - "httpHeaderName", "httpCookie", "useSourceIP", "minimumRingSize", Modes for simple - "LEAST_CONN", "ROUND_ROBIN", "RANDOM", "PASSTHROUGH" // choices of the mode must be explicit
"httpCookie": "user2" // Name of the cookie to maitain stick sessions
// Circuit Breaking
"maxConnections": 10 //connection pool for tcp and http traffic
"concurrenthttp2Requests": 1000 // concurent http2 requests which can be allowed
"httpRequestPerConnection": 100 //number of http requests per connection. Valid only for http traffic
"consecutiveErrors": 8 // Default is 5. Number of consecutive error before the host is removed from load balancing pool
"baseEjectionTime" : 15 // Default is 5, time for which the host will be removed from load balancing pool when it returns error for no of times more than "consecutiveErrors" limit
"intervalSweep": 5m, //time limit before the removed hosts are added back to the load balancing pool.
// credentials for mTLS.
"Servicecertificate" : "" // Present actual certificate here.
"ServicePrivateKey" : "" // Present actual private key here.
"caCertificate": "" // Trusted caCertificates used to verify the client
// Access Control
namespaces: [] // Workloads from this namespaces can access the inbound service
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "inbound service created"
}
Add Clients to inbound service 02
Client 01
POST - traffic intent to add clients for accessing a specific inbound service
POST
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients
POST BODY:
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "connectivity intent add client communication"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec: {
"clientServiceName": "sleep", // Name of the client service
"headless": "false", // default is false. Option "True" will generate the required configs for all the instances of headless service
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Client created"
}
Add Security detail 01 for client 01
WARNING - This task requires mutual TLS enabled because the following examples use principal and namespace in the policies
GET
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients/sleep/security/security-intent
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "Security intent"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec:{
serviceAccountAccess : {"cluster.local/ns/default/sa/sleep": {"GET": "/static"}} // {istioobject - authorizationPolicy, will be applied for the inbound service}
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Security Rule created"
}
Add Security detail 02 for client 01
WARNING - This task requires mutual TLS enabled because the following examples use principal and namespace in the policies
GET
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients/sleep/security/security-intent
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "Security intent"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec:{
serviceAccountAccess : {"cluster.local/ns/default/sa/sleep" : {"GET": "/api/v1/products"}} // {istioobject - authorizationPolicy, will be applied for the inbound service}
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Security Rule created"
}
Client 02
POST - traffic intent to add clients for accessing a specific inbound service
POST
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients
POST BODY:
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "connectivity intent add client communication"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec: {
"clientServiceName": "bookinfo-user", // Name of the client service
"headless": "false", // default is false. Option "True" will generate the required configs for all the instances of headless service
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Client created"
}
Add Security details for client 02
WARNING - This task requires mutual TLS enabled because the following examples use principal and namespace in the policies
GET
URL: /v2/projects/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-group-intent/uservice-to-uservice-intent/clients/sleep/security/security-intent
{
"metadata": {
"name": "<name>" // unique name for each intent
"description": "Security intent"
"application": "<app1>",
"userdata1": <>,
"userdata2": <>
}
spec:{
serviceAccountAccess : {"cluster.local/ns/default/sa/sleep": {"GET": "/api/v1/products"}}, // {istioobject - authorizationPolicy, will be applied for the inbound service}
}
}
RETURN STATUS: 201
RETURN BODY:
{
"name": "<name>"
"Message": "Security Rule created"
}
Generate Istio object resources
| Name of the Cluster | Microservice | Istio Configuration | Comments | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Cluster01 |
| ||||||||||||||
| Cluster02 |
|
Cluster01 Resources
1. ServiceEntry - To enable sleep to access to httpbin (logicalcloud01)
ServiceEntry
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: service-entry-httpbin
namespace: <> // namespace where the client service are deployed
spec:
hosts:
- httpbin.<namespace_of_service>.logicalcloud02
# template for the remote service name - <servicename.namespace.global>
# Treat remote cluster services as part of the service mesh
# as all clusters in the service mesh share the same root of trust.
location: MESH_INTERNAL
ports:
- name: http1
number: 8000
protocol: http
resolution: DNS
addresses:
# the IP address to which httpbin.<namespace>.<logicalcloudname> will resolve to
# must be unique for each remote service, within a given cluster.
# This address need not be routable. Traffic for this IP will be captured
# by the sidecar and routed appropriately.
- 240.0.0.2
endpoints:
# This is the routable address of the istio ingress gateway in cluster02
# routed to this address.
- address: 172.25.55.50 // IP of the istio-ingress-gateway
ports:
http1: 15443 //Sni. Do not change this
2. DestinationRule for TLS - sleep (logicalcloud01)
DestinationRule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: sleep-dr
namespace: <namespace_of_sleep>
spec:
host: "sleep"
trafficPolicy:
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
3. DestinationRule for TLS, Loadbalancing and circuit breaking - productpage (logicalCloud02)
DestinationRule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: bookinfo-productpage-dr
namespace: <namespace_of_productpage>
spec:
host: "productpage"
trafficPolicy:
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
loadbalancer:
consistentHash:
httpCookie: "user2"
connectionPool:
tcp:
maxConnections: 10
http:
http2MaxRequests: 1000
maxRequestsPerConnection: 100
outlierDetection:
consecutiveErrors: 7
interval: 5m
baseEjectionTime: 15m
4. AuthorizationPolicy for bookinfo-productpage - (logicalCloud02)
AuthorizationPolicy
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: deny-all
namespace: <namespace_of_prodfuct-page>
spec:
selector:
matchLabels:
app: <name_used_for_productpage>
rules:
- from:
- source:
principals: ["cluster.global/ns/default/sa/sleep", "cluster.global/ns/default/sa/bookinfo-user" ]
to:
- operation:
methods: ["GET"]
paths: ["/static*"]
- operation:
methods: ["GET"]
paths: ["/api/v1/products"]
Cluster 02 Resources
1. ServiceEntry - To enable access to bookinfo-productpage - (logicalCloud01)
ServiceEntry
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: service-entry-bookinfo-productpage
namespace: namespace01
spec:
hosts:
- productpage.namespace01.logicalcloud01 // format is <svc>.<namespace>.<logical_cluster_domain>
# template for the remote service name - <servicename.namespace.global>
# Treat remote cluster services as part of the service mesh
# as all clusters in the service mesh share the same root of trust.
location: MESH_INTERNAL
ports:
- name: http1
number: 8000
protocol: http
resolution: DNS
addresses:
# the IP address to which httpbin.<namespace>.<logicalcloudname> will resolve to
# must be unique for each remote service, within a given cluster.
# This address need not be routable. Traffic for this IP will be captured
# by the sidecar and routed appropriately.
- 240.0.0.3
endpoints:
# This is the routable address of the istio ingress gateway in cluster02
# routed to this address.
- address: 172.25.55.210
ports:
http1: 15443 //Sni. Do not change this
2. DestinationRule for TLS - sleep - (logicalCloud01)
DestinationRule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: sleep-dr
namespace: namespace01
spec:
host: "sleep"
trafficPolicy:
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
3. DestinationRule for bookinfo-user - (logicalCloud01)
DestinationRule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: sleep-dr
namespace: namespace01
spec:
host: "bookinfo-user"
trafficPolicy:
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
4. DestinationRule for simple TLS, Loadbalancing and circuit breaking for httpbin - (logicalCloud02)
DestinationRule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin-dr
namespace: namespace02
spec:
host: "httpbin"
trafficPolicy:
tls:
mode: MUTUAL
serverCertificate: /etc/certs/cert-chain.pem
privateKey: /etc/certs/key.pem
caCertificates: /etc/certs/root-cert.pem
loadbalancer:
consistentHash:
httpCookie: "user1"
connectionPool:
tcp:
maxConnections: 10
http:
http2MaxRequests: 1000
maxRequestsPerConnection: 100
outlierDetection:
consecutiveErrors: 7
interval: 5m
baseEjectionTime: 15m
5. AuthorizationPolicy for httpbin - (logicalCloud02)
AuthorizationPolicy
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: deny-all
namespace: namespace02
spec:
selector:
matchLabels:
app: <app_Name_of_httpbin>
rules:
- from:
- source:
principals: ["cluster.local/ns/default/sa/sleep"]
to:
- operation:
methods: ["GET"]
paths: ["/status*"]
- operation:
methods: ["GET"]
paths: ["/headers"]