...
When instantiating a distributed app with user facing microservices, a DNS Provider Intent is included as part of the Traffic Control Intent Set includes information to automate indicate the update set of external DNS servers which should be updated with appropriate DNS records for the user facing services. Each element of the DNS Provider Intent points to a collection of DNS Provider Information that provides the details for the specific DNS server that is to be updated. DNS Provider Information can be defined per cluster or per logical cloud for a given project. At the Project level, DNS Update Records are supplied to identify the type of DNS Provider to be updated along with a set of parameters required to perform the DNS record update. The 'external-dns' package is used to perform DNS update to a selected set of DNS providers. After the DNS Update Records have been created, they can be associated with specific logical clouds or clusters utilizing the Key Value pair feature of those APIs.
Each instance of a DNS Provider Information record Update Record has a unique name within the Project and is created and managed by the Traffic Controller API. The record will contain all of the parameters needed to invoke 'external-dns' for that specific provider. The unique (within a project) DNS Provider 'name' will be used to construct a key that is added to either the logical cloud or cluster(s) KV list to associate the DNS Provider Info with the cloud/cluster. More than one DNS Provider Information record can be associated with a logical cloud or a cluster.
TBD: format of key - such as: (composed of 'project', 'dns provider name'), what would the value be ? (perhaps some IP address label?)
The following example illustrates the DNS PRovider Key Value pair. The idea is that the 'externalDnsParameters' portion will be used to create the parameters list for the 'external-dns' deployment that will be created to handle this DNS Provider.
...
| theme | Midnight |
|---|---|
| title | DNS Provider Information Key Value Pair |
...
The following example illustrates the DNS Update Record API.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
URL: /v2/project/{project-name}/dns-update-records/
POST BODY:
{
"metadata": {
"name": "name_of_dns_information_record",
"description": "description of the DNS information",
"userdata1": "some user data",
"userdata2": "some different user data"
},
"spec": {
"provider": "coredns",
"external-dns-parameters": {
"oneOf": [ ## schema-like (array structure not in body) - one of the following is selected to match the 'provider' - e.g. 'coredns' in this case
{
"aws-zone-type": "public",
"aws-zone-tags": "zone tags",
"aws-assume-role": "arn:aws:iam::123455567:role/external-dns",
"aws-batch-change-size": 1000,
"aws-batch-change-interval": "1s",
"aws-evaluate-target-health": true,
"no-aws-evaluate-target-health": true,
"aws-api-retries": 3,
"aws-prefer-cname": true
},
{
"azure-config-file": "/etc/kubernetes/azure.json",
"azure-resource-group": "resource group",
"azure-subscription-id": "subscription id",
"azure-user-assigned-identity-client-id": "client id"
},
{
"coredns-prefix": "skydns"
},
{
"rfc2136-host": "host.sample.com",
"rfc2136-zone": "",
"rfc2136-insecure": false,
"rfc2136-tsig-keyname": "tsig key",
"rfc2136-tsig-secret": "tsig secret",
"rfc2136-tsig-secret-alg": "tsig secret alg",
"rfc2136-tsig-axfr": "axfr",
"request-timeout": "30s"
}
],
"contour-load-balancer": "heptio-contour/contour", ## from here on, possible parameters used by external-dns - not all required
"fqdn-template": "template",
"combine-fqdn-annotation": true,
"ignore-hostname-annotation": true,
"compatibility": "mate",
"publish-internal-services": true,
"publish-host-ip": true,
"service-type-filter": "all",
"domain-filter": "example.com",
"exclude-domains": "example.com",
"zone-id-filter": "zone filter",
"tls-ca": "tls ca path",
"tls-client-cert": "tls client cert path",
"tls-client-cert-key": "tls client cert key path",
"policy": "sync",
"registry": "txt",
"txt-owner-id": "default",
"txt-prefix": "custom string",
"txt-cache-interval": "30s",
"interval": "30s",
"once": true,
"dry-run": true,
"log-format": "text",
"metrics-address": ":7979",
"log-level": "info"
}
}
}
|
The following shows an OpenAPI component definition of the above DNS Update Record structure (note: in progress). Parameters with descriptions are shown in more detail.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
openapi: 3.0.0
info:
title: DNS update information record
description: Defines information needed to update external DNS
version: 2.0.0
paths:
<...tbd...>
components:
schemas:
dnsrecord:
properties:
metadata:
$ref: '#/components/schemas/metadata'
spec:
$ref: '#/components/schemas/extdns_info'
extdns_info:
type: "object"
properties:
provider:
type: "string"
enum: ["aws", "aws-sd", "azure", "azure-dns", "azure-private-dns", "coredns", "rfc2136"]
example: "coredns"
external-dns-parameters:
allOf:
- $ref: '#/components/schemas/extdns_provider'
- $ref: '#/components/schemas/extdns_general'
metadata:
type: "object"
required:
- name
properties:
name:
type: "string"
example: "name_of_dns_information_record"
description:
type: "string"
example: "description of the DNS information"
userdata1:
type: "string"
example: "some user data"
userdata2:
type: "string"
example: "some different user data"
extdns_general:
type: "object"
required:
- provider
properties:
request-timeout:
type: "string"
example: "30s"
description: "Request timeout when calling Kubernetes APIs. 0s means no timeout"
contour-load-balancer:
type: "string"
example: "heptio-contour/contour"
default: "heptio-contour/contour"
description: "The fully-qualified name of the Contour load balancer service."
fqdn-template:
type: "string"
example: "template"
description: "A templated string that's used to generate DNS names from sources that don't define a hostname themselves, or to add a hostname suffix when paired with the fake source (optional). Accepts comma separated list for multiple global FQDN."
combine-fqdn-annotation:
type: "boolean"
example: true
description: "Combine FQDN template and Annotations instead of overwriting"
ignore-hostname-annotation:
type: "boolean"
example: true
default: false
description: "Ignore hostname annotation when generating DNS names, valid only when using fqdn-template is set (optional, default: false)"
compatibility:
type: "string"
example: "mate"
enum: ["mate", "molecule"]
description: "Process annotation semantics from legacy implementations (optional, options: mate, molecule)"
publish-internal-services:
type: "boolean"
example: true
description: "Allow external-dns to publish DNS records for ClusterIP services (optional)"
publish-host-ip:
type: "string"
example: true
description: "Allow external-dns to publish host-ip for headless services (optional)"
service-type-filter:
type: "string"
example: "all"
default: "all"
enum: ["all", "ClusterIP", "NodePort", "LoadBalancer", "ExternalName"]
description: "The service types to take care about (default: all, expected: ClusterIP, NodePort, LoadBalancer or ExternalName)"
domain-filter:
type: "string"
example: "example.com"
description: "Limit possible target zones by a domain suffix; specify multiple times for multiple domains (optional)"
exclude-domains:
type: "string"
example: "example.com"
description: "Exclude subdomains (optional)"
zone-id-filter:
type: "string"
example: "zone filter"
description: "Filter target zones by hosted zone id; specify multiple times for multiple zones (optional)"
tls-ca:
type: "string"
example: "tls ca path"
description: "When using TLS communication, the path to the certificate authority to verify server communications (optionally specify --tls-client-cert for two-way TLS)"
tls-client-cert:
type: "string"
example: "tls client cert path"
description: "When using TLS communication, the path to the certificate to present as a client (not required for TLS)"
tls-client-cert-key:
type: "string"
example: "tls client cert key path"
description: "When using TLS communication, the path to the certificate key to use with the client certificate (not required for TLS)"
policy:
type: "string"
example: "sync"
default: "sync"
enum: ["sync", "upsert-only", "create-only"]
description: "Modify how DNS records are synchronized between sources and providers (default: sync, options: sync, upsert-only, create-only)"
registry:
type: "string"
example: "txt"
default: "txt"
enum: ["txt", "noop", "aws-sd"]
description: "The registry implementation to use to keep track of DNS record ownership (default: txt, options: txt, noop, aws-sd)"
txt-owner-id:
type: "string"
example: "default"
default: "default"
description: "When using the TXT registry, a name that identifies this instance of ExternalDNS (default: default)"
txt-prefix:
type: "string"
example: "custom string"
description: "When using the TXT registry, a custom string that's prefixed to each ownership DNS record (optional)"
txt-cache-interval:
type: "string"
example: "30s"
description: "The interval between cache synchronizations in duration format (default: disabled)"
interval:
type: "string"
example: "30s"
default: "1m"
description: "The interval between two consecutive synchronizations in duration format (default: 1m)"
once:
type: "boolean"
example: true
description: "When enabled, exits the synchronization loop after the first iteration (default: disabled)"
dry-run:
type: "boolean"
example: true
description: "When enabled, prints DNS record changes rather than actually performing them (default: disabled)"
log-format:
type: "string"
example: "text"
default: "text"
enum: ["text", "json"]
description: "The format in which log messages are printed (default: text, options: text, json)"
metrics-address:
type: "string"
example: ":7979"
description: "Specify where to serve the metrics and health check endpoint (default: :7979)"
log-level:
type: "string"
example: "info"
default: "info"
enum: ["panic", "debug", "info", "warning", "error", "fatal"]
description: "Set the level of logging. (default: info, options: panic, debug, info, warning, error, fatal"
extdns_provider:
oneOf:
- $ref: '#/components/schemas/extdns_provider_aws'
- $ref: '#/components/schemas/extdns_provider_azure'
- $ref: '#/components/schemas/extdns_provider_coredns'
- $ref: '#/components/schemas/extdns_provider_rfc2136'
extdns_provider_aws:
type: "object"
properties:
aws-zone-type:
type: "string"
example: "public"
enum: ["public", "private"]
description: "When using the AWS provider, filter for zones of this type (optional, options: public, private)"
aws-zone-tags:
type: "string"
example: "zone tags"
description: "When using the AWS provider, filter for zones with these tags"
aws-assume-role:
type: "string"
example: "arn:aws:iam::123455567:role/external-dns"
description: "When using the AWS provider, assume this IAM role. Useful for hosted zones in another AWS account. Specify the full ARN, e.g. `arn:aws:iam::123455567:role/external-dns` (optional)"
aws-batch-change-size:
type: "integer"
example: 1000
description: "When using the AWS provider, set the maximum number of changes that will be applied in each batch."
aws-batch-change-interval:
type: "string"
example: "1s"
description: "When using the AWS provider, set the interval between batch changes."
aws-evaluate-target-health:
type: "boolean"
example: true
description: "When using the AWS provider, set whether to evaluate the health of a DNS target (default: enabled, disable with --no-aws-evaluate-target-health)"
no-aws-evaluate-target-health:
type: "boolean"
example: true
description: "When using the AWS provider, set whether to evaluate the health of a DNS target (default: enabled, disable with --no-aws-evaluate-target-health)"
aws-api-retries:
type: "integer"
example: 3
description: "When using the AWS provider, set the maximum number of retries for API calls before giving up."
aws-prefer-cname:
type: "boolean"
example: true
description: "When using the AWS provider, prefer using CNAME instead of ALIAS (default: disabled)"
extdns_provider_azure:
type: "object"
properties:
azure-config-file:
type: "string"
example: "/etc/kubernetes/azure.json"
description: "When using the Azure provider, specify the Azure configuration file (required when --provider=azure"
azure-resource-group:
type: "string"
example: "resource group"
description: "When using the Azure provider, override the Azure resource group to use (required when --provider=azure-private-dns)"
azure-subscription-id:
type: "string"
example: "subscription id"
description: "When using the Azure provider, specify the Azure configuration file (required when -ws-zone-type-provider=azure-private-dns)"
azure-user-assigned-identity-client-id:
type: "string"
example: "client id"
description: "When using the Azure provider, override the client id of user assigned identity in config file (optional)"
extdns_provider_coredns:
type: "object"
properties:
coredns-prefix:
type: "string"
example: "skydns"
description: "When using the CoreDNS provider, specify the prefix name"
extdns_provider_rfc2136:
type: "object"
properties:
rfc2136-host:
type: "string"
example: "host.sample.com"
description: "When using the RFC2136 provider, specify the host of the DNS server"
rfc2136-zone:
type: "string"
example: ""
description: "When using the RFC2136 provider, specify the zone entry of the DNS server to use"
rfc2136-insecure:
type: "boolean"
example: false
description: "When using the RFC2136 provider, specify whether to attach TSIG or not (default: false, requires --rfc2136-tsig-keyname and --rfc2136-tsig-secret)"
rfc2136-tsig-keyname:
type: "string"
example: "tsig key"
description: "When using the RFC2136 provider, specify the TSIG key to attached to DNS messages (required when --rfc2136-insecure=false)"
rfc2136-tsig-secret:
type: "string"
example: "tsig secret"
description: "When using the RFC2136 provider, specify the TSIG (base64) value to attached to DNS messages (required when --rfc2136-insecure=false)"
rfc2136-tsig-secret-alg:
type: "string"
example: "tsig secret alg"
description: "When using the RFC2136 provider, specify the TSIG (base64) value to attached to DNS messages (required when --rfc2136-insecure=false)"
rfc2136-tsig-axfr:
type: "string"
example: "axfr"
description: "When using the RFC2136 provider, specify the TSIG (base64) value to attached to DNS messages (required when --rfc2136-insecure=false)"
|
The Key Value pair list that is added to the logical-clouds and clusters will look like the following:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
{ "metadata":{ "name":"dns-update-record-list", "description":"list of the DNS update records associated with this logical-cloud or cluster", "userData1":"<user data>", "userData2":"<user data>" }, "spec":{ "kv":[ { When using the AWS provider, set the maximum number of retries for API calls before giving up. "aws-prefer-cname":"disabled" "key1":"dns-update-record-1" }, { When using the AWS provider, prefer using CNAME instead of ALIAS (default: disabled)"key2":"dns-update-record-47" } ] etc. } } |
...
Traffic Control Intent Handling - DNS Update Records
After DNS Provider KV pairs Update Records are created and associated with logical cloud and/or clusters and these DNS Providers are added to a traffic control intent set, the traffic intent set will be associated with an intent group, which in turn will be provided as a parameter when a call to prepare for the instantiation of a distribued application is made/or clusters and associated with the Traffic Control intents of the distributed application, a call is made to instantiate a Profile. Eventually, the multicloud orchestrator will invoke the traffic controller to process the traffic controller intents. One part of that process will be to handle the DNS Provider IntentsUpdate Records associated with the Profile (via the intent).
The following sequence diagram illustrates what happens:
The DNS Provider Intent Update Record handling consists of two key tasks:
...
The 'name' of the DNS Endpoint CR can be derived from the sub-app (microservice) that is exposing these endpontsendpoints.
The DNS Provider Intent Update Record handler creates the manifests for these DNS Endpoint CRs using the following algorithm:
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
for each DNS providerUpdate intentRecord find the KV pair for this intent in either the logical cloud or a cluster prepare a list of clusters that apply for this intent for each cluster for each sub-app with an external FQDN defined (e.g. in an ISTIO gateway) acquire the IP(s) associated with cluster and scope of the DNS Provider create DNSendpointCRD(s) manifests including a label with the DNSproviderName |
...
The following illustrates how the external DNS updates are triggered once the Resource Synchronization of the distributed app is performed.
The external-dns deployment for a given DNS Provider may be utilized for more than one distributed application, so some method for detecting that there is already a deployment running is needed.
...