This section covers the design for how external DNS records are updated.
Create DNS Provider Intent
When instantiating a distributed app with user facing microservices, a DNS Provider Intent is included as part of the Traffic Control Intent Set to automate the update of DNS servers with DNS records for the user facing services. Each element of the DNS Provider Intent points to a collection of DNS Provider Information that provides the details for the specific DNS server that is to be updated. DNS Provider Information can be defined per cluster or per logical cloud for a given project.
Each instance of a DNS Provider Information record has a unique name within the Project and is created and managed by the Traffic Controller API. The record will contain all of the parameters needed to invoke 'external-dns' for that specific provider. The unique (within a project) DNS Provider 'name' will be used to construct a key that is added to either the logical cloud or cluster(s) KV list to associate the DNS Provider Info with the cloud/cluster. More than one DNS Provider Information record can be associated with a logical cloud or a cluster.
TBD: format of key - such as: (composed of 'project', 'dns provider name'), what would the value be ? (perhaps some IP address label?)
The following example illustrates the DNS PRovider Key Value pair. The idea is that the 'externalDnsParameters' portion will be used to create the parameters list for the 'external-dns' deployment that will be created to handle this DNS Provider.
URL: /v2/project/{project-name}/traffic-controller/dnsproviders
POST BODY:
{
"name": "<dnsprovidername>",
"description": "description for DNS Provider",
"externalDnsParameters": { // list will be supplied to external-dns as parameters.
"aws-zone-type": "", When using the AWS provider, filter for zones of this type (optional, options: public, private)
"aws-zone-tags": "", When using the AWS provider, filter for zones with these tags
"aws-assume-role":"", When using the AWS provider, assume this IAM role. Useful for hosted zones in another AWS account. Specify the full ARN, e.g. `arn:aws:iam::123455567:role/external-dns` (optional)
"aws-batch-change-size":"1000", When using the AWS provider, set the maximum number of changes that will be applied in each batch.
"aws-batch-change-interval":"1s", When using the AWS provider, set the interval between batch changes.
"aws-evaluate-target-health":"enabled", When using the AWS provider, set whether to evaluate the health of a DNS target (default: enabled, disable with --no-aws-evaluate-target-health)
"aws-api-retries":"3", When using the AWS provider, set the maximum number of retries for API calls before giving up.
"aws-prefer-cname":"disabled" When using the AWS provider, prefer using CNAME instead of ALIAS (default: disabled)
etc.
}
}
Alternative approach (?) - DNS Providers becomes an explicit resource (e.g. v2/project/{project-name}/dns-providers/{dns-provider}) which are associated to logical-clouds and clusters via labels.
The following illustrates the API for creating a DNS Provider intent. To associate multiple DNS Providers with a DNS Provider Intent, multiple calls are made.
URL: /v2/project/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-intent-sets/{traffic-intent-set-name}/dnsproviders
POST BODY:
{
"name": "<dnsprovidername>" // values used as a key to find the DNS Provider Information details from the logical-cloud or cluster KV pair list.
}
URL: /v2/project/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-intent-sets/{traffic-intent-set-name}/dnsproviders
GET RESPONSE BODY:
[
{
"name": "dnsprovidername1"
},
{
"name": "dnsprovidername2"
},
{
"name": "dnsprovidername3"
},
{
"name": "dnsprovidername4"
}
]
URL: /v2/project/{project-name}/composite-apps/{composite-app-name}/{version}/traffic-intent-sets/{traffic-intent-set-name}/dnsproviders/dnsprovidername2
GET RESPONSE BODY:
{
"name": "dnsprovidername2"
}
DNS Provider Intent Handling
After DNS Provider KV pairs are created and associated with logical cloud and/or clusters and these DNS Providers are added to a traffic control intent set, the traffic intent set will be associated with an intent group, which in turn will be provided as a parameter when a call to prepare for the instantiation of a distribued application is made. Eventually, the multicloud orchestrator will invoke the traffic controller to process the traffic controller intents. One part of that process will be to handle the DNS Provider Intents.
The following sequence diagram illustrates what happens:
The DNS Provider Intent handling consists of two key tasks:
- Prepare manifests for external-dns Deployments which will handle the updating of specific DNS-Providers. There is a separate external-dns Deployment for each DNS Provider (based on current understanding of how external-dns works).
- For a given project, there should only need to be one external-dns Deployment to handle all distributed Apps that are deployed - so, maybe the App Context is not the right location for these Deployment manifests.
- The 'source' for external-dns will always be a DNSendpointCRD for a namespace in the project. An enhancement to external-dns is expected necessary to handle filtering the DNSendpointCRD based on a label to associate specific CRs with a DNS Provider.
- Prepare DNSendpointCRD manifests for user facing services present in the App Context.
The following show an example of a DNS Endpoint CR instance.
apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
name: microservice1-dnsrecords
namespace: projectnamespace1
labels:
dnsprovider: dnsprovidername2
spec:
endpoints:
- dnsName: abc.123.com
recordTTL: 180
recordType: A
targets:
- 1.2.3.4
- 5.6.7.8
see also: https://github.com/kubernetes-sigs/external-dns/tree/master/docs/contributing/crd-source
To associate these with specific DNS Providers, the proposal is to add a label which will allow external-dns (after modification) to handle only DNS Endpont CRs that match the label (in addition to the namespace) for a given DNS Provider.
The 'name' of the DNS Endpoint CR can be derived from the sub-app (microservice) that is exposing these endponts.
The DNS Provider Intent handler creates the manifests for these DNS Endpoint CRs using the following algorithm:
for each DNS provider intent
find the KV pair for this intent in either the logical cloud or a cluster
prepare a list of clusters that apply for this intent
for each cluster
for each sub-app with an external FQDN defined (e.g. in an ISTIO gateway)
acquire the IP(s) associated with cluster and scope of the DNS Provider
create DNSendpointCRD(s) manifests including a label with the DNSproviderName
Notes:
- There will need to be a way to find the appropriate set of IPs to use. Some IPs will be appropriate for a public scope (e.g. update a DNS Provider associated with the logical cloud) and others may be local to the cluster network (e.g. cluster DNS Providers).
Resource Synchronization
The following illustrates how the external DNS updates are triggered once the Resource Synchronization of the distributed app is performed.
The external-dns deployment for a given DNS Provider may be utilized for more than one distributed application, so some method for detecting that there is already a deployment running is needed.
Proposal: annotate the deployment with something like "resource bundle/resource bundle version/profile" when the external-dns deployment is created. Additional apps will use the same deployment and annotate the deployment. If a distributed app is removed, then the annotation for that app is removed.
Note: The Resource Sync of the App Context is applying these external DNS related resources to the central controller cluster instead of the edge or other clusters where most of the distributed app resources will be applied.