External DNS provider update design and intent API

UPDATES PENDING

Following is in progress - stay tuned for updates ...

This section covers the design for how external DNS records are updated.

Create DNS Provider Intent

When instantiating a distributed app with user facing microservices, a DNS Provider Intent is included as part of the Traffic Control Intent Set to automate the update of DNS servers with DNS records for the user facing services. Each element of the DNS Provider Intent points to a collection of DNS Provider Information that provides the details for the specific DNS server that is to be updated. DNS Provider Information can be defined per cluster or per logical cloud for a given project.

Each instance of a DNS Provider Information record has a unique name within the Project. The name was be used as a 'key' and the associated 'value' will be a JSON object which contains all of the parameters needed to invoke 'external-dns' for that specific provider. This key value pair is then added to the key value pair list of the logical cloud or the appropriate cluster. More than one DNS Provider Information record can be associated with a logical cloud or a cluster.

The following example illustrates the DNS PRovider Key Value pair. The idea is that the 'externalDnsParameters' portion will be used to create the parameters list for the 'external-dns' deployment that will be created to handle this DNS Provider.

DNS Provider Information Key Value Pair

"<dnsprovidername>": {
  "description": "description for DNS Provider",
  "externalDnsParameters": {   // list will be supplied to external-dns as parameters.
    "aws-zone-type": "",          When using the AWS provider, filter for zones of this type (optional, options: public, private)
    "aws-zone-tags": "",          When using the AWS provider, filter for zones with these tags
    "aws-assume-role":"",         When using the AWS provider, assume this IAM role. Useful for hosted zones in another AWS account. Specify the full ARN, e.g. `arn:aws:iam::123455567:role/external-dns` (optional)
    "aws-batch-change-size":"1000",  When using the AWS provider, set the maximum number of changes that will be applied in each batch.
    "aws-batch-change-interval":"1s", When using the AWS provider, set the interval between batch changes.
    "aws-evaluate-target-health":"enabled", When using the AWS provider, set whether to evaluate the health of a DNS target (default: enabled, disable with --no-aws-evaluate-target-health)
    "aws-api-retries":"3",           When using the AWS provider, set the maximum number of retries for API calls before giving up.
    "aws-prefer-cname":"disabled"           When using the AWS provider, prefer using CNAME instead of ALIAS (default: disabled)
     etc.
  }
}

Alternative approach (?) - DNS Providers becomes an explicit resource (e.g. v2/project/{project-name}/dns-providers/{dns-provider}) which are associated to logical-clouds and clusters via labels.

The following illustrates the API for creating a DNS Provider intent. To associate multiple DNS Providers with a DNS Provider Intent, multiple calls are made.

POST

URL: /v2/project/{project-name}/rb/{rb-name}/{rb-version}/traffic-intent-sets/{traffic-intent-set-name}/dnsproviders
POST BODY:
{
  "name": "<dnsprovidername>"   // values used as a key to find the DNS Provider Information details from the logical-cloud or cluster KV pair list.
}

GET

URL: /v2/project/{project-name}/rb/{rb-name}/{rb-version}/traffic-intent-sets/{traffic-intent-set-name}/dnsproviders
GET RESPONSE BODY:
[
  {
    "name": "dnsprovidername1"
  },
  {
    "name": "dnsprovidername2"
  },
  {
    "name": "dnsprovidername3"
  },
  {
    "name": "dnsprovidername4"
  }
]

URL: /v2/project/{project-name}/rb/{rb-name}/{rb-version}/traffic-intent-sets/{traffic-intent-set-name}/dnsproviders/dnsprovidername2
GET RESPONSE BODY:
{
  "name": "dnsprovidername2"
}

DNS Provider Intent Handling

After DNS Provider KV pairs are created and associated with logical cloud and/or clusters and these DNS Providers are added to a traffic control intent set, the traffic intent set will be associated with an intent group, which in turn will be provided as a parameter when a call to prepare for the instantiation of a distribued application is made. Eventually, the multicloud orchestrator will invoke the traffic controller to process the traffic controller intents. One part of that process will be to handle the DNS Provider Intents.

The following sequence diagram illustrates what happens:

The DNS Provider Intent handling consists of two key tasks:

Prepare manifests for external-dns Deployments which will handle the updating of specific DNS-Providers. There is a separate external-dns Deployment for each DNS Provider (based on current understanding of how external-dns works).
1. For a given project, there should only need to be one external-dns Deployment to handle all distributed Apps that are deployed - so, maybe the App Context is not the right location for these Deployment manifests.
2. The 'source' for external-dns will always be a DNSendpointCRD for a namespace in the project. An enhancement to external-dns is expected necessary to handle filtering the DNSendpointCRD based on a label to associate specific CRs with a DNS Provider.
Prepare DNSendpointCRD manifests for user facing services present in the App Context.

The following show an example of a DNS Endpoint CR instance.

DNS Endpoint CRD

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: microservice1-dnsrecords
  namespace: projectnamespace1
  labels:
    dnsprovider: dnsprovidername2
spec:
  endpoints:
  - dnsName: abc.123.com
    recordTTL: 180
    recordType: A
    targets:
    - 1.2.3.4
    - 5.6.7.8

To associate these with specific DNS Providers, the proposal is to add a label which will allow external-dns (after modification) to handle only DNS Endpont CRs that match the label (in addition to the namespace) for a given DNS Provider.

The 'name' of the DNS Endpoint CR can be derived from the sub-app (microservice) that is exposing these endponts.

The DNS Provider Intent handler creates the manifests for these DNS Endpoint CRs using the following algorithm:

DNS Endpoint CRD generation algorithm

for each DNS provider intent
    find the KV pair for this intent in either the logical cloud or a cluster
    prepare a list of clusters that apply for this intent
    for each cluster
        for each sub-app with an external FQDN defined (e.g. in an ISTIO gateway)
            acquire the IP(s) associated with cluster and scope of the DNS Provider
            create DNSendpointCRD(s) manifests including a label with the DNSproviderName

Notes:

There will need to be a way to find the appropriate set of IPs to use. Some IPs will be appropriate for a public scope (e.g. update a DNS Provider associated with the logical cloud) and others may be local to the cluster network (e.g. cluster DNS Providers).

Resource Synchronization

<tbd>