Page Comparison

...

1. A Central-deployed DMaaP component muse be able to route to an Edge-deployed component, and distinguish between the same component deployed at different Edge sites.  Examples include:
   1. dr-prov periodically sends provisioning info to each dr-node
   2. A centrally-deployed dr-node may transfer a file to an Edge-deployed dr-node for delivery to a subscriber in that Edge, based on an egress rule
   3. A central mirrormaker subscribes to an Edge-deployed message-router kafka
2. An Edge-deployed DMaaP component must be able to route to a central-deployed service.  Examples include:
   1. dr-node periodically syncs with dr-prov
   2. dr-node authenticates publish requests using aaf
   3. message-router authenticates client requests using aaf
   4. dbc-client makes request to dmaap-bc API during post-install provisioning
   5. Edge mirrormaker subscribes to central message-router kafka
3. Localized DR Routing between a Data File Collector (DFC) and a PM Mapper deployed in the same Edge X.
   1. Localized DR Routing means DR Node is deployed in the same Edge site so data doesn't need to leave the site.
   2. DFC will be a publisher to a feed provisioned at deployment time.
   3. PM Mapper will be a subscriber provisioned at deployment time.
   4. The feed should be unique per site so that when there are multiple sites, PM Mapper only receives its locally produced data.
4. Localized messaging from PM Mapper and DFC.  This will signal DFC that a file was processed.
   1. Localized messaging implies a Message Router instance in the same edge location.
   2. PM Mapper will a publisher provisioned at deployment time 
   3. DFC will be a subscriber provisioned at deployment time.
   4. Communication will utilize an authenticated topic in the MR deployed in the same edge site. 
      1. PM Mapper and DFC will use AAF credentials to authenticate.
      2. PM Mapper identity will be authorized to publish on the topic
      3. DFC identity will be authorized to subscribe on the topic
5. Inter-site messaging from PM Mapper to VES perf3gpp
   1. Inter-site messaging means sending a message from an edge location publisher to a central location subscriber.
   2. PM Mapper, deployed at Edge,  will be a publisher using AAF credentials
   3. VES perf3gpp, deployed in Central, will be a subscriber using AAF credentials
   4. Communication will utilize an authenticated topic on the MR deployed in the same edge site.
      1. PM Mapper and DFC will use AAF credentials to authenticate.
      2. PM Mapper identity will be authorized to publish on the topic
      3. VES perf3gpp identity will be authorized to subscribe on the topic
   5. Furthermore, messages on this topic will be replicated to the central MR instance.
   6. Are there any other subscribers?  (esp, are there any other at edge?)

Solution Options for Dublin

This section is based on a discussion with Jack Lucas about possible approaches that we might consider within the Dublin feature set.

Ways to route to a k8s service in another k8s cluster:

1. Extend the configuration of the Jack's proxy to include DMaaP services.  Note: Current capability will route from edge to central. (See Jack's demo)
   1. Include central deployed DMaaP services with existing node ports in proxy config: dr-prov, message-router, dmaap-bc
   2. Expose central deployed DMaaP service on node port and add to proxy configuration: dr-node
2. K8S External Service.  Deploy services at Edge which map to central services.
   1. REF: https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-mapping-external-services
3. Add entries for central services into /etc/hosts on Edge pods so they can route properly
4. Provision some external DNS service that is able to resolve to required IP addresses in other k8s cluster
5. Determine how clients can specify FQDN (service name) but designate IP address to use.  
   1. See  --resolve option in curl for example of how this might work.
6. Apply k8s thinking to DMaaP component design:
   1. Abandon the DR publish redirect protocol and simply use dr-node service instead.
      1. if dr-node is local to the cluster, then client will route to local dr-node pod for publishing (which is desired)
      2. if dr-node isn't local to cluster, then client will route to central dr-node via proxy  (fallback)
   2. Change dr-prov algorithm for distributing prov data to dr-node so dr-prov doesn't need to know how to address every pod

Open Issues

  dmaap:

    dmaap-message-router:

      enabled: true

    dmaap-bus-controller:

      enabled: false

    dmaap-dr-prov:

      enabled: false

    dmaap-dr-node:

      enabled: true

...