DMaaP from the Client Perspective (under construction)
- Dominic Lunanuova
Background
Developers of ONAP software components may consider utilizing the DMaaP services. Those software components are typically referred to as "publishers" (i.e. they introduce or produce the data) and "subscribers" (i.e. they read or consume the data), but collectively are referred to as "clients" of the DMaaP services. This article is intended to provide some guidance to developers of DMaaP clients.
Data Movement as a Platform, or "DMaaP" for short, is a nice abstraction that typically shows up as a single box on an architecture slide. But in reality, a developer of an ONAP application considering use of this Platform Service needs to know the details. The first decision for an ONAP application developer is whether they want to use DMaaP services or not. Assuming the choice to use DMaaP services, then an app developer needs to decide between Message Router and Data Router as the transport protocol. At its simplest, this is the difference between "small" messages distributed via a Topic versus "large" files distributed via a Feed, but there are other considerations that will be discussed.
Before an app can use a DMaaP, the service must be properly provisioned. We will discuss how this can be implemented as part of an ONAP deployment (to avoid manual steps).
And then, finally, data movement can occur between endpoints. Examples and conventions will be discussed.
Design Choices
Use DMaaP or not?
As our application components have become more granular due to the trends of microservice architectures and containers, there is a growing need to transport data between components. DMaaP is intended to standardize the implementation of this transport so that developers don't have to re-invent approaches, and can rely on a robust, feature rich platform service. At the same time, this approach does introduce some overhead, and subjects the client to resource constraints of a shared service.
Choosing between MR and DR
Some considerations for choosing between Message Router and Data Router:
Attribute | Message Router | Data Router | Comments |
|---|---|---|---|
Payload description | frequent, small (~KBs) messages | periodic, large sets of records (files) | |
Max payload size | 1 MB | x GB | |
Authentication Model | AAF Credentials via Basic Authorization | Local credentials via Basic Authentication | |
Persistence | Yes (kafka data store on PV) | Yes (Linux file on PV) | |
High Availability | Using ZK ensemble | Using horizontal scaling | |
Data ordering | Possible | Not guaranteed | |
Subscriber implementation | HTTP Client | HTTP Server | NOTE: the HTTP Server implementation implies the installation of a server SSL certificate. |
Some ways MR and DR are the same:
Attribute | Message Router | Data Router | Comments |
|---|---|---|---|
Bus Architecture | Yes | Yes | Both use "publisher" and "subscriber" terminology for end points |
HTTP/HTTPS protocol | Yes | Yes | |
Data Expiration | Yes | Yes | NOTE: different default values. MR: 7 days. DR: 24 hours. |
Publisher implementation | HTTP Client | HTTP Client |
DMaaP Provisioning
Provisioning for Message Router Clients
The provisioning of Authenticated Topics and the subsequent publish/subscribe actions on that Authenticated Topic rely on the proper AAF Permissions to be granted to various Identities.
The following Identities are referenced:
IdentityMR - the AAF Identity used by Message Router to access AAF and perform authorization checks.
IdentityTopicMgr - the AAF Identity used by Bus Controller to access AAF and create Permissions and Roles, and grant those Permissions to Roles and Identities.
IdentityPub - the AAF Identity used by the publisher of the Topic when accessing the MR API. The fully qualified Identity typically belongs to a namespace related to the application. i.e. IdentityPub@NamespacePub
IdentitySub - the AAF Identity used by the subscriber of the Topic when accessing the MR API. The fully qualified Identity typically belongs to a namespace related to the application. i.e. IdentitySub@NamespaceSub
Pre-requisites:
Identities created in AAF
Credentials for Identities available in kubernetes values (probably as Secrets)
IdentityTopicMgr has access permission granted for all Application Namespaces