Page Status: Updated for Istanbul -
Component Status: Pending PTL updates and ArchCom Review
Last Reviewed on:
Certified by:
DMaaP (Data Movement as a Platform):
1 High Level Component Definition and Architectural Relationships
DMaaP is component that provides data movement services that transports and processes data from any source to any target. It provides the capability to:
Support the transfer of messages between ONAP components, as well as to other components
Support the transfer of data between ONAP components as well as to other components.
Data Filtering capabilities
Data Compression capabilities
Data routing (file based transport)
Message routing (event based transport)
Batch and event based processing
The data providers and data consumers are
- Service Design and Creation
- Policy
- Data Collection Analytics and Events
- Service Orchestrator
- Active and Available Inventory
- SDNC
- CPS
- CDS
DMaaP will also now leverage the Strimzi Apache Kafka project for the deployment of Kafka, including some of the relevant Strimzi offerings.
This will replace the existing custom Confluent Kafka and Zookeeper images used by DMaaP Message Router.
This will also add a new dependency for any existing Kafka Native clients within ONAP (CPS, HV-VES, CDS, etc), including DMaaP MR.
Strimzi uses Kubernetes Operators to deploy it's Custom Resource Definitions (CRDs) to the k8s cluster.
The Cluster Operator manages the deployment of the Kafka Cluster and the Entity Operator (User and Topic Operators) which are used to manage Topic creation and Topic access control via the KafkaUser CRD.
In addition, DMaaP will offer the Strimzi Kafka Bridge which becomes an alternative ResT based messaging interface towards the Kafka Cluster:
2. DMaaP API definitions
DMaaP provides the following interfaces:
| Interface Name | Interface Definition | Interface Capabilities | API Spec (Swagger) |
|---|---|---|---|
| DMaaP-1 | DMaaP Bus Controller Interface Allows a DMaaP User to control the behavior of DMaaP | Provides services to enable a DMaaP user to:
| onap-dmaap-dbcapi swagger |
| DMaaP-2 | DMaaP Message Router Source Interface Provides a message sending service to the DMaaP user | The message sender can:
| onap-dmaap-messagerouter swagger |
| DMaaP-3 | DMaaP Message Router Consuming Interface Provides a message receiving service to the DMaaP user | The message receiver can:
| onap-dmaap-messagerouter swagger |
| DMaaP-4 | DMaaP Data Routing Source Interface Provides a data sending service to the DMaaP user | The data sender can:
| onap-dmaap-datarouter swagger |
| DMaaP-5 | DMaaP Data Routing Consumption Interface Provides a data consumtion service to the DMaaP user | The data consumer can:
| onap-dmaap-datarouter swagger |
Note: xxxI interface is a Component internal interface. xxxxE interface is a component external interface
The current API documents can be found at:
- Message Router Interface: https://docs.onap.org/projects/onap-dmaap-messagerouter-messageservice/en/latest/offeredapis/offeredapis.html
- DMaaP Bus Controller Interface: https://docs.onap.org/projects/onap-dmaap-dbcapi/en/latest/api-table.html
- DMaaP Data Routing Interface: https://docs.onap.org/projects/onap-dmaap-datarouter/en/latest/offeredapis.html
The additional Strimzi Kafka Bridge api which provides ResT protocol messaging towards Kafka can be found at:
The following interfaces are consumed by DMaaP
| Interface Name | Purpose Reason For Use | API Spec (Swagger) |
|---|---|---|
| N/A | N/A |
3. Component Description:
DMaaP consists of 3 software systems
- Message Router (MR) - Message Router is a reliable, high-volume pub/sub messaging service with a RESTful HTTP API. It is intended to be deployed by Platform Service providers so that it is available to Platform clients as a web service. The service is initially built over Apache Kafka.
- Data Router (DR) - The Data Routing System project is intended to provide a common framework by which data producers can make data available to data consumers and a way for potential consumers to find feeds with the data they require. The interface to DR is exposed as a RESTful web service known as the DR Publishing and Delivery API
- Data Bus Controller - Provisioning API of the Data Movement Platform that can be used for provisioning topics in the Message Router and feeds in Data Router.
4. Known system limitations
Current know system limitations
- DMaaP uses persistent volumes to store data used by topics and feeds. Persistent volumes need to have enough storage space to handle anticipated volume.
5. Used Models
DMaaP does not use any models.
6. System Deployment Architecture
DMaaP consists of 3 main components:
- DMaaP Message Router
- DMaaP Data Router
- DMaaP Buss Controller
7. New Capabilities in this Release
The addition of the Strimzi Apache Kafka project for the management of the Kafka deployment within Kubernetes
8. References
- DMaaP Wiki: Data Movement as a Platform Project
- DMaaP on readthedocs: https://docs.onap.org/en/latest/guides/onap-developer/developing/index.html#dmaap-data-management-as-a-platform
- Message Router: https://docs.onap.org/projects/onap-dmaap-messagerouter-messageservice/en/latest/index.html#master-index
- Bus Controller: https://docs.onap.org/projects/onap-dmaap-buscontroller/en/latest/index.html#master-index
- Data Router: https://docs.onap.org/projects/onap-dmaap-datarouter/en/latest/index.html#master-index