This is a DRAFT page for collecting ideas and opinions on possible evolution of DMaaP

Byung-Woo Jun, Fiachra Corcoran

Message Router

Current State

DMaaP Message Router utilizes an HTTP REST API to service all Publish and Consume transactions. HTTP and REST standards are followed so clients as varied as CURL, Java applications and even Web Browsers will work to interact with Message Router.

Message Router uses AAF for user’s authentication and authorization.

The MessageRouter service has no requirements on what publishers can put onto a topic. The messages are opaque to the service and are treated as raw bytes. In general, passing JSON messages is preferred, but this is due to higher-level features and related systems, not the MessageRouter broker itself.

Challenges to be addressed

The current message router implementation does not give the receiver control over commits, severely limiting the bus semantics and the performance of the bus.

...

• Kafka-pixy - supports gRPC and REST. gRPC is intended to be used in production. Supports some level of security. Apache 2.0 license
• Strimzi Kafka Bridge provides a fine-grained API to manage Kafka consumers, that is pretty much a translation of the work done by the native Kafka adapter. That includes individual and cumulative commits, creation and deletion of clients, partition assignment etc. It also supports group send/receive of messages for better efficiency and Kafka headers
• Confluent REST proxy is similar in concept to the Strimzi bridge. It is stateful, and in case of scale out the client needs visibility of the REST proxy instances or guarantee that it is always routed towards the same instance by any gateway/load balancers in front of it. That implies the same deployment constraints as the Strimzi Bridge

TODO: create a checklist for selectionStrimzi looks like a suitable candidate, propose analysis in two aspects:

Native broker API: for low-latency, high volume use cases, how do we securely expose native Kafka APIs?

...