Design Analysis: Forwarding Cm Data Notification
- Lee Anjella Macabuhay
- Gliffy Diagram for Confluence
Overview
Objective
The objective of this feature is for CPS-NCMP to successfully forward notification to the correct topic(s) of a relevant subscription.
High-Level Work Flow
| 1 | Trigger event | DMI cm data notification event |
| 2 | Process flow |
|
| 3 | Outcome | Event forwarded to correct topics of subscribers |
Feature Components
Component | Responsibilities |
|---|---|
NCMP |
|
Kafka | Consumer
|
Producer
| |
Database | Subscription information |
DME |
|
Service Layer Design
Mapping of events to correct subscriptions
Prerequisite:
consumed event
For each xPath collect subscription IDs by querying database , i.e.
final Collection<DataNode> existingNodes =
cpsQueryService.queryDataNodes(NCMP_DATASPACE_NAME, CM_SUBSCRIPTIONS_ANCHOR_NAME,
isOngoingCmSubscriptionCpsPathQuery, OMIT_DESCENDANTS);
if (existingNodes.isEmpty()) {
return Collections.emptyList();
}
return (List<String>) existingNodes.iterator().next().getLeaves().get("subscriptionIds")
Identifying topic for given subscriptions
Process each subscription IDs and get a set of topics
Prerequisite:
format of subscription ID
format of topic name
Assumption
simple regex for the subscription id to extract topic name
(.+)#(.*)
group 1 = subscription id
group 2 = topic name
Forwarding Notification based on subscription
Validating topic? (can this be done in creation of subscription?)
Prerequisite:
consumed event
topic name
Error Handling
Description |
|
|
|---|---|---|
Dead-letter queues? |
|
|
No write access to kafka topic | Log failed request |
|
Kafka topic does not exist | Log failed request |
|
No valid topic name in subscription ID | Reject* | move up top |
Performance Considerations
NCMP
Caching
Deserializing**
regex
Kafka
Configuration on polling
Security Considerations
Data validation
Event consumed will be forwarded as is
Access control (write permissions) to topics is controlled by client
Testing
unit tests
integration tests
E2E
k6?
Scenario | Expected Result | |
|---|---|---|
| 1 | Consume event from DMI | No error , Log trace level ? |
| 2 | Publish event | Published event with correct details (topic name, event key, event) |
| 3 | No subscription Ids returned for certain xpath (Notification maps to no subscription) | No forwarded event Increase counter for not forwarded events |
| 4 | Notification maps to one subscription | Event forwarded to one topic |
| 5 | Notification maps to multiple subscription (no duplication on topics) | Event forwarded to more than one topic |
| 6 | Notification maps to two subscriptions having the same topic | Event forwarded to one topic ONLY once |
| 7 | No topic extracted from subscription ID | Reject event |
| 8 | No write access to topic | Log error |
| 9 | Kafka not available? | Log error |
Monitoring and Logging
Metrics
count notification on not forwarded events
Logs
log of errors? log key events? messaged produced, log consumed events?
Failed request when a client topic does not exist