Versions Compared

Version Old Version 4 New Version Current
Changes made by

Lee Anjella Macabuhay

Lee Anjella Macabuhay

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Component

Responsibilities

NCMP

  1. Process the event consumed and identify subscriptions based on

    1. cm handle id/alternate id

    2. xPath

  2. Identify topics for given subscription

  3. Interact with Kafka through Consumer/Producer set up

  4. Handle errors

Kafka

Consumer

  1. Consume events with the following

    1. Consumer Group: ncmp-group

    2. Topic: app.ncmp.avc.cm-events-topic

  2. Deserializing of event

Producer

  1. Produce events with the following

    1. Topic: based on subscription

    2. Event key: same as consumed event

    3. Event: consumed event

  2. Use of right keys

  3. Handle failures/retries

Database

Subscription information

DME

  1. Life cycle of topic

  2. Creation/Deletion of subscriptions

Potential Challenges with Dependencies

Challenge

Event ordering

Service Layer Design

Mapping of events to correct subscriptions

...

  • format of subscription ID

  • format of topic name

Assumption

  • simple regex for the subscription id to extract topic name

    • (.+)#(.*)

      • group 1 = subscription id

      • group 2 = topic name

Forwarding Notification based on subscription

...

Prerequisite:

  • consumed event

  • topic name

Retry logic

Error Handling

Description

Dead-letter queues?

No write access to kafka topic

Log failed request

Kafka topic does not exist

Log failed request

No valid topic name in subscription ID

Reject*

move up top

Performance Considerations

NCMP

  • Caching

  • Deserializing**

  • regex

Kafka

  • Configuration on polling

Security Considerations

  • Data validation

    • Event consumed will be forwarded as is

  • Access control (write permissions) to topics is controlled by client

...

k6?

Scenario

Expected Result

1

Consume event from DMI

No error , Log info trace level ?

2

Publish event

Published event with correct details (topic name, event key, event)

3

No subscription Ids returned for certain xpath (Notification maps to no subscription)

No forwarded event

Increase counter for not forwarded events

4

Notification maps to one subscription

Event forwarded to one topic

5

Notification maps to multiple subscription (no duplication on topics)

Event forwarded to more than one topic

6

Notification maps to two subscriptions having the same topic

Event forwarded to one topic ONLY once

7

No topic extracted from subscription IDlog error?

Reject event

8

No write access to topic

log Log error

9

Kafka not available?

Log error

Monitoring and Logging

Metrics

...