CPS-1298: Add new topics to OOM
References
CPS-1298: Add new topics to OOM config.Closed
Kafka Topic Configurations for Confluent Platform | Confluent Documentation
Assumptions
# | Assumption | Notes |
---|---|---|
1 | Partition count and Replica count are been configured at the cluster level. | Partitions : 10 Replicas : 3 |
Issues & Decisions ( ncmp-events )
We publish the Lifecycle Management Events to this topic whenever there is change in the CmHandleState.
# | Issue | Notes | Decision | Comments |
---|---|---|---|---|
1 | What is the Throughput | Number of messages produced in a certain amount of time | Not Needed | No longer need to know as dimension is done at deployment and runtime |
2 | Retention Time | How long to keep the messages in the kafka topic before(deleting/compacting it)? | 2hrs | Since some other teams will be consuming these messages so we will give them some time to consume. |
3 | Segment Bytes | Segment file size ie. Max. total topic size Default 1Gi (1073741824) | Not set in CPS Helm | @Fiachra Corcoran advised us to just set Retention time |
4 | Access Control | Which user has access to Read , Write from the topic? | cps-kafka-user | already defined in the OOM charts. |
5 | Consumer Groups | Does CPS team define the consumer group name or the consuming application. | Yes: ncmp-group | Already defined in the application.yml config |
Issues & Decisions ( ncmp-async-m2m )
This topic is currently used for the async communication. (update more details)
# | Issue | Notes | Decision | Comments |
---|---|---|---|---|
1 | What is the Throughput | Number of messages produced in a certain amount of time | Not Needed | No longer need to know as dimension is done at deployment and runtime |
2 | Retention Time | How long to keep the messages in the kafka topic before(deleting/compacting it)? | 10mins (allow dor restart) | Machine to machine communication between cps and dmi. |
3 | Segment Bytes | Segment file size ie. Max. total topic size Default 1Gi (1073741824) | Not set in CPS Helm | @Fiachra Corcoran advised us to just set Retention time |
4 | Access Control | Which user has access to Read , Write from the topic? | cps-kafka-user | already defined in the OOM charts. |
5 | Consumer Groups | Does CPS team define the consumer group name or the consuming application. | Yes: ncmp-group | Already defined in the application.yml config |