CPS-1298: Add new topics to OOM

References

CPS-1298: Add new topics to OOM config.Closed

Kafka Topic Configurations for Confluent Platform | Confluent Documentation

Assumptions

#

Assumption

Notes

#

Assumption

Notes

1

Partition count and Replica count are been configured at the cluster level.
AS explained by @Fiachra Corcoran 

Partitions : 10

Replicas : 3

Issues & Decisions ( ncmp-events )

  • We publish the Lifecycle Management Events to this topic whenever there is change in the CmHandleState.

#

Issue

Notes 

Decision

Comments

#

Issue

Notes 

Decision

Comments

1

What is the Throughput

Number of messages produced in a certain amount of time

Not Needed

No longer need to know as dimension is done at deployment and runtime

2

Retention Time

How long to keep the messages in the kafka topic before(deleting/compacting it)?

2hrs

Since some other teams will be consuming these messages so we will give them some time to consume.

3

Segment Bytes

Segment file size ie. Max. total topic size

Default 1Gi (1073741824)

Not set in CPS Helm

@Fiachra Corcoran advised us to just set Retention time

4

Access Control

Which user has access to Read , Write from the topic?

cps-kafka-user

already defined in the OOM charts.

5

Consumer Groups

Does CPS team define the consumer group name or the consuming application.

Yes: ncmp-group

Already defined in the application.yml config



Issues & Decisions ( ncmp-async-m2m )

  • This topic is currently used for the async communication. (update more details)



#

Issue

Notes 

Decision

Comments

#

Issue

Notes 

Decision

Comments

1

What is the Throughput

Number of messages produced in a certain amount of time

Not Needed

No longer need to know as dimension is done at deployment and runtime

2

Retention Time

How long to keep the messages in the kafka topic before(deleting/compacting it)?

10mins (allow dor restart)

Machine to machine communication between cps and dmi.

3

Segment Bytes

Segment file size ie. Max. total topic size

Default 1Gi (1073741824)

Not set in CPS Helm

@Fiachra Corcoran advised us to just set Retention time

4

Access Control

Which user has access to Read , Write from the topic?

cps-kafka-user

already defined in the OOM charts.

5

Consumer Groups

Does CPS team define the consumer group name or the consuming application.

Yes: ncmp-group

Already defined in the application.yml config