| Table of Contents |
|---|
References
CPS-2146: Analysis of Out of Memory and related Errors in NCMP
| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
...
| # | Issue | Notes | Decision |
|---|---|---|---|
| 1 | Placeholder for issue |
...
Background
The use of Hazelcast during NCMP's CM-handle Module Sync is leading to:
...
A PoC is being constructed: WIP Remove hazelcast map for module sync | https://gerrit.nordix.org/c/onap/cps/+/20724
From the PoC, the collision rate of multiple NCMP instances syncing the same handles will be reported. From this, it will be decided as to whether synchronization between the instances is neededit was determined that when running multiple instances of NCMP, there was approximately 10% of batches being processed by both instances simultaneously, which led to some handles going to LOCKED state, due to database exceptions. Two solutions proposed:
- Add a distributed lock (from Hazelcast) to create a critical section, allowing only 1 instance to to move handles to SYNCING state
- Allow collisions, by gracefully handling AlreadyDefinedExceptions in the code
Solution 1 is verified to work, and gives 50% faster registration than now. Solution 2 is not yet tested, so it is yet to be determined which has better performance/reliability.