References:
...
- With previous versions we were able to register 10k cmhandles (in 4 threads) in 12 minutes.
- With the current CPS master we sent the 10k cmHandles around 10 minutes to NCMP, so it is not faster than before. What I see is that NCMP starts getting modules in a background thread (seems to be one thread) and handling one cmhandle takes more seconds.
- Currently in 12 minutes only 150 cmHandles got available with ready state in NCMP, because getting modules and changing the state is quite slow this way.
...
| # | Parameter | Value |
|---|---|---|
| 1 | timers: advised-modules-sync: sleep-time-ms: | 30000 |
| 2 | timers: locked-modules-sync: sleep-time-ms: | 300000 |
| 3 | timers: cm-handle-data-sync: sleep-time-ms: | 30000 |
| 4 | data-sync: cache: enabled: | false |
Improvement Details
| Performance matrix of improved methods that are called by ModuleSyncWatchdog or Cm hanlde registration API | |||||||||
| Test Environment | API | Sequential processing time | Parallel processing time | Number of DB calls reduced by parallel / batch processing | |||||
| Best Case | Average Case | Worst Case | Best Case | Average Case | Worst Case | ||||
| Method Name | Task Parallelism Level = 1 | ||||||||
| Local Development Machine with CSIT docker deployment (with dmi-plugin that is implemented by CPS team) | org.onap.cps.ncmp.api.impl.event.lcm.LcmEventsCmHandleStateHandlerImpl#registerNewCmHandle | ||||||||
| org.onap.cps.ncmp.api.inventory.InventoryPersistence#saveCmHandleStates | |||||||||
| org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#addChildDataNodes | |||||||||
| org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#replaceDataNodeTree | |||||||||
| org.onap.cps.spi.repository.FragmentRepositoryCpsPathQueryImpl#findByAnchorAndCpsPath | |||||||||
| org.onap.cps.spi.repository.SchemaSetYangResourceRepositoryImpl#insertSchemaSetIdYangResourceId | |||||||||
| org.onap.cps.spi.repository.YangResourceNativeRepository#getResourceIdsByModuleNameAndRevision | |||||||||
| Task Parallelism Level = 30 | |||||||||
| org.onap.cps.ncmp.api.impl.event.lcm.LcmEventsCmHandleStateHandlerImpl#registerNewCmHandle | |||||||||
| org.onap.cps.ncmp.api.inventory.InventoryPersistence#saveCmHandleStates | |||||||||
| org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#addChildDataNodes | |||||||||
| org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#replaceDataNodeTree | |||||||||
| org.onap.cps.spi.repository.FragmentRepositoryCpsPathQueryImpl#findByAnchorAndCpsPath | |||||||||
| org.onap.cps.spi.repository.SchemaSetYangResourceRepositoryImpl#insertSchemaSetIdYangResourceId | |||||||||
| org.onap.cps.spi.repository.YangResourceNativeRepository#getResourceIdsByModuleNameAndRevision | |||||||||
| # | Operations | Average Time elapsed (in ~ ms) | |||
|---|---|---|---|---|---|
| 1 | To save cm handle state with id | Batch #.Iteration # | Time elapsed (in ~ ms) | ||
| 1.1 | 293 | ||||
| 1.2 | 84 | ||||
| 2.1 | 14 | ||||
| 2.2 | 8 | ||||
| Average Time elapsed per cm handle (in ~ ms) | 53 | ||||
| 2 | To synchronize, create schema set and anchor with id | Batch #.Iteration # | Time elapsed (in ~ ms) | ||
| 1.1 | 38393 | ||||
| 1.2 | 599 | ||||
| 2.1 | 237 | ||||
| 2.2 | 122 | ||||
| Average Time elapsed per cm handle (in ~ ms) | 0.483 | ||||
| 3 | Delete schema set If exists with id | Batch #.Iteration # | Time elapsed (in ~ ms) | ||
| 1.1 | 17 | ||||
| 1.2 | 2 | ||||
| 2.1 | 2 | ||||
| 2.2 | 2 | ||||
| Average Time elapsed per cm handle (in ~ ms) | 11.5 | ||||
| 4 | Get cm handle state with id | Batch #.Iteration # | Time elapsed (in ~ ms) | ||
| 1.1 | 83 | ||||
| 1.2 | 7 | ||||
| 2.1 | 10 | ||||
| 2.2 | 12 | ||||
| Average Time elapsed per cm handle (in ~ ms) | 56 | ||||
| 5 | To get yang models | Batch # | Batch Size | Time elapsed (in ~ ms) | Time elapsed per cm handle (in ~ ms) |
| 1 | 601 | 63107 | 105.00 | ||
| 2 | 857 | 37337 | 43.56 | ||
| Average Time elapsed per iteration (in ~ ms) | 74.28 | ||||
| 6 | To shuffle cm handles | Batch # | Batch Size | Time elapsed (in ~ ms) | |
| 1 | 601 | 0 | |||
| 2 | 857 | 0 | |||
| Average Time elapsed per iteration (in ~ ms) | 0 | ||||
| 7 | Get / fetch cm handles by state ADVISED | Batch # | Batch Size | Time elapsed (in ~ ms) | Time elapsed per cm handle (in ~ ms) |
| 1 | 601 | 19404 | 32.28 | ||
| 2 | 857 | 3485 | 4.06 | ||
| Average Time elapsed per iteration (in ~ ms) | 18.17 | ||||
...