References:
...
| Test Environment | Secs Taken for ModuleSyncWatchdog (Cm Handle moves ADVISED->READY) | Secs taken for Registration sub-process | Commit id | Parrealel Request to DMI | Main improvement | Comments |
|---|---|---|---|---|---|---|
Ericsson Testing cluster (with Model adapter that is implementation of dmi-plugin) Total 1458 cm Handles | 6,998 → 0.20/sec | 2,278 → 0.64/sec | 1410509e33142c0c79ff0e111c63abc47f5936d3 https://gerrit.onap.org/r/gitweb?p=cps.git;a=commit;h=1410509e33142c0c79ff0e111c63abc47f5936d3 | N/A (1) | N/A | Baseline as per the bug |
| 600 → 2.43/sec | 70 → 20.82/sec | c3596fe3a85381668598c75a1ffee4bf973a8518 | N/A (1) |
Note : Root cause : We have 3 watchdogs (2 @Scheduled into module sync and 1 into data sync) that was blocking each other as all @Scheduled methods share a single thread by default. ) | ||
| 540 → 2.70 /sec | 88(?!) → 16.57/sec | 6e97bc6ee9f1cb2722aafc42325918e98da5cff2 | N/A (1) |
| ||
| TBA | TBA | e367b520ac01432715b701af6e2bd75d4f81bccf | 30 (TBC) | 1. Introduced timeout for connect and reading for RestTemplate while connecting dmi-plugin. 2. Modified data structure of moduleSyncSemaphore to hold YangModelCmHandle referance (cm handle id) as key of map 3. Working of Modulewatchdog is modified with parallelism and broken into async tasks. 4. ModuleSyncSemaphoreMap is now populated with advised cm handles and processed by Modulewatchdog by batch size (Dynamic batch size at cm registration). 5. Introduled parallelism level for async tasks that are suppiled. 6. Includes cm handle registration preformance code change. 7. Performance Improvement: Postgres Pool Configuration 8. Performance Improvement: Async Notification Pool Configuration. 9. Performance Improvement: Save/update batch of CM-Handle States in one operation. 10. Performance Improvement: Updating (many) child datanodes/datanode tree into one SQL operation 11. Performance Improvement: Native query to more efficiently get all yangResourceIds. 12. Performance Improvement: Detaching fragment entities early. 13. Performance Improvement: Decouple Temporal DB Notification from main thread. Note : It includes bottleneck of in my recursive method (Joe K is working on it to improve. org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#toDataNode) | Unable to capture performance result as Module adapter is not able to handle concurrent requests(Csaba is working on fix) | |
Local Development Machine with CSIT docker deployment (with dmi-plugin that is implemented by CPS team) Note : dmi-plugin is able to handle only 5-8 cm handle per seconds concurrently. | 240 → 6.08 /sec | 30 → 48.6/ sec | e482e74e2dab8455d0642829005c03212936b73d | 30 |
Note : As a team we decided to not to store YangModelCmHandle object as key of map as cache. | |
Local Development Machine with CSIT docker deployment (with dmi-plugin that is implemented by CPS team) Note : dmi-plugin is able to handle only 5-8 cm handle per seconds concurrently. | 246 +356 = 602 → 2.49/sec Note : 246 sec to fetch 1500 advised cm handles (It includes bottleneck of in my recursive method : org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#toDataNode) | 31 → 48.39/ sec | 30 |
Note : It includes bottleneck of in my recursive method (Joe K is working on it to improve. (org.onap.cps.spi.impl.CpsDataPersistenceServiceImpl#toDataNode) |
Test Configuration
Use Case
...
| # | Parameter | Value |
|---|---|---|
| 1 | timers: advised-modules-sync: sleep-time-ms: | 30000 |
| 2 | timers: locked-modules-sync: sleep-time-ms: | 300000 |
| 3 | timers: cm-handle-data-sync: sleep-time-ms: | 30000 |
| 4 | data-sync: cache: enabled: | false |
Improvement Details
Performance matrix of improved methods that are called by ModuleSyncWatchdog or Cm handle registration API
...