Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

References:

...

Expectation/ goal: cm-handle registration + module sync → 25 cm handles / sec (see CPS-NCMP-I-01 Specification: Performance Requirements)

Improvements Overview

Test Environment

Secs Taken for

ModuleSyncWatchdog (Cm Handle moves ADVISED->READY)

Secs taken for
Registration sub-process
Commit idParrealel Request to DMIMain improvementComments

Ericsson Testing cluster (with Model adapter that is implementation of dmi-plugin)

Total 1458 cm Handles

6,998 → 0.20/sec2,278 → 0.64/sec1410509e33142c0c79ff0e111c63abc47f5936d3N/A (1)N/ABaseline as per the bug
600 → 2.43/sec70 → 20.82/secc3596fe3a85381668598c75a1ffee4bf973a8518N/A (1)
  1. Created a dedicated threadpool for scheduler. - Tuned async threadpool of notification executor from setting RejectedExecutionHandler and application.yml.

Note : Root cause : We have 3 watchdogs (2 @Scheduled into module sync and 1 into data sync) that was blocking each other as all @Scheduled methods share a single thread by default. )


540 → 2.70 /sec88(?!) → 16.57/sec6e97bc6ee9f1cb2722aafc42325918e98da5cff2N/A (1)
  1. Mapped additional properties of YangModelCmHandle.
  2. @Async : scheduler are executing tasks asynchronously 

TBATBAe482e74e2dab8455d0642829005c03212936b73d30 (TBC)

1. Introduced timeout for connect and reading for RestTemplate while connecting dmi-plugin.

2. Modified data structure of moduleSyncSemaphore to hold YangModelCmHandle referance (cm handle id) as key of map

3. Working of Modulewatchdog is modified with parallelism and broken into async tasks.

4. ModuleSyncSemaphoreMap is now populated with advised cm handles and processed by Modulewatchdog by batch size (Dynamic batch size at cm registration).

5. Introduled parallelism level for async tasks that are suppiled.

6. Includes cm handle registration preformance code change.

7. Performance Improvement: Postgres Pool Configuration

8. Performance Improvement: Async Notification Pool Configuration.

9. Performance Improvement: Save/update batch of CM-Handle States in one operation.

10. Performance Improvement: Updating (many) child datanodes/datanode tree into one SQL operation

11. Performance Improvement: Native query to more efficiently get all yangResourceIds.

12. Performance Improvement: Detaching fragment entities early.

13. Performance Improvement: Decouple Temporal DB Notification from main thread.

Unable to capture performance result as Module adapter is not able to  handle concurrent requests(Csaba is working on fix)

Local Development Machine with CSIT docker deployment (with dmi-plugin that is  implemented by CPS team) 

Note : dmi-plugin is able to handle only 5-8 cm handle per seconds concurrently.

240 → 6.08 /sec30 → 48.6/ sece482e74e2dab8455d0642829005c03212936b73d30

Test Configuration 

Use Case

...

#Parameter
Value
1
timers:
    advised-modules-sync:
        sleep-time-ms: 

30000

2
timers:
    locked-modules-sync:
        sleep-time-ms: 
300000
3
timers:
   cm-handle-data-sync:
       sleep-time-ms: 
30000
4
data-sync:
    cache:
        enabled: 

false

Improvement Details

Performance matrix of improved methods that are called by ModuleSyncWatchdog or Cm handle registration API 

...