CPS-2478: Module Sync Inefficiencies

Calls to DB for modules (check existing Tag)

Could easily cache Module Set Tag in memory to reduce this

First batch (on each thread) calls to DMI for same Tag

Use cache from #1 or store first cm Handle in DB immediately instead of as part of Batch

Store new schema set for each cm handle (instead of Tag)

Use schema set concept (in CPS Core) to only store each new Module Set Tag once. This seems the correct usage of Schema Set concept and wil have the greatest performance benefit. This requires more costly and difficult solution as NCMP code is develop assuming each cm handle schema set name is the same as its id.
Affected use-cases:

1. Initial Registration

2. Upgrade

Cm Handles

500

Module Set Tags

2

250 CM Handles Each

Worker Threads (parallelism)

2

Environment

Windows 11. 13th Gen Intel(R) Core(TM) i9-13900H 2.60 GHz

# Calls

Time Spent (ms)

%

# Calls

Time Spent (ms)

%

query module references

500

1,017

7%

2

5

0%

get modules from DMI

100-200

1,326

9%

2

13

0%

store schema set

500

10,449

73%

500

5,156

86%

2 x faster. Probably due to less contention with read queries

update states

5+

1,429

10%

5+

833

14%

1.7 x faster

Total

14,221

100%  

6,006

100%  

> 2 x faster!

query module references

40,670

7%

get modules from DMI

54,050

9%

add 200ms delay for first 10 batch of 100

store schema set

417,960

73%

update states

57,170

10%

Total

569,850

~ 9.50 Minutes