...
Method | Before (avg. 4 runs) | After (avg. 6 runs) | Notes (improvements) | ||||
|---|---|---|---|---|---|---|---|
# Calls | Time Spent (ms) | % | # Calls | Time Spent (ms) | % | ||
query module references | 500 | 1,017 | 7% | 2 | 5 | 0% | Used ‘privateModuleSetCache’ map to locally store required data on each thread. Data discarded when thread finishes but this eliminates vast majority of DB calls. |
get modules from DMI | 100-200 | 1,326 | 9% | 2 | 13 | 0% | Use a Hazelcast distributed Set: ‘moduleSetTagsBeingProcessed’ to prevent multiple threads/instances attempting to process the same new tag. |
store schema set | 500 | 10,449 | 73% | 500 | 5,156 | 86% | 2 x faster. Probably due to less contention with read queries |
update states | 5+ | 1,429 | 10% | 5+ | 833 | 14% | 1.7 x faster |
Total | 14,221 | 6,006 | > 2 x faster! | ||||
...
Methods | Before | After | Notes | ||
|---|---|---|---|---|---|
Time Spent (ms) | % | Time Spent (ms) | % | ||
query module references | 40,670 | 7% | 5 | 0% | |
get modules from DMI | 54,050 | 9% | 2,667 | 1% | add 200ms delay for first 10 batch of 100 |
store schema set | 417,960 | 73% | 206,220 | 85% | |
update states | 57,170 | 10% | 33,313 | 14% | |
Total | 569,850 | ~9m30s | 242,205 | ~4m3s | Need to a add 2 minutes for initial delay: ~6m ~ 55 CM Handles/sec |