Datafile Collector
Performance
The following performance tests can be found in the repo, https://gerrit.onap.org/r/admin/repos/integration, under "test/mocks/datafilecollector-testharness/auto-test".
700 nodes (700 events) with 1MB file each, 5 polls → total 3500 files (3.5 GB) | ||||||
|---|---|---|---|---|---|---|
| HW | Test Case | FTP prot | Dublin version: 1.1.2 | Dublin version: 1.1.3 | ||
| Execution time (minutes) | Files per second | Execution time (minutes) | Files per second | |||
| Ubuntu 16, 8 vCPU, 32GB | FTC10 | SFTP | 8,5 | 6,9 | 6,5 | 9 |
| Ubuntu 16, 8 vCPU, 32GB | FTC11 | FTPS | 3,5 | 16,7 | 2,5 | 23 |
700 nodes (3500 event) with 100 1MB files in 100 polls → total 72800 files (72.8 GB) (From polling to all files downloaded and published) | ||||||
|---|---|---|---|---|---|---|
| HW | Test Case | FTP prot | Dublin version: 1.1.2 | Dublin version: 1.1.3 | ||
| Execution time (minutes) | Files per second | Execution time (minutes) | Files per second | |||
| Ubuntu 16, 8 vCPU, 32GB | FTC20 | SFTP | 137 | 8,9 | 123 | 9,8 |
| Ubuntu 16, 8 vCPU, 32GB | FTC21 | FTPS | 68 | 17,8 | 60 | 20 |
Max load test on DFC, polling 3500 new files every poll for 24h (from 700 nodes) | ||||||
|---|---|---|---|---|---|---|
| HW | Test Case | FTP prot | Dublin version: 1.1.2 | Dublin version: 1.1.3 | ||
| Polled, downloaded and published files | Files per second | Polled, downloaded and published files | Files per second | |||
| Ubuntu 16, 8 vCPU, 32GB | MaxFiles1MB_SFTP_24h | SFTP | 215000 | 2,5 | 365000 | 4,2 |
| Ubuntu 16, 8 vCPU, 32GB | MaxFiles1MB_FTPS_24h | FTPS | 990500 | 11,5 | 1063741 | 12,3 |
3 day test. 100 files from 700 nodes in the first event from each node. Then one new file per node every 15 min. Dublin version: 1.1.2 | ||||
|---|---|---|---|---|
| HW | Test Case | FTP prot | Dublin version: 1.1.2 | Dublin version 1.1.3 |
| Polled, downloaded and published files | ||||
| Ubuntu 16, 8 vCPU, 32GB | Stability1MB_SFTP_72h | SFTP | 270900 (expected number of files) | 270900 (expected number of files) |
| Ubuntu 16, 8 vCPU, 32GB | Stability1MB_FTPS_72h | FTPS | 270060 (840 files less than target. Not all 72h polls where made probably due to delays in the simulators or too agressive test criterias. However all polled files downloaded and published successfully. ) | |