Performance | 0 | 1 | Need to work with the benchmarking project and define the critera | - 0 -- none
- 1 – baseline performance criteria identified and measured
- 2 & 3 – performance improvement plans created & implemented
|
Stability | 0 | 1 (stretch goal - 2) | 1.The load should be defined with the consensus of the benchmarking project 2.Need to soak test the SO for 72 hours with the defined load | - 0 – none
- 1 – 72 hours component level soak w/random transactions
- 2 – 72 hours platform level soak w/random transactions
- 3 – 6 months track record of reduced defect rate
|
Resiliency | 1 | 2
| Have to incorporate the rollback flows for all the active workflows. | - 0 – none
- 1 – manual failure and recovery (< 30 minutes)
- 2 – automated detection and recovery (single site)
- 3 – automated detection and recovery (geo redundancy)
|
Security | 0 | 1 (stretch goal - 2) | 1.The current code coverage needs improvement from 29% to 50% 2.Improve the documentation (readthedocs) of the SO describing what the software does (what problem does it solve?). 3. use at least one automated test suite that is publicly released as FLOSS | - 0 – none
- 1 – CII Passing badge + 50% Test Coverage
- 2 – CII Silver badge; internal communication encrypted; role-based access control and authorization for all calls
- 3 – CII Gold
|
Scalability | 0 | 1 | SO will have - new APIs for manual and Auto scaling for platform enhancement
- at component level SO is identifying the ways to support scaling.
| - 0 – no ability to scale
- 1 – single site horizontal scaling
- 2 – geographic scaling
- 3 – scaling across multiple ONAP instances
|
Manageability | 0 | 1( | stretch goal - 2)1.SO should use the common logging for all its logs 2.SO dockers (both MSO and MariaDB) should be instantiated within one hour | - 1 – single logging system across components; instantiation in < 1 hour
- 2 – ability to upgrade a single component; tracing across components; externalized configuration management
|
Usability | 0 | 1(stretch goal - 2) | 1.Improve SO user guide 2. Improve the API documentation | - 1 – user guide; deployment documentation; API documentation
- 2 – UI consistency; usability testing; tutorial documentation
|