Performance

Level 1 - Design

• Policy GUI

• PAP

Level 1+ - Runtime

• PAP (level 1)

• Drools

• XACML

• Apex (level 1)

• Distribution (level 1)

Level 1 - Design

• Policy GUI

• PAP (to be deprecated)

Level 2 - Runtime

• PAP

• API

• Drools

• XACML

• Apex

• Distribution

POLICY-1458: Create S3P JMeter Tests for Policy APIClosed

POLICY-1460: Create S3P JMeter Tests for PAPClosed

POLICY-1461: Create S3P JMeter Tests for Policy XACML Engine (2nd Generation)Closed

POLICY-1462: Create S3P JMeter Tests for Policy SDC DistributionClosed

Performance

•  

  • Level 0: no performance testing done

  • Level 1: baseline performance criteria identified and measured  (such as response time, transaction/message rate, latency, footprint, etc. to be defined on per component)

  • Level 2: performance improvement plan created 

  • Level 3: performance improvement plan implemented for 1 release (improvement measured for equivalent functionality & equivalent hardware)

Stability

Level 2

Level 2

POLICY-1458: Create S3P JMeter Tests for Policy APIClosed

POLICY-1460: Create S3P JMeter Tests for PAPClosed

POLICY-1461: Create S3P JMeter Tests for Policy XACML Engine (2nd Generation)Closed

POLICY-1462: Create S3P JMeter Tests for Policy SDC DistributionClosed

Stability

•  

  • Level 0: none beyond release requirements

  • Level 1: 72 hour component-level soak test (random test transactions with 80% code coverage exercising major code branches; steady load)

  • Level 2: 72 hour platform-level soak test (random test transactions with 80% code coverage exercising major code branches; steady load)

  • Level 3: track record over 6 months of reduced defect rate

Resiliency

Level 2

Level 2

POLICY-1458: Create S3P JMeter Tests for Policy APIClosed

POLICY-1460: Create S3P JMeter Tests for PAPClosed

POLICY-1461: Create S3P JMeter Tests for Policy XACML Engine (2nd Generation)Closed

POLICY-1462: Create S3P JMeter Tests for Policy SDC DistributionClosed

Resiliency Levels

•  

  • Level 0: no redundancy

  • Level 1: support manual failure detection & rerouting or recovery within a single site; tested to complete in 30 minutes

  • Level 2: support automated failure detection & rerouting 

    • within a single geographic site

    • stateless components: establish baseline measure of failed requests for a component failure within a site 

    • stateful components: establish baseline of data loss for a component failure within a site

  • Level 3: support automated failover detection & rerouting 

    • across multiple sites 

    • stateless components 

      • improve on # of failed requests for component failure within a site 

      • establish baseline for failed requests for site failure 

    • stateful components 

      • improve on data loss metrics for component failure within a site 

      • establish baseline for data loss for site failure

Security

Level 1

Level 1+

• Stretch Goal: CII Silver Badge

• All components have encrypted communications

• All components will support CADI/AAF

POLICY-1223: CII Silver Badging RequirementsClosed

Security Levels

• Project-level requirements

  • Level 0: None

  • Level 1: CII Passing badge

    • Including no critical and high known vulnerabilities > 60 days old

  • Level 2: CII Silver badge, plus:

    • All internal/external system communications shall be able to be encrypted.

    • All internal/external service calls shall have common role-based access control and authorization using CADI framework.

  • Level 3: CII Gold badge 

Scalability

Level 1

Level 1

POLICY-1458: Create S3P JMeter Tests for Policy APIClosed

POLICY-1460: Create S3P JMeter Tests for PAPClosed

POLICY-1461: Create S3P JMeter Tests for Policy XACML Engine (2nd Generation)Closed

POLICY-1462: Create S3P JMeter Tests for Policy SDC DistributionClosed

Scalability

•  

  • Level 0: no ability to scale

  • Level 1: supports single site horizontal scale out and scale in, independent of other components

  • Level 2: supports geographic scaling, independent of other components

  • Level 3: support scaling (interoperability) across multiple ONAP instances

Manageability

Level 1

Level 2

• TBD component level upgrade, may simply being able to support patches. Wait for more clarity.

https://lf-onap.atlassian.net/browse/POLICY-1284

Manageability

•  

  • Level 1:

    • All ONAP components will use a single logging system.

    • Instantiation of a simple ONAP system should be accomplished in <1 hour with a minimal footprint

  • Level 2:

    • A component can be independently upgraded without impacting operation interacting components

    • Component configuration to be externalized in a common fashion across ONAP projects

    • All application logging to adhere to ONAP Application Logging Specification v1.2

    • Implement guidelines for a minimal container footprint

  • Level 3

    • Transaction tracing across components

Usability

Level 1

Level 2

Stretch goal to conform to API Common Versioning Strategy

Usability

•  

  • Level 1:

  •  

    • User guide created

    • Deployment documentation

    • API documentation

    • Adherence to coding guidelines

  • Level 2:

  •  

    • API Documentation

      • All new API’s must adhere to the ONAP API Common Versioning Strategy and Documentation Guidelines

      • All existing APIs must be documented in Swagger 2.0

  • Level 3

  •  

    • Consistent UI across ONAP projects

    • Usability testing conducted

    • API Documentation

      • All new API’s, all external APIs, and all existing API’s that are modified must adhere to the ONAP API Common Versioning Strategy and Documentation Guidelines

  • Level 4

  •  

    • API Documentation

      • All API’s for a given project must adhere to the ONAP API Common Versioning Strategy and Documentation Guidelines