Objectives

Badging  for ONAP must fit within the policy constrains of the LFN C&V committee. The currently inforce C&V policies, refer to lab qualification procedures. The numner/scope of individual badging efforts is  controlled by the LFN projects' (in our case ONAP's) TSC. This committee seems to be converging on leveraging the OPNFV Dovetail framework for aggregating testing results.

The objective is to  extract / summarize VNF Requirements into a VNF Badge. The VNF Requirements were original developed as prototype RFP text. Not all of the Requirements are testable ( either currently or in the near future).ONAP VNF Requirements evolve with ONAP releases. ( See e.g. Casablanca release notes ).  The test implementations are also expected to evolve with ONAP releases.  The VVP project is implementing HEAT related test scripts. The VNFSDK project is implementing TOSCA based test scripts. Test scripts may be invoked in a variety of different use cases.  The proportion of testable requirements having tests implemented is expected to increase with each ONAP release. The scope of testing is expected to evolve from onboarding (Package inspection), to Life Cycle operations and Functional testing. In Casablanca test descriptions are available for VNF Package inspection tests. Test Case Descriptions for Life Cycle operations may be candidate work items for future releases.

The specific badge would need to be aligned with the LFN guidance on logos, but something along the following lines may be a place to start.

Open

There is some discretion to adjust portions of the logo to reflect the scope of the assurance proposed by the badge. This example proposes a scope around "VNF Management" but other scopes may be possible. More than one badge may be possible if the material is sufficiently independent. The "2018.11" reflects the ONAP release which provides the conformance reference for the badge. VNF Badging should avoid overlap/ duplication with VNF validation based on testing, ideally the two should be linked.

One approach is to use a format similar to the CII Badging website  see e.g. CII Badge for VVP. There are some differences for VNF badging vs CII badging ( beyond the obvious subject matter). This would allow human entered data to be supplemented by automated test results (e.g. from the Dovetail framework).

VNF Requirements Analysis

The Casablanca VNF Requirements are available for download from the docs.onap.org website.

The breakdown of requirements by keyword is

Scoring  with simple counts of Met/Unmet will not be sufficient:

     R-89571 provides for alternatives (one or more of) : Netconf/ Chef/ Ansible

    Section 7.4 has MAY or SHOULD  requirements which should not have the same weight as a MUST requirement.  

The breakdown of requirements by target is

VNF Badge Scoping Alternatives

As an initial target, it has been proposed to focus on the requirements associated with VNF Management (i.e. Chapter 7 of the ONAP VNF Requirements) . This Chapter 7 represents ~190 of the >750 requirements ie ~25% of the VNF requirements identified.

The current CII badge has ~70 requirements, in 6 pulldown menus. The max  requirements in a pulldown is 16.

For usability, we should constrain the number of pulldowns and the number of requirements per pulldown to a similar order of magnitude  e.g. pulldowns < 10, reqts/pulldown <20

VNF Package Inspection

VNF Test Case Descriptions are provided for VNF Package Inspection from Bejing.   

  Of the ~192 requirements  in section 7 about 25 are identified in the Test Description Annex  as being inspectable in the VNF package delivered from the VNF provider to the ONAP operator.

     Ch 7 includes a number of requirements on VNF Package or xNF Package  that are not included in that Test Description Annex (yet) but are presumably inspectable in the Package at some point even if the inspection tests are not yet available in VVP or VNFSDK as part of the Casablanca release. VNFRQTS-241: the VNF Test Descriptions Annex should be updated to reflect test implementationsClosed provides for updates to this document as part of the Casablanca release.

     Eliminating the requirements that are inspectable in the  xNF Package would reduce the number of requirements in the table above  -  ~30 of the 190 requirements are on the xNF or VNF Package ie ~15%

    ONAP supports several different VNF Package formats for onboarding : HEAT based, TOSCA Based (VFC:Generic VNFM, VFC:SVNFM) and  (Proposed for Dublin: Helm based ( see Req #1)). In Casablanca the number of Requirements & tests for these formats is:

The VNF Package as a handoff between the VNF Provider and the ONAP Operator includes a variety of documentation about the VNF. This documentation is required by the Network operators staff to correctly use the VNF. It is not used directly by the ONAP platform in any automated fashion.  Testing for documentation is problematic - at best, automated testing can detect the presence of some artifact, but not validate the "correctness" of the documentation. There is some overlap between VNF "package" requirements and VNF "documentation" requirements.

VNF Life Cycle Operations

VNF Test Case Descriptions for VNF Life Cycle operations are not available in Casablanca.

Developing new test case descriptions for Life Cycle operations is not trivial. The set of life cycle commands supported by the platform is expected to expand in future releases.

Life cycle operations require running instances of the VNF. The resource requirements of ONAP platform may make the use of the ONAP platform problematic  as a part of the test case description, in which case some process may be required to maintain synchronization between the test cases and the platform command (API) evolution.

VNF Functional Requirements

TBD

VNF Performance Requirements

TBD

Further details

Some Analysis of candidates for VNF Badging are discussed below: