Page Comparison

Deploy VNF test cases once and trigger it safely from anywhere 

Objectives

• as part of LFN CVP ONAP wants test platform where VNF packages could be certified using ONAP requirements to drive industry adoption
• Provide an platform where vendor/operator can develop, deploy, run test cases and query the results
• Test cases, test results and VNF should be manageable .i,e with authorization, so only user with given roles is allowed to perform operation like
  VNF package upload/download, run compliance verification tests, allow only specific VIM for specific users, etc.
• Test results should be persisted and should be available for human analysis later via LFN infrastructure.
• Provides test flow where author make flow across different test cases for a given program like compliance verification and  VNFREQS/SOL0004.
• Provide integration with OPNFV dovetail to run test cases across dovetail and VNFSDK.
• Deployable as docker container.

Architecture

More Details

VNFSDK - VNF Test platform (VTP)

REST API

Casablanca

...

Casablanca 

"/onapapi/vnfsdk-marketplace/v1/vtp/tests": {
      "get": {
        "tags": [
          "VNF Test Platform"
        ],
        "summary": "VTP Test cases",
        "description": "Returns the list of test cases",
        "operationId": "listTests",
        "produces": [
          "application/json"
        ],
        "parameters": [],
        "responses": {
          "200": {
            "description": "successful operation",
            "schema": {
              "type": "string"
            }
          },
          "500": {
            "description": "Failed to retrieve the tests",
            "schema": {
              "type": "string"
            }
          }
        }
      }
    },
    "/onapapi/vnfsdk-marketplace/v1/vtp/tests/{testName}/run": {
      "post": {
        "tags": [
          "VNF Test Platform"
        ],
        "summary": "Run VTP testcase",
        "description": "Runs the given test case and returns the result",
        "operationId": "runTest",
        "consumes": [
          "application/json"
        ],
        "produces": [
          "application/json"
        ],
        "parameters": [
          {
            "name": "testName",
            "in": "path",
            "description": "test Name",
            "required": true,
            "type": "string"
          }
        ],
        "responses": {
          "404": {
            "description": "Test case not found",
            "schema": {
              "type": "string"
            }
          },
          "500": {
            "description": "VTP internal failure",
            "schema": {
              "type": "string"
            }
          }
        }
      }
    }

Dublin

...

Tests

GET on /v1/vtp/tests

[
{
testsuite: "validation",
testcase: "csar-validate"
}
]

GET on /v1/vtp/tests/csar-validate

{
outputs: [
{
name: "error",
description: "Validation error details",
scope: "short",
type: "string"
}
],
name: "csar-validate",
description: "Validate CSAR package formats",
inputs: [
{
name: "csar",
description: "CSAR file path",
long_option: "csar",
short_option: "b",
type: "binary",
is_optional: false
}
],
testsuite: "validation"
}

Executions & results 

POST /v1/vtp/tests/csar-validate/run
Body: {"csar":"/tmp/VoLTE.csar"}

{
"results" : [ {
"error" : "SUCCESS"
} ],
"build_tag" : null,
"criteria" : "PASS"
}
}

This will be enabled with async executon in furture with following API (TBD)

POST on /vtp/v1/tests/test-id for given VNF id, profile id returns execution id

GET on /vtp/v1/tests/test-id/executions?vnf-id=xxx&&test-id=yyy

...

Provides required API for qureying and executing the test cases as part of VSP Compliance check use case in dublin.

Wiki Markup

swagger: '2.0' info: version: 1.0.0 title: '' basePath: /onapapi/vnfsdk-marketplace/v1 tags: - name: VTP Scenario - name: VTP Execution paths: '/vtp/scenarios/{scenario}/testsuites': get: tags: - VTP Scenario summary: ' List available test suties in given scenario' description: '' operationId: listTestSuties produces: - application/json parameters: - name: scenario in: path description: Test scenario name required: true type: string responses: '200': description: successful operation schema: type: array items: $ref: '#/definitions/VTPTestSuite' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' '/vtp/scenarios/{scenario}/testcases': get: tags: - VTP Scenario summary: ' List available test cases' description: '' operationId: listTestcases produces: - application/json parameters: - name: scenario in: path description: Test scenario name required: true type: string - name: testSuiteName in: query description: Test suite name required: false type: string responses: '200': description: successful operation schema: type: array items: $ref: '#/definitions/VTPTestCase' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' '/vtp/scenarios/{scenario}/testsuites/{testSuiteName}/testcases/{testCaseName}': get: tags: - VTP Scenario summary: Retrieve test cases details like inputs outputs and test suite name description: '' operationId: getTestcase produces: - application/json parameters: - name: scenario in: path description: Test scenario name required: true type: string - name: testSuiteName in: path description: Test case name required: true type: string - name: testCaseName in: path description: Test case name required: true type: string responses: '200': description: successful operation schema: $ref: '#/definitions/VTPTestCase' '404': description: Test case does not exist schema: $ref: '#/definitions/VTPError' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' '/vtp/executions/{executionId}': get: tags: - VTP Execution summary: ' Retrieve test execution complete details' description: '' operationId: getTestExecution produces: - application/json parameters: - name: executionId in: path description: Test execution Id required: true type: string responses: '200': description: successful operation schema: $ref: '#/definitions/VTPTestExecution' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' /vtp/scenarios: get: tags: - VTP Scenario summary: ' List available test scenarios' description: '' operationId: listTestScenarios produces: - application/json parameters: [] responses: '200': description: successful operation schema: type: array items: $ref: '#/definitions/VTPTestScenario' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' /vtp/executions: get: tags: - VTP Execution summary: ' List test executions' description: '' operationId: listTestExecutions produces: - application/json parameters: - name: requestId in: query description: Test request Id required: false type: string - name: scenario in: query description: Test scenario name required: false type: string - name: testsuiteName in: query description: Test suite name required: false type: string - name: testcaseName in: query description: Test case name required: false type: string - name: profileName in: query description: Test profile name required: false type: string - name: startTime in: query description: Test execution start time required: false type: string - name: endTime in: query description: Test execution end time required: false type: string responses: '200': description: successful operation schema: type: array items: $ref: '#/definitions/VTPTestExecution' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' post: tags: - VTP Execution summary: 'Execute the test case with given inputs in ''executions'' form-data as key-value pair of parameter''s name vs parameter''s value. If parameter is binary type thenmulti-part form-data ''file'' should be used to feed the binary file content and it can be more than once. To use the given file as input parameter, prefix the value with file://<filename>.' description: '' operationId: executeTestcases1 consumes: - multipart/form-data produces: - application/json parameters: - name: requestId in: query description: Request Id required: false type: string - name: file in: formData description: Testcase File arguments required: false type: array items: $ref: '#/definitions/FormDataBodyPart' collectionFormat: multi - name: executions in: formData required: false type: string responses: '200': description: successful operation schema: type: array items: $ref: '#/definitions/VTPTestExecution' '500': description: Failed to perform the operation schema: $ref: '#/definitions/VTPError' definitions: VTPTestSuite: type: object properties: name: type: string VTPError: type: object properties: code: type: string message: type: string httpStatus: type: integer format: int32 VTPTestCase: type: object properties: scenario: type: string testCaseName: type: string testSuiteName: type: string descripton: type: string author: type: string inputs: type: array items: $ref: '#/definitions/VTPTestCaseInput' outputs: type: array items: $ref: '#/definitions/VTPTestCaseOutput' VTPTestCaseInput: type: object properties: name: type: string description: type: string type: type: string defaultValue: type: string isOptional: type: boolean metadata: type:object VTPTestCaseOutput: type: object properties: name: type: string description: type: string type: type: string VTPTestExecution: type: object properties: scenario: type: string testCaseName: type: string testSuiteName: type: string executionId: type: string requestId: type: string profile: type: string parameters: $ref: '#/definitions/JsonNode' results: $ref: '#/definitions/JsonNode' status: type: string startTime: type: string endTime: type: string VTPTestScenario: type: object properties: name: type: string

DRAFTs

VNFs (TBD)

GET on /vtp/v1/vnfs

GET on /vtp/v1/vnfs/vnf-id

...

PUT on /vtp/v1/vnfs/vnf-id/content - update vnf content

DELETE on /vtp/v1/vnf-id

Profiles/configurations (TBD)

GET on /vtp/v1/profiles?vnf-id=xxx

...