Test Results @CMCC lab
Image Added
Category: External System Registration
V00001: WindRiver OpenStack VIM Resource OnboardingRegistration
Test Case Id | V00001 |
Test Case Name | WindRiver OpenStack VIM |
Resource OnboardingRegistration |
Description | WindRiver OpenStack VIM |
Resource OnboardingRegistration |
Release | Amsterdam |
Preconditions | - The WindRiver OpenStack VIM is configured with tenant information and provider networks
- The WindRiver OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
|
Testing Steps | - Add VIM from ONAP GUI with VIM location information, tenant authentication information
- VIM default service url can be configured based on real VIM settings
|
Expected Results | - VIM
|
resource onboarded - registered successfully
- Tenant authentication information is stored
|
successfully- correctly
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab |
|
Tester Name |
|
V00002: WindRiver VMWare OpenStack 2nd VIM Resource OnboardingRegistration
Test Case Id | V00002 |
Test Case Name |
WindRiver 2nd Resource OnboardingThis is to test multi VIM supportVMWare OpenStack VIM Registration |
Release | Amsterdam |
Preconditions | - The
|
first WindRiver OpenStack VIM has been successfully onboardedA new WindRiver - VMWare OpenStack VIM is configured with tenant information and provider networks
- The
|
new WindRiver - VMWare OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
|
Testing Steps | - Add
|
the new - VIM from ONAP GUI with VIM location information, tenant authentication information
|
The new - VIM default service url can be configured based on
|
the new - real VIM settings
|
Expected Results |
The new resource onboarded The new VIM - Tenant authentication information is stored
|
successfully- correctly
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab |
|
Tester Name |
|
V00003: VMWare OpenStack VIM Resource OnboardingData Center Gateway Controller Registration
Description | VMWare OpenStack VIM Resource Onboarding |
Release | Preconditions | - The VMWare OpenStack VIM is configured with tenant information and provider networks
- The VMWare OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
|
Testing Steps | - Add VIM from ONAP GUI with VIM location information, tenant authentication information
- VIM default service url can be configured based on real VIM settings
|
Expected Results | VIM resource is onboarded successfullyTenant authentication information is stored successfullyTest Case Id | V00003 |
Test Case Name |
VMWare OpenStack VIM Resource Onboarding | Data Center Gateway Controller Registration |
Description | This is part of WAN setup testing. Each data center has its own gateway and controlled by local SDN controller |
Release | Amsterdam |
Preconditions | - Each DC gateway has been manually installed and configured correctly
- Each DC gateway local controller has been manually installed and configured correctly
|
Testing Steps | - Register each DC gateway local controller from ONAP ESR GUI with the needed information (location, IP, port, authentication information, etc)
|
Expected Results | - The DC local controller is registered successfully
- The DC local controller information is stored correctly
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Tester Name |
|
V00004: VMWare OpenStack 2nd VIM Resource OnboardingSDN WAN Controller Registration
Test Case Id | V00004 |
Test Case Name |
VMWare OpenStack 2nd VIM Resource OnboardingSDN WAN Controller Registration |
Description | This is |
to test multi VIM supportRelease | Preconditions | - The first VMWare OpenStack VIM has been successfully onboarded
- A new WindRiver OpenStack VIM is configured with tenant information and provider networks
- The new WindRiver OpenStack VIM services (console, auth, neutron, image, glance, etc) are accessible from ONAP by IP
|
Testing Steps | - Add the new VIM from ONAP GUI with VIM location information, tenant authentication information
- The new VIM default service url can be configured based on the new VIM settings
|
Expected Results | The new VIM resource is onboarded successfullyThe new VIM Tenant authentication information is stored successfullypart of WAN setup testing. In this test, all PEs in the network are controlled by the same SDN WAN controller |
Release | Amsterdam |
Preconditions | - Each PE router has been manually installed and configured correctly
- SDN WAN controller has been manually installed and configured correctly
- All PEs are managed by SDN WAN Controller
|
Testing Steps | - Register SDN WAN controller from ONAP ESR GUI with the needed information (IP, port, authentication information, etc)
|
Expected Results | - The SDN WAN controller is registered successfully
- SDN WAN controller information is stored correctly
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Tester Name |
|
V00005: Data Center Gateway Resource Onboarding SVNFM Registration
Test Case Id | V00005 |
Test Case Name |
Data Center Gateway Resource Onboarding | Description | This is part of WAN setup testing. Each data center has its own gateway and controlled by local SDN controllerEach DC gateway has been manually Vendors' SVNFMs Registration |
Description | Register all VNF vendors' SVNFMs |
Release | Amsterdam |
Preconditions |
VIMs from the same vendor have been successfully onboarded- All VNF vendors' SVNFMs have been installed and configured
|
correctlyEach DC gateway local controller has been manually installed and configured correctlyOnboard each DC gateway local controller from ONAP GUI (need to confirm which ONAP GUI supports SDN local controller - OOM or UsecaseUI?) The DC local controller - Register all VNF vendors' SVNFMs using ONAP ESR GUI with the needed information (IP, port, authentication information, etc)
|
Expected Results |
The DC local controller is onboarded successfully- All SVNFMs are registered successfully with ONAP
- All SVNFM information is stored
|
successfully- correctly in ONAP
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Tester Name |
|
V00006: WAN PE Resource Onboarding EMS Registration
Test Case Id | V00006 |
Test Case Name |
WAN PE Resource Onboarding | Description | This is part of WAN setup testing. There are at least two PEs in the network, controlled by one SDN WAN controllerVendors' EMSs Registration |
Description | Register all VNF vendors' EMSs |
Release | Amsterdam |
Preconditions |
Each PE router has been manually - All VNF vendor's EMS have been installed and configured
|
correctlySDN WAN controller has been manually installed and configured correctlyOnboard SDN WAN controller from ONAP GUI (need to confirm which ONAP GUI supports SDN local controller - OOM or UsecaseUI?) - Register all vendors' EMSs using ONAP ESR GUI with the needed information (IP, port, authentication information, etc)
|
Expected Results |
The SDN WAN controller is onboarded successfully- All EMSs are registered successfully with ONAP
- All EMSs information is stored correctly in ONAP
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Tester Name |
|
Category: VNF Onboarding and Service Creation
V00007: VoLTE VNFs Onboarding
Test Case Id | V00007 |
Test Case Name | VoLTE VNFs Onboarding |
Description | VNFD csar files need to be onboarded one by one |
Release | Amsterdam |
Preconditions | - All vEPC and vIMS VNFD csar files can pass VNFSDK checking
- VNF license has been created
|
Testing Steps | - Use SDC to
|
onboard - import, test and certify the following vEPC VNFD csar files one by one:
- vSPGW
- vEPDG
- vPCRF
- vHSS
- vMME
- Use SDC to
|
onboard - import, test and certify the following vIMS VNFD csar files one by one:
- vPCSCF
- vI/SCSCF
- vTAS
|
Expected Results | - vEPC
|
csar files - VNFs are onboarded successfully
- vIMS
|
csar files - VNFs are onboarded successfully
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
V00008: vEPC Service Creation
Test Case Id | V00008 |
Test Case Name | vEPC Service Creation |
Description | This creates vEPC service |
Release | Amsterdam |
Preconditions | - All vEPC VNFD csar files can pass VNFSDK checking
- All vEPC VNFs have been onboarded
- All VNFs images files have been uploaded manually into
|
ONAP- VIM
|
Testing Steps | - Use SDC to create vEPC service with the following VNFs
- vSPGW
- vEPDG
- vPCRF
- vHSS
- vMME
- As part of vEPC service composition, create Virtual Links (VL) to connect the VNFs together according to vEPC service design
|
Expected Results | - vEPC service is created successfully
- vnfmInfo is correctly pointed to vendor's SVNFM
- vEPC service is successfully stored in catalog
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
V00009: vIMS Service Creation
Test Case Id | V00009 |
Test Case Name | vIMS Service Creation |
Description | This test creates vIMS service |
which includes VNFs and Network Service |
Release | Amsterdam |
Preconditions | - All vIMS
|
VNFs - VNFD csar files can pass VNFSDK checking
- All vIMS VNFs have been onboarded
- All VNFs images files have been uploaded manually into ONAP
|
Testing Steps | - In SDC create vIMS service with the following VNFs
- vPCSCF
- vI/SCSCF
- vTAS
- As part of vIMS service composition, create Virtual Links (VL) to connect the VNFs together according to vIMS service design
|
Expected Results | - vIMS service is created successfully
- vnfmInfo is correctly pointed to vendor's SVNFM
- vIMS service is successfully stored in catalog
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
V00010: WAN Resource Onboarding (Underlay and Overlay templates are preloaded into SDC, this test case can be removed if not needed anymore)
Test Case Id | V00010 |
Test Case Name | WAN Resource Onboarding |
Description | This test onboards WAN resource using SDC |
Release | Amsterdam |
Preconditions | - WAN underlay network template (.yaml) has been defined and packaged in a CSAR file
- WAN overlay network template (.yaml) has been defined and packaged in a CSAR file
|
Testing Steps | - In SDC create a VL resource and upload WAN underlay network CSAR file. Make sure controllerInfo property is set to SDNC in the template
- In SDC create a VL resource and upload WAN overlay network CSAR file. Make sure controllerInfo property is set to SDNC
|
Expected Results | - WAN underlay network resource onboarded successfully and shown in catalog
- WAN overlay network resource onboarded successfully and shown in catalog
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
- VoLTE e2e service creation: Network, workflow, alarms, DGs are designed or uploaded. VoLTE e2e network service, including intra data center and inter data center connectivity, are designed successfully.
- Close loop design: DCAE template, Holmes correlation rules and operational policy are designed or uploaded, and distributed by CLAMP successfully
- Instantiation: VNFs are deployed in edge and core data centers successfully. Intra data center network, underlay and overlay WAN network cross the two data centers are setup correctly
- Auto-healing for VIM event: DCAE VES collector receives the VNF event from VIM, and forwards it to Holmes. Holmes does the correlation and generates signature event and sends to Policy. Then auto-healing policy is triggered and proper actions are taken to mitigate the situation
- Auto-healing for VNF event: When alarm is generated by VNF, VFC will receive the alarm via vendor’s EMS and forwards to DCAE VES collector. The DCAE collector will then forward alarm to Holmes via DMaaP. Holmes does the alarm correlation and sends signature event to Policy. Finally the auto-healing policy is triggered and proper actions are taken to mitigate the situation
...
V00011: VoLTE Service Creation
Test Case Id | V00011 |
Test Case Name | VoLTE Service Creation |
Description | This test creates VoLTE service. To workaround SDC limitation on nested service support, in R1 we create separate dummy VNFs in VoLTE service, and link those VNFs to vEPC, vIMS and WAN services |
Release | Amsterdam |
Preconditions | - vEPC service has been created
- vIMS service has been created
- WAN underlay and overlay resources have been onboarded
- All required ONAP artifacts (DG, DCAE template, Holmes blueprint, SO workflow, etc) have been designed and uploaded to corresponding components
|
Testing Steps | - In SDC onboard a dummy VNF, called EPC, which links its property nsServiceInvariantUUID to vEPC service property invariantUUID. Set the VNF controllerInfo property to VFC
- Onboard a dummy VNF, called IMS, which links its property nsServiceInvariantUUID to vIMS service property invariantUUID. Set the VNF controllerInfo property to VFC
- Create a new service called VoLTE service and compose it with EPC VNF, IMS VNF and WAN underlay and overlay VL resources
- Test, certify and approve VoLTE service
- Distribute VoLTE service
|
Expected Results | - VoLTE service is created successfully
- VoLTE service is stored in catalog
- Service Creation event is sent via DMaaP
- Other modules (SO, VID, Usecase UI, CLAMP, VID, etc) receives the event
- VoLTE service CSAR file can be retrieved from SDC catalog by other modules using SDC API
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
V00012: Closed Loop Configuration
Test Case Id | V00012 |
Test Case Name | Closed loop configuration |
Description | This test uses CLAMP to configure closed loop for VoLTE. There are 4 configurations: - Provisional rules for Policy
- Correlation rules for Holmes
- Holmes microservice Blueprint for DCAE
- VES configuration (in R1, there is no configuration needed)
|
Release | Amsterdam |
Preconditions | - DCAE template has been imported (R1 SDC doesn't support DCAE template design)
- Holmes blueprint has been imported
- Holmes correlation rules have been defined
|
Testing Steps | - In CLAMP, choose VNF SPGW and events (VNF Fault and VM Fault) to monitor
- Bring up CLAMP configuration policy GUI and copy Holmes correlation rules in it
- Bring up CLAMP provisional policy GUI and define healing provisional policy
- Save the configuration
|
Expected Results | - Closed loop template and blueprint for Holmes are distributed to DCAE
- Configuration policy has been distributed to DCAE
- Provisional policy has been distributed to Policy engine
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
Category: Service Instantiation and Monitoring
V00013: VoLTE Service Instantiation
Test Case Id | V00013 |
Test Case Name | VoLTE Service Instantiation |
Description | This test covers the whole VoLTE service instantiation process |
Release | Amsterdam |
Preconditions | - All external systems (VIM, SDN WAN controller, SDN local controller, vendors SVNFM and EMS) have registered with ESR
- VoLTE service has been created, V00011 has passed
- Closed loop has been configured, V00012 has passed
- PEs have been configured and integrated with corresponding SDN WAN controller
- DC GWs have been configured and integrated with corresponding SDN local controller
- EVPN VXLAN tunnel has been set up manually between edge and core DC gateway. At this moment, underlay network between PEs has not been set up yet, so EVPN and VXLAN tunnel is configured but may not be working, but it's ok.
- Install a web server in core data center
|
Testing Steps | - ONAP user logins Usecase UI GUI
- Usecase UI will guide the user to fill in or verify all needed parameters for vEPC, vIMS, WAN underlay and overlay
- User will click a button to deploy VoLTE service
|
Expected Results | - vEPC service is instantiated successfully, including all vEPC VNFs and VLs
- vIMS service is instantiated successfully, including all vIMS VNFs and VLs
- WAN service is instantiated successfully, including underlay and overlay networks
- Holmes analytics container is brought up correctly by DCAE
- Client UE can access network through mobile network, e.g. access the web server
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
V00014: System Performance and Alarm Monitoring
Test Case Id | V00014 |
Test Case Name | System Performance and Alarm Monitoring |
Description | This test uses Usecase UI GUI to monitor system performance and event |
Release | Amsterdam |
Preconditions | - VoLTE service has been instantiated
- Closed loop for VoLTE has been configured and started
|
Testing Steps | - ONAP user uses Usecase UI GUI to check system performance, such as CPU, memory and disk usage
- ONAP user views the VNF alarms collected by ONAP
|
Expected Results | - System performance (including system CPU, memory and disk usage) is shown on ONAP Usecase UI GUI
- VNF alarms, specified in CLAMP for the closed loop, are shown in Usecase UI GUI
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
Category: Closed Loop
V00015: VoLTE Auto-healing
Test Case Id | V00015 |
Test Case Name | VoLTE Auto-healing |
Description | This test covers the closed loop with alarm correlation and auto-healing |
Release | Amsterdam |
Preconditions | - VoLTE service has been instantiated
- VoLTE closed loop has been configured and started
- SPGW VNF supports event generation
- VNF guest OS supports event generation
- MultiVIM supports event collection from guest OS and event forwarding
|
Testing Steps | - Manually shutdown standby MPU VM inside SPGW VNF. This should trigger the VNF and VIM to generate alarms
|
Expected Results | - EMS collects standby MPU offline event from SPGW VNF and sends the event to VFC
- MultiVIM doesn't receive heartbeat event from standby MPU guest OS for a period of time and triggers alarm which is sent to DCAE VES collector
- Holmes receives MPU offline events
- Holmes receives MPU guest OS heartbeat missing events
- Holmes does correlation of the two types of events and generate one signature event and sends it to Policy
- Policy triggers provisional policy and healing action is sent to VFC
- VFC uses SVNFM driver to heal standby MPU VM
- Standby MPU is healed
- Clear event is sent from SPGW VNF about standby MPU back online
- Heartbeat from standby MPU VM is received by MultiVIM and clear event is sent by MultiVIM about heartbeat recovery
- Holmes does the event correlation and sends Clear signature event to Policy
- Policy stops the healing action
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|
Category: Service Termination
V00016: VoLTE Service Termination
Test Case Id | V00016 |
Test Case Name | VoLTE Service Termination |
Description | This test covers VoLTE service termination process |
Release | Amsterdam |
Preconditions | - VoLTE service has been created
- Closed loop has been configured and started
- PEs have been configured and integrated with corresponding SDN WAN controller
- DC GWs have been configured and integrated with corresponding SDN local controller
|
Testing Steps | - ONAP user uses Usecase UI GUI to remove VoLTE service
|
Expected Results | - Closed loop is stopped
- vEPC service is removed successfully, including all vEPC VNFs and VLs
- vIMS service is removed successfully, including all vIMS VNFs and VLs
- WAN service is removed successfully, including underlay and overlay network setup
- Holmes analytics container is removed by DCAE
- Closed loop related artifacts (including DCAE template, configurational policy and provisional policy, DG, workflows) are removed from ONAP
|
Actual Results |
|
Conclusion (Pass/Fail) |
|
Testing Lab | CMCC OpenLab |
Testing Date |
|
Tester Name |
|