POC Test aspects

Notes

These test cases are for the POC. Only OOF-SDNR pairwsie test is part of integration testing for Casablanca and that has been competed.
Policy pairwise testing status for Casablanca is tracked at Policy Pair Wise Testing for Casablanca Release.
OOF testing status for Casablanca is tracked at https://wiki.onap.org/pages/viewpage.action?pageId=41420882.

Pair-wise Testing

1. Policy↔OOF Pair-wise testing

S.No.	Test case ID	Other component involved	Test case	Status	Remarks
1.	POLICY_OOF_PAIRWISE_001	OOF	(Initial) Configuration of OOF	Tested	Tested with templates for PCI based policies, there is an issue with policy portal to upload the config policies (https://jira.onap.org/browse/POLICY-1206)

2. Policy↔PCI-MS pairwise testing

S.No.	Test case ID	Other component involved	Test case	Status	Remarks
1.	POLICY_PCIMS_PAIRWISE_001	PCI-MS	(Initial) Configuration of PCI-MS	Tested
2.	POLICY_PCIMS_PAIRWISE_002	PCI-MS	Sending PCI changes to Policy (control loop)	Tested

3. Policy↔SDN-R pairwise testing

S.No.	Test case ID	Other component involved	Test case	Status	Remarks
1.	POLICY_SDNR_PAIRWISE_001	SDN-R	Sending PCI changes to SDN-R (control loop)	In Progress	SDN-R implementation to receive DMaaP message tbd
2.	POLICY_SDNR_PAIRWISE_002	SDN-R	Handling response to PCI change recommendation	Not tested	Dependent on previous test case

4. OOF↔SDN-R pairwise testing

S.No.	Test case ID	Components	Test case	Status	Remarks
1.	OOF_SDNR_PAIRWISE_001	OOF, SDN-R	Config DB API access (for getNbrList)	Tested	In Windriver lab against ConfigDB API, using simulated ConfigDB

5. OOF↔PCI-MS pairwise testing

S.No.	Test case ID	Components	Test case	Status	Remarks
1.	OOF_PCIMS_PAIRWISE_001	OOF, PCI-MS	Request for PCI optimization	Tested
2.	OOF_PCIMS_PAIRWISE_002	OOF, PCI-MS	PCI optimization result	Tested

6. SDN-R↔PCI-MS pairwise testing

S.No.	Test case ID	Components	Test case	Status	Remarks
1.	SDNR_PCIMS_PAIRWISE_001	SDN-R, PCI-MS	Handle neighbor list change notification from SDN-R	In progress	Generated a DMaaP notification and testing PCI-MS, real SDN-R not yet integrated
2.	SDNR_PCIMS_PAIRWISE_002	SDN-R, PCI-MS	Config DB API access (for getNbrList)	Tested
3.	SDNR_PCIMS_PAIRWISE_003	SDN-R, PCI-MS	Config DB API access (for pnfName)	Tested

7. SDN-R↔RAN pairwise testing

S.No.	Test case ID	Components	Test case	Status	Remarks
1.	SDNR_RAN_PAIRWISE_001	SDN-R, RAN (Sim)	Connecting SDN-R with RAN (Sim) (netconf servers)	In progress	Testing with a netconf client, not yet started with real SDN-R instance due to netconf notification handling issues
2.	SDNR_RAN_PAIRWISE_002	SDN-R, RAN (Sim)	Receiving neighbor list change notification from RAN	Not tested	SDN-R notification handling tbd
3.	SDNR_RAN_PAIRWISE_003	SDN-R, RAN (Sim)	Send netconf message(s) to RAN-Sim with the PCI changes	Not tested	SDN-R notification handling tbd
4.	SDNR_RAN_PAIRWISE_004	SDN-R, RAN (Sim)	Successful update of PCI values	Not tested	SDN-R notification handling tbd

Test Case ID

POLICY_OOF_PAIRWISE_001

Test Case Name

Configuration of OOF

Description

Test if the OOF is onboarded and configuration policies are fetched and loaded correctly (Step 1c of flow)

Release

Casablanca

Pre-conditions

(a) Policy module is onboarded.

(b) Config policies related to OOF are provisioned properly in Policy module.

Testing Steps

Steps

Expected Result

Bring up the OOF using the installation steps.
Check the OOF database to see the contents of the OOF config policy for PCI optimization use case.

OOF comes online and the database and local datastructures for PCI optimization use case are initialized correctly.
Config-policy is correctly fetched and stored in the OOF DB.
Sample config policy contents are given below:
ALGO_CATEGORY:OOF-PCI-OPTIMIZATION
PCI_OPTMIZATION_ALGO_NAME:OOF-PCI-OPTIMIZATION-LEVEL1
PCI_OPTIMIZATION_NW_CONSTRAINT:MAX5PCICHANGESONLY
PCI_OPTIMIZATION_PRIORITY: 2
PCI_OPTIMIZATION_TIME_CONSTRAINT:NONE

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

POLICY_PCIMS_PAIRWISE_001

Test Case Name

Configuration of PCI-MS

Description

Test if the PCI-MS is onboarded and configuration policies are fetched and loaded correctly (Step 1b of flow)

Release

Casablanca

Pre-conditions

(a) Policy module is onboarded.

(b) Config policies related to PCI-MS are provisioned properly in Policy module.

Testing Steps

Steps

Expected Result

Bring up the PCI-MS using the intallation steps.
Check the PCI-MS database to see the contents of the PCI-MS config policy.

PCI-MS comes online and the database and local datastructures (e.g., thread pool) are initialized correctly.
Config-policy is correctly fetched and stored in the PCI-MS DB.
Sample config policy contents are given below:
PCI_NEIGHBOR_CHANGE_CLUSTER_TIMEOUT_IN_SECS:60
PCI_MODCONFIG_POLICY_NAME:ControlLoop-vPCI-fb41f388-a5f2-11e8-98d0-529269fb1459 PCI_OPTMIZATION_ALGO_CATEGORY_IN_OOF:OOF-PCI-OPTIMIZATION
PCI_SDNR_TARGET_NAME:SDNR

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

POLICY_PCIMS_PAIRWISE_002

Test Case Name

Sending DMaaP message to Policy module with the PCI changes.

Description

Sending of DMaaP message (Topic: DCAE_EVENT_OUTPUT) by PCI-MS to Policy with the PCI changes

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network using the RAN initialization file.

(c) SDNR_PCIMS_PAIRWISE_003 has been completed (if not, write some stub code in PCI-MS to send the DMaaP message).

Testing Steps

Steps	Expected Result
PCI-MS prepares the DMaaP message (with payload layout corresponding to the yang model) to be sent to Policy. It sends 1 message for each PNF.	DMaaP message(s) is successfully received by Policy.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

POLICY_SDNR_PAIRWISE_001

Test Case Name

Invoking the control loop in Policy

Description

Invoking the control loop and sending DMaaP message to SDN-R with the PCI changes.

Release

Casablanca

Pre-conditions

(a) Control Loop Policy is successfully loaded and configured in Policy module.

(b) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(c) POLICY_PCIMS_PAIRWISE_002 has been completed.

Testing Steps

Steps	Expected Result
Policy executes the control loop for PCI use case, and prepares the DMaaP message (topic - SDNR-CL, and payload layout taken from the DMaaP message received from PCI-MS) to the yang model) to be sent to Policy. It sends 1 message for each PNF.	DMaaP message(s) is successfully received by SDN-R.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

POLICY_SDNR_PAIRWISE_002

Test Case Name

Acknowledgement of successful update of PCI values.

Description

RAN-Sim sending success response to netconf (edit-config) message(s) to SDN-R.

Release

Casablanca

Pre-conditions

(a) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(b) RAN-Sim is initialized and configured properly.

(d) POLICY_SDNR_PAIRWISE_001 has been completed (also preferably, SDNR_RAN_PAIRWISE_004 should be completed).

Testing Steps

Steps	Expected Result
SDN-R sends a DMaaP message (Topic: SDNR-CL-RSP) to Policy.	Policy receives the DMaaP message, logs the contents, and the Control Loop status can be viewed in the logs.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker M

Test Case ID

OOF_SDNR_PAIRWISE_001

Test Case Name

Config DB API access by OOF

Description

Accessing Config DB APIs by OOF for PCI optimization algorithm execution

Release

Casablanca

Pre-conditions

(a) OOF has fetched the config policies successfully from the Policy module.

(b) OOF_PCIMS_PAIRWISE_001 has been completed successfully (otherwise some dummy code for REST API call to Config DB to be written in OOF).

Testing Steps

Steps

Expected Result

After receiving a request for PCI optimization, OOF does a Config DB API query for getting the cellList (for a given networkID) and nbrList (for the set of cells sent in the PCI optimization request by PCI-MS).
API contents are available in SDN-R impacts document.

SDN-R accepts the API and sends a response with the requested details.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

OOF_PCI_PAIRWISE_001

Test Case Name

Request for PCI optimization

Description

Request for PCI optimization algorithm to be executed by OOF to address PCI collision/confusion

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network.

(c) SDNR_PCIMS_PAIRWISE_002 has been completed (otherwise some dummy code for REST API call to OOF to be written in PCI-MS).

Testing Steps

Steps

Expected Result

After receiving neighbor list change notification from SDN-R, PCI-MS processes it and determines OOF to be called.
PCI-MS invokes the OOF PCI API (https://{host}:{port}/api/oof/v1/pci) for executing the PCI optimization.
An example of the API contents is available in PCI Optimization API

OOF accepts the API request and sends a 200 OK response. An example of the (synchronous) response contents is available in PCI Optimization API

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

OOF_PCIMS_PAIRWISE_002

Test Case Name

PCI optimization result reported by OOF to PCI-MS

Description

PCI optimization result sent by OOF to PCI-MS via REST call.

Release

Casablanca

Pre-conditions

(a) OOF has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network.

(c) OOF_PCIMS_PAIRWISE_001 has been completed (otherwise some dummy code for REST API call to PCI-MS to be written in OOF).

Testing Steps

Steps	Expected Result
After executing the PCI optimization algorithm, OOF invokes the callback API provided by the PCI-MS in the PCI optimization request API call. This API contains the results of the PCI optimization, i.e., the cells whose PCI values should be modified. An example of the API contents is available in PCI Optimization API	PCI-MS accepts the API and sends a 200 OK response.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

SDNR_PCIMS_PAIRWISE_001

Test Case Name

Handle neighbor list change notification from SDN-R

Description

Handle neighbor list change notification from SDN-R and form a cell cluster.

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) SDNR_RAN_PAIRWISE_002 has been completed, and neighbor list change notification has been received by SDN-R.

Testing Steps

Steps	Expected Result
SDN-R sends a DMaaP message to PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO. DMaaP client in PCI-MS receives the message and forwards it to the PCI-MS core for further processing.	DMaaP message is received by Main thread of PCI-MS (verify by viewing logs).

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

SDNR_PCIMS_PAIRWISE_002

Test Case Name

Config DB API access

Description

Test the working of Config DB APIs

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network using the RAN initialization file.

(c) SDNR_PCIMS_PAIRWISE_001 has been completed (if not, write some stub code in PCI-MS to do the ConfigDB query after initialization).

Testing Steps

Steps	Expected Result
PCI-MS does a query to the configDB using the getNbrList API to get the neighbor list.	API query is successful and results match with DB contents. API contents are provided in SDN-R impacts document.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

SDNR_PCIMS_PAIRWISE_003

Test Case Name

Config DB API access

Description

Test the working of Config DB APIs (getPnfName)

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network using the RAN initialization file.

(c) OOF_PCIMS_PAIRWISE_002 has been completed (if not, write some stub code in PCI-MS to do the ConfigDB query).

Testing Steps

Steps	Expected Result
For the list of cells (whose PCI values should be changed) received from OOF in the PCI-optimization result, PCI-MS does a query to the configDB using the getPnfName for obtaining the PNF names for those cells whose PNF names are not available in its local DB.	API query is successful and results match with DB contents. API details are provided in SDN-R impacts document.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

SDNR_RAN_PAIRWISE_001

Test Case Name

Connecting SDN-R with RAN (netconf servers)

Description

Establish connection between SDN-R and RAN

Release

Casablanca

Pre-conditions

(a) SDN-R is up and running

(b) RAN initialization file is available

Testing Steps

Steps	Expected Result
Bring up the RAN-Simulator using the installation steps. Observe the connection status between SDN-R and netconf server(s)	RAN-Simulator is successfully instantiated. GUI also shows the appropriate topology view. SDN-R and netconf servers are connected (RAN-Simulator sends a mount request (REST) for each netconf server.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

SDNR_RAN_PAIRWISE_002

Test Case Name

Receiving neighbor list change notification from RAN

Description

Receive a neighbor list change notification from RAN and act upon it by sending a DMaaP notification to PCI-MS (after storing relevant info in Config DB).

Release

Casablanca

Pre-conditions

(a) SDN-R is up and running

(b) RAN is connected to SDN-R

(c) GUI shows RAN topology view properly, GUI is able to show the details of a particular cell (PCI, neighbor list, CellId).

Testing Steps

Steps	Expected Result
Modify the neighbor list of a cell (via RAN simulator GUI) The netconf server to which the cell is connected to sends a netconf notification to SDN-R.	SDN-R updates the Config DB (neighbor list of the cell). SDN-R sends a DMaaP message to the PCI-MS with the topic PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

SDNR_RAN_PAIRWISE_003

Test Case Name

Sending message(s) to RAN-Sim to update PCI values

Description

SDN-R sending netconf (edit-config) message(s) to RAN-Sim to update the PCI values

Release

Casablanca

Pre-conditions

(a) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(b) RAN-Sim is initialized and configured properly.

(c) POLICY_SDNR_PAIRWISE_001 has been completed.

(d) SDNR_RAN_PAIRWISE_002 has been completed.

Testing Steps

Steps	Expected Result
Upon reception of the DMaaP message from Policy (topic: SDNR-CL) with the PCI changes, SDN-R prepares a netconf message (edit-config) and sends it to the RAN-Sim.	RAN-Sim receives the message, and makes necessary updates to the PCI values (GUI reflects these changes).

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker M

Test Case ID

SDNR_RAN_PAIRWISE_004

Test Case Name

Acknowledgement of successful update of PCI values.

Description

RAN-Sim sending success response to netconf (edit-config) message(s) to SDN-R.

Release

Casablanca

Pre-conditions

(a) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(b) RAN-Sim is initialized and configured properly.

(d) SDNR_RAN_PAIRWISE_003 has been completed.

Testing Steps

Steps	Expected Result
RAN-Sim sends a success response to SDN-R upon receipt of netconf edit-config message from SDN-R. Upon receipt of this response, SDN-R prepares a DMaaP message (Topic: SDNR-CL-RSP)	Policy receives the DMaaP message, and the Control Loop status can be viewed in the logs.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker M

Integration Testing

Initialization (RAN-Sim, GUI, OOF, PCI-MS, Policy, SDN-R, Config DB)
Neighbor list change notification handling (via RAN-Sim GUI)
Successful application of PCI changes
Advanced scenarios
1. More than 1 notification required for triggering OOF
2. Neighbor list change when OOF is executing PCI optimization algorithm
3. Neighbor list change for more than 1 cluster (is this relevant for PoC?)
4. ???