Pair-wise Testing

1. Policy↔OOF Pair-wise testing

S.No.	Test case ID	ComponentsOther component involved	Test case	Status	Remarks
1.	POLICY_OOF_PAIRWISE_001	Policy, OOF	(Initial) Configuration of OOF	Not tested	2.	POLICY_PCIMS_PAIRWISE_001	Policy, PCI-MS		Tested	Tested with templates for PCI based policies, there is an issue with policy portal to upload the config policies ( https://jira.onap.org/browse/POLICY-1206)

2. Policy↔PCI-MS pairwise testing

S.No.	Test case ID	Other component involved	Test case	Status	Remarks
1.	POLICY_PCIMS_PAIRWISE_001	PCI-MS	(Initial) Configuration of PCI-MS

Not tested

In progress

3

2.

SDNR

POLICY_

RAN

PCIMS_PAIRWISE_

001

002

SDN-R, RAN (Sim)Connecting SDN-R with RAN (Sim) (netconf servers

PCI-MS

Sending PCI changes to Policy (control loop)

Not tested

4.SDNR_RAN_PAIRWISE_002SDN-R, RAN (Sim)

Receiving neighbor list change notification from RAN

Not tested5.SDNR_PCIMS_PAIRWISE_001SDN-R, PCI-MS

Handle neighbor list change notification from SDN-R

Not tested6.SDNR_PCIMS_PAIRWISE_002SDN-R, PCI-MS

3. Policy↔SDN-R pairwise testing

S.No.	Test case ID	Other component involved	Test case	Status	Remarks
1.	POLICY_SDNR_PAIRWISE_001	SDN-R	Sending PCI changes to SDN-R (control loop)	Not tested
2.	POLICY_SDNR_PAIRWISE_002	SDN-R	Handling response to PCI change recommendation	Not tested

4. OOF↔SDN-R pairwise testing

S.No.	Test case ID	Components	Test case	Status
1.	OOF_SDNR_PAIRWISE_001	OOF, SDN-R	Config DB API access (for getNbrList)	Not tested

7

5. OOF↔PCI-MS pairwise testing

S.No.	Test case ID	Components	Test case	Status
1.	OOF_PCIMS_PAIRWISE_001	OOF, PCI-MS	Request for PCI optimization	Not tested

8

2.

OOF_

SDNR_PAIRWISE_001OOF, SDN-RConfig DB API access (for getNbrList)Not tested9.OOF_

PCIMS_PAIRWISE_002

OOF, PCI-MS

PCI optimization result

Not tested

10.

6. SDN-R↔PCI-MS pairwise testing

S.No.	Test case ID	Components	Test case	Status
1.	SDNR_PCIMS_PAIRWISE_

003

001	SDN-R, PCI-MS	Handle neighbor list change notification from SDN-R	Not tested
2.	SDNR_PCIMS_PAIRWISE_002	SDN-R, PCI-MS	Config DB API access (for

pnfName

getNbrList)

Not tested

11

3.

POLICY

SDNR_PCIMS_PAIRWISE_

002

003

Policy

SDN-R, PCI-MS

Sending DMaaP message by PCI-MS to Policy

Config DB API access (for pnfName)

Not tested

12.POLICY_SDNR_PAIRWISE_001Policy, SDN-RInvoking control loop in PolicyNot tested13

7. SDN-R↔RAN pairwise testing

S.No.	Test case ID	Components	Test case	Status
1.	SDNR_RAN_PAIRWISE_

003

001	SDN-R, RAN (Sim)

Send netconf message(s) to RAN-Sim with the PCI changes

Connecting SDN-R with RAN (Sim) (netconf servers)

Not tested

14

2.	SDNR_RAN_PAIRWISE_

004

002	SDN-R, RAN (Sim)

Successful update of PCI valuesOOF

Receiving neighbor list change notification from RAN

Not tested

Test Case ID

AnchorPOLICY_

3.	SDNR_RAN_PAIRWISE_

001POLICY_OOF_PAIRWISE_001POLICY_OOF_PAIRWISE_001

Test Case Name

Configuration of OOF

Description

Test if the OOF is onboarded and configuration policies

003	SDN-R, RAN (Sim)	Send netconf message(s) to RAN-Sim with the PCI changes	Not tested
4.	SDNR_RAN_PAIRWISE_004	SDN-R, RAN (Sim)	Successful update of PCI values	Not tested

Test Case ID

 Anchor
POLICY_OOF_PAIRWISE_001
POLICY_OOF_PAIRWISE_001
POLICY_OOF_PAIRWISE_001

Test Case Name

Configuration of OOF

Description

Test if the OOF is onboarded and configuration policies are fetched and loaded correctly (Step 1c of flow)

Release

Casablanca

Pre-conditions

(a) Policy module is onboarded.

(b) Config policies related to OOF are provisioned properly in Policy module.

Testing Steps

Steps

Expected Result

Bring up the OOF using the installation steps.
Check the OOF database to see the contents of the OOF config policy for PCI optimization use case.

OOF comes online and the database and local datastructures for PCI optimization use case are initialized correctly.
Config-policy is correctly fetched and stored in the OOF DB.
Sample config policy contents are given below:
ALGO_CATEGORY:OOF-PCI-OPTIMIZATION
PCI_OPTMIZATION_ALGO_NAME:OOF-PCI-OPTIMIZATION-LEVEL1
PCI_OPTIMIZATION_NW_CONSTRAINT:MAX5PCICHANGESONLY
PCI_OPTIMIZATION_PRIORITY: 2
PCI_OPTIMIZATION_TIME_CONSTRAINT:NONE

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

...

Test Case ID

 Anchor
SDNR_PCIMS_PAIRWISE_002
SDNR_PCIMS_PAIRWISE_002
SDNR_PCIMS_PAIRWISE_002

Test Case Name

Config DB API access

Description

Test the working of Config DB APIs

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network using the RAN initialization file.

(c) SDNR_PCIMS_PAIRWISE_001 has been completed (if not, write some stub code in PCI-MS to do the ConfigDB query after initialization).

Testing Steps

Steps	Expected Result
PCI-MS does a query to the configDB using the getNbrList API to get the neighbor list.	API query is successful and results match with DB contents. API contents are provided in SDN-R impacts document.document.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
OOF_PCIMS_PAIRWISE_001
OOF_PCIMS_PAIRWISE_001
OOF_PCI_PAIRWISE_001

Test Case Name

Request for PCI optimization

Description

Request for PCI optimization algorithm to be executed by OOF to address PCI collision/confusion

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network.

(c) SDNR_PCIMS_PAIRWISE_002 has been completed (otherwise some dummy code for REST API call to OOF to be written in PCI-MS).

Testing Steps

Steps

Expected Result

After receiving neighbor list change notification from SDN-R, PCI-MS processes it and determines OOF to be called.
PCI-MS invokes the OOF PCI API (https://{host}:{port}/api/oof/v1/pci) for executing the PCI optimization.
An example of the API contents is available in PCI Optimization API

OOF accepts the API request and sends a 200 OK response. An example of the (synchronous) response contents is available in PCI Optimization API

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
OOF_PCIMSSDNR_PAIRWISE_001
OOF_PCIMSSDNR_PAIRWISE_001
OOF_PCISDNR_PAIRWISE_001

Test Case NameRequest for PCI optimization

Config DB API access by OOF

Description

Request Accessing Config DB APIs by OOF for PCI optimization algorithm to be executed by OOF to address PCI collision/confusionexecution

Release

Casablanca

Pre-conditions

(a) PCI-MS OOF has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network.(c) SDNR) OOF_PCIMS_PAIRWISE_002 001 has been completed successfully (otherwise some dummy code for REST API call to OOF Config DB to be written in PCI-MSOOF).

Testing Steps

Steps

Expected Result

After receiving neighbor list change notification from SDN-R, PCI-MS processes it and determines OOF to be called.
PCI-MS invokes the OOF PCI API (https://{host}:{port}/api/oof/v1/pci) for executing the PCI optimization.
An example of the API contents is available in PCI Optimization API

OOF accepts the API request and sends a 200 OK response. An example of the (synchronous) response contents is available in PCI Optimization API

a request for PCI optimization, OOF does a Config DB API query for getting the cellList (for a given networkID) and nbrList (for the set of cells sent in the PCI optimization request by PCI-MS).
API contents are available in SDN-R impacts document.

SDN-R accepts the API and sends a response with the requested details.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
OOF_SDNRPCIMS_PAIRWISE_001002
OOF_SDNRPCIMS_PAIRWISE_001002
OOF_SDNRPCIMS_PAIRWISE_001002

Test Case NameConfig DB API access

PCI optimization result reported by OOF to PCI-MS

DescriptionAccessing Config DB APIs

PCI optimization result sent by OOF for PCI optimization algorithm executionto PCI-MS via REST call.

Release

Casablanca

Pre-conditions

(a) OOF has fetched the config policies successfully from the Policy modulethe Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network.

(bc) OOF_PCIMS_PAIRWISE_001 has been completed successfully (otherwise some dummy code for REST API call to Config DB PCI-MS to be written in OOF).

Testing Steps

Steps

Expected Result

After receiving a request for executing the PCI optimization algorithm, OOF does a Config DB API query for getting the cellList (for a given networkID) and nbrList (for the set of cells sent in the PCI optimization request by PCI-MS).
API contents are available in SDN-R impacts document.

SDN-R

invokes the callback API provided by the PCI-MS in the PCI optimization request API call. This API contains the results of the PCI optimization, i.e., the cells whose PCI values should be modified.
An example of the API contents is available in PCI Optimization API

PCI-MS accepts the API and sends a 200 OK response with the requested details.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
OOFSDNR_PCIMS_PAIRWISE_002003OOF
SDNR_PCIMS_PAIRWISE_002003
OOFSDNR_PCIMS_PAIRWISE_002003

Test Case Name

PCI optimization result reported by OOF to PCI-MS

Description

PCI optimization result sent by OOF to PCI-MS via REST call.

Config DB API access

Description

Test the working of Config DB APIs (getPnfName)

Release

Casablanca

Pre-conditions

(a) OOF PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network using the RAN initialization file.

(c) OOF_PCIMS_PAIRWISE_001 002 has been completed (otherwise some dummy code for REST API call to if not, write some stub code in PCI-MS to be written in OOFdo the ConfigDB query).

Testing Steps

Steps

Expected Result

After executing For the list of cells (whose PCI values should be changed) received from OOF in the PCI-optimization algorithm, OOF invokes the callback API provided by the result, PCI-MS in the PCI optimization request API call. This API contains the results of the PCI optimization, i.e., the cells whose PCI values should be modified.
An example of the API contents is available in PCI Optimization API

PCI-MS accepts the API and sends a 200 OK response

does a query to the configDB using the getPnfName for obtaining the PNF names for those cells whose PNF names are not available in its local DB.

API query is successful and results match with DB contents. API details are provided in SDN-R impacts document.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
SDNRPOLICY_PCIMS_PAIRWISE_003002SDNR
POLICY_PCIMS_PAIRWISE_003002
SDNRPOLICY_PCIMS_PAIRWISE_003002

Test Case NameConfig DB API access

Sending DMaaP message to Policy module with the PCI changes.

DescriptionTest the working of Config DB APIs (getPnfName)

Sending of DMaaP message (Topic: DCAE_EVENT_OUTPUT) by PCI-MS to Policy with the PCI changes

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Policy module.

(b) (SDN-R) Config DB should be populated with the details of the cells in the network using the RAN initialization file.

(c) OOF SDNR_PCIMS_PAIRWISE_002003 has been completed (if not, write some stub code in PCI-MS to do send the ConfigDB queryDMaaP message).

Testing Steps

Steps

Expected Result

For the list of cells (whose PCI values should be changed) received from OOF in the PCI-optimization result, PCI-MS does a query to the configDB using the getPnfName for obtaining the PNF names for those cells whose PNF names are not available in its local DB.API query is successful and results match with DB contents. API details are provided in SDN-R impacts document-MS prepares the DMaaP message (with payload layout corresponding to the yang model) to be sent to Policy. It sends 1 message for each PNF.

DMaaP message(s) is successfully received by Policy.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
POLICY_PCIMSSDNR_PAIRWISE_002001
POLICY_PCIMSSDNR_PAIRWISE_002001
POLICY_PCIMSSDNR_PAIRWISE_002001

Test Case Name

Sending DMaaP message to Policy module with the PCI changes.Invoking the control loop in Policy

Description

Sending of DMaaP message (Topic: DCAE_EVENT_OUTPUT) by PCI-MS to Policy

Invoking the control loop and sending DMaaP message to SDN-R with the PCI changes.

Release

Casablanca

Pre-conditions

(a) PCI-MS has fetched the config policies successfully from the Control Loop Policy is successfully loaded and configured in Policy module.

(b) ( SDN-R ) Config DB should be populated with the details of the cells in the network using the RAN initialization fileis initialized, configured and connected to the RAN-Sim (netconf servers).

(c) SDNRPOLICY_PCIMS_PAIRWISE_003 002 has been completed (if not, write some stub code in PCI-MS to send the DMaaP message).

Testing Steps

Steps	Expected Result
PCI-MS Policy executes the control loop for PCI use case, and prepares the DMaaP message (with payload layout corresponding topic - SDNR-CL, and payload layout taken from the DMaaP message received from PCI-MS) to the yang model) to be sent to Policy. It sends 1 message for each PNF.	DMaaP message(s) is successfully received by PolicySDN-R.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker Machine

Test Case ID

 Anchor
POLICYSDNR_SDNRRAN_PAIRWISE_001003POLICY
SDNR_SDNRRAN_PAIRWISE_001003
POLICYSDNR_SDNRRAN_PAIRWISE_001003

Test Case Name

Invoking the control loop in Policy

Description

Invoking the control loop and sending DMaaP message to SDN-R with the PCI changes.Sending message(s) to RAN-Sim to update PCI values

Description

SDN-R sending netconf (edit-config) message(s) to RAN-Sim to update the PCI values

Release

Casablanca

Pre-conditions

(a) Control Loop Policy is successfully loaded and configured in Policy module.(b) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(b) RAN-Sim is initialized and configured properly.

(c) POLICY_PCIMS_SDNR_PAIRWISE_001 has been completed.

(d) SDNR_RAN_PAIRWISE_002 has been completed.

Testing Steps

Steps

Expected Result

Policy executes the control loop for PCI use case, and prepares Upon reception of the DMaaP message from Policy (topic - : SDNR-CL, and payload layout taken from the DMaaP message received from PCI-MS) to the yang model) to be sent to Policy. It sends 1 message for each PNF. SDNR-CLDMaaP message(s) is successfully received by SDN-R) with the PCI changes, SDN-R prepares a netconf message (edit-config) and sends it to the RAN-Sim.

RAN-Sim receives the message, and makes necessary updates to the PCI values (GUI reflects these changes).

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker MachineM

Test Case ID

 Anchor
SDNR_RAN_PAIRWISE_003004
SDNR_RAN_PAIRWISE_003004
SDNR_RAN_PAIRWISE_003004

Test Case NameSending message(s) to RAN-Sim to update

Acknowledgement of successful update of PCI values.

Description

SDNRAN-R Sim sending success response to netconf (edit-config) message(s) to RAN-Sim to update the PCI valuesSDN-R.

Release

Casablanca

Pre-conditions

(a) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(b) RAN-Sim is initialized and configured properly.(c) POLICY_SDNR_PAIRWISE_001 has been completed.

(d) SDNR_RAN_PAIRWISE_002 003 has been completed.

Testing Steps

Steps	Expected Result
Policy executes the control loop for PCI use case, and prepares the DMaaP message (topic - SDNR-CL, and payload layout taken from the DMaaP message received from PCI-MS) to the yang model) to be sent to Policy. It sends 1 message for each PNF. SDNR-CL	RAN-Sim receives the message, and makes necessary updates to the PCI values (GUI reflects these changes)RAN-Sim sends a success response to SDN-R upon receipt of netconf edit-config message from SDN-R. Upon receipt of this response, SDN-R prepares a DMaaP message (Topic: SDNR-CL-RSP)	Policy receives the DMaaP message, and the Control Loop status can be viewed in the logs.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker M

Test Case ID

 Anchor
SDNR_RAN_PAIRWISE_004
SDNR_RAN_PAIRWISE_004
SDNR_RAN_PAIRWISE_004

Test Case Name

Acknowledgement of successful update of PCI values.

Description

RAN-Sim sending success response to netconf (edit-config) message(s) to SDN-R.

Release

Casablanca

Pre-conditions

(a) SDN-R is initialized, configured and connected to the RAN-Sim (netconf servers).

(b) RAN-Sim is initialized and configured properly.

(d) SDNR_RAN_PAIRWISE_003 has been completed.

Testing Steps

Steps	Expected Result
RAN-Sim sends a success response to SDN-R upon receipt of netconf edit-config message from SDN-R. Upon receipt of this response, SDN-R prepares a DMaaP message (Topic: SDNR-CL-RSP)	Policy receives the DMaaP message, and the Control Loop status can be viewed in the logs.

Conclusion (Pass /Fail)

Testing Lab

https://jenkins.onap.org/view/CSIT/ Ubuntu Docker M

Integration Testing

Initialization (RAN-Sim, GUI, OOF, PCI-MS, Policy, SDN-R, Config DB)
Neighbor list change (via RAN-Sim GUI)
Application of PCI changes
Advance scenarios
1. More than 1 notification required for triggering OOF
2. Neighbor list change when OOF is executing PCI optimization algorithm
3. Neighbor list change for more than 1 cluster (is this relevant for PoC?)
4. ???

...

Version	Old Version 22	New Version 23
Changes made by	Swaminathan Seetharaman	Swaminathan Seetharaman
Saved on	Oct 16, 2018	Oct 25, 2018

Versions Compared

Key

Pair-wise Testing

1. Policy↔OOF Pair-wise testing

2. Policy↔PCI-MS pairwise testing

3. Policy↔SDN-R pairwise testing

4. OOF↔SDN-R pairwise testing

5. OOF↔PCI-MS pairwise testing

6. SDN-R↔PCI-MS pairwise testing

7. SDN-R↔RAN pairwise testing

Integration Testing

Content Comparison

Versions Compared

Key

Pair-wise Testing

1. Policy↔OOF Pair-wise testing

2. Policy↔PCI-MS pairwise testing

3. Policy↔SDN-R pairwise testing

4. OOF↔SDN-R pairwise testing

5. OOF↔PCI-MS pairwise testing

6. SDN-R↔PCI-MS pairwise testing

7. SDN-R↔RAN pairwise testing

Integration Testing