Integration Testing

Note: The OOF-PCI use case integration testing is being done at Windriver lab.

Note: DMaaP should be setup for most of the test cases below.

#	Component(s)	Test Case	Status	Remarks
IT#001	DCAE (SON Handler MS)	SON Handler Micro-service successfully on-boarded on to DCAE	COMPLETE
IT#002	DCAE (SON Handler MS)	SON Handler Micro-service's DB is up and the Micro-service is able to read/write data.	COMPLETE
IT#003	DCAE (SON Handler MS)	SON Handler Micro-service is able to successfully fetch config policies from Consul.	COMPLETE
IT#004	OOF and Policy	OOF is able to successfully fetch config policy from Policy	NOT APPLICABLE	This test case is no longer applicable, as there is no policy-driven actions in OOF for this use case in Dublin.
IT#005	DCAE (SON Handler MS)	SON Handler Micro-service is able to successfully receive a message over DMaaP containing PM/FM data (from VES Collector)	COMPLETE
IT#006	DCAE (SON Handler MS) and DMaaP	SON Handler Micro-service is able to successfully receive a message over DMaaP containing neighbor list update	COMPLETE
IT#007	DCAE (SON Handler MS) REST interface	SON Handler Micro-service is able to successfully fetch neighbor list details using REST interface (Config DB APIs)	COMPLETE
IT#008	DCAE (SON Handler MS) and OOF	SON Handler Micro-service invokes REST API of OOF for PCI optimization	COMPLETE	DCAEGEN2-1529 - Getting issue details... STATUS OPTFRA-505 - Getting issue details... STATUS
IT#009	OOF and Config DB (REST API)	OOF is able to obtain relevant neighbor list information for optimization algorithm from ConfigDB (REST API)	COMPLETE	OPTFRA-508 - Getting issue details... STATUS
IT#010	DCAE (SON Handler MS) and OOF	SON Handler Micro-service receives optimization result from OOF (via callback URL) and sends response	COMPLETE	OPTFRA-508 - Getting issue details... STATUS
IT#011	DCAE (SON Handler MS) and Config DB (REST API)	SON Handler Micro-service is able to successfully fetch PNF details from Config DB (using REST API call)	COMPLETE
IT#012	DCAE (SON Handler MS) and Policy	SON Handler Micro-service is able to send PCI/ANR updates via DMaaP to Policy	COMPLETE	DCAEGEN2-1529 - Getting issue details... STATUS
IT#013	Policy and DMaaP	Policy is able to successfully send a DMaaP message (to SDN-R) with PCI/neighbor list updates.	COMPLETE
IT#014	Policy and DMaaP	Policy is able to successfully receive a DMaaP message with PCI/neighbor list update response	COMPLETE
IT#015	DCAE (SON Handler MS) and Policy	SON Handler Micro-service is able to receive PCI/neighbor list update response via DMaaP from Policy, process it and logs the results.	COMPLETE
IT#016	DCAE (SON Handler MS)	SON Handler Micro-service is able to process PM KPI data, store as needed, and apply rules to recommend removal of a neighbor relationship.	COMPLETE
IT#017	SDN-R	Upon receiving message from Policy, SDN-R sends netconf message to RAN-Sim.	PROGRESSING pending completion of 2 known issues (Medium): CCSDK-1399 and CCSDK-1400	Will be completed when CCSDK-1399 and CCSDK-1400 are delivered. CCSDK-1364 - Getting issue details... STATUS SDNC-772 - Getting issue details... STATUS The DMaaP listener docker with the code fix is not yet deployed in SB-00 where the testing is done. The code, fix however, is already merged on to Dublin and Master branches. So, once the latest docker is deployed, we will continue and complete the test case at Windriver. We submitted additional bug fixes and hope that the new Jenkins release would be deployed in the integration labs for testing Jira: CCSDK-1399 https://jira.onap.org/browse/CCSDK-1399 Code commit: https://gerrit.onap.org/r/c/ccsdk/sli/northbound/+/89792 One bug/issue is still under investigation and has been reported onto the SDNR distribution list. https://lists.onap.org/g/onap-SDNR/message/386 Jira: CCSDK-1400 (https://jira.onap.org/browse/CCSDK-1400)
IT#018	DCAE (VES Collector)	VES Collector receives VES messages from RAN-Simulator	COMPLETE
IT#019	SDNR (DMAAP Listener)	SDNR is able to receive DMAAP message from Policy
IT#020	SDNR	SDNR is able to receive a DMAAP message to configure PCI value (Action: ModifyConfig) and invoke RPC: configure-phy-cell-id		Prerequisite: RAN simulator is configured and Netconf mounted onto SDNR
IT#021	SDNR	SDNR is able to receive a DMAAP message to configure PCI value and/or blacklisted flag of neigbor cell/s (Action: ModifyConfigANR) and invoke RPC: generic-neighbor-configuration		Prerequisite: RAN simulator is configured and Netconf mounted onto SDNR
IT#022	SDNR & RAN simulator	RPC execution (configure-phy-cell-id): Verify if Restconf transactions against the Netconf mounted RAN simulator succeed in configuring PCI value of a cell		This can be verified by reviewing either SDNR/DG logs (/opt/opendaylight/current/data/log/configuration-phy-cell-id.log) or executing GET Restconf transaction from the SDNR Restconf API explorer
IT#023	SDNR & RAN simulator	RPC execution (generic-neighbor-configuration): Verify if Restconf transactions against the Netconf mounted RAN simulator succeed in configuring PCI value and/or blacklisted flag of neighbor cell/s		This can be verified by reviewing either SDNR/DG logs (/opt/opendaylight/current/data/log/generic-neighbor-config.log) or executing GET Restconf transaction from the SDNR Restconf API explorer
IT#024	SDNR & RAN simulator	RAN simulator generates Netconf notification to indicate neighbor list change notification		Prerequisite: RAN simulator is configured and Netconf mounted onto SDNR
IT#025	SDNR & RAN simulator	SDNR after receiving Netconf notification invokes RPC handle-nbrlist-change-notif		Prerequisite: RAN simulator is configured and Netconf mounted onto SDNR
IT#026	SDNR & ConfigDB	RPC execution (handle-nbrlist-change-notif): Update configDB by adding new neighbors per NETRCONF notification		Verify ConfigDB is updated by executing GET transaction from Postman Verify neighbor additions are aligned with neighbor list change Netconf notification
IT#027	SDNR/DMAAP message	RPC execution (handle-nbrlist-change-notif): DMAAP message ( PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO) is published from SDNR to be consumed by SON MS		Verify DMAAP message is published onto message router Verify payload of the DMAAP message aligns with the original neighbor list change Netconf notification

Test Case ID

IT#001

Test Case Name

SON Handler Micro-service on-boarding

Description

SON Handler Micro-service is successfully on-boarded on to DCAE

Release

Dublin

Pre-conditions

(a) DCAE up and running

(b) DCAE CLI available

Testing Steps

Steps	Expected Result
SON-Handler component spec validated. SON-Handler is brought up manually on DCAE. SON-Handler Blueprint validated successfully. DCAE is brought online.	SON-Handler comes up online and is stable.

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#002

Test Case Name

SON-Handler Micro-service DB on-boarding

Description

SON-Handler Micro-service's PG DB is up, and SON-Handler Micro-service is able to write data e.g., local configuration data and read it.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is up and running

Testing Steps

Steps	Expected Result
SON-Handler component and its DB is on-boarded on to DCAE. Local configurable attributes are written/read by SON-Handler MS (verified by viewing logs)	Verify DB working by viewing SON-Handler MS logs.

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#003

Test Case Name

Policy fetch by SON-Handler MS

Description

SON Handler Micro-service is able to successfully fetch config policies from Consul.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is up and running

(b) DCAE Platform including Consul is up and running

Testing Steps

Steps	Expected Result
Upload config policies for SON-Handler MS. SON-Handler MS should fetch the policies from Consul	Verify Policy fetch and store via logs and checking in PG DB

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#004

Test Case Name

Policy fetch by OOF

Description

OOF is able to successfully fetch the config policies from Policy

Release

Dublin

Pre-conditions

(a) Policy component is up and running

(b) Config policies related to OOF are provisioned in Policy component

Testing Steps

Steps

Expected Result

Bring up the OOF using the installation steps.
Check the OOF database to see the contents of the OOF config policy for PCI optimization use case (i.e., OOF should invoke Policy API to fetch the config policies).

OOF comes online and the database and local datastructures for PCI optimization use case are initialized correctly.
Config-policy is correctly fetched and stored in the OOF DB. Sample config policy contents are given below:
ALGO_CATEGORY:OOF-PCI-OPTIMIZATION
PCI_OPTMIZATION_ALGO_NAME:OOF-PCI-OPTIMIZATION-LEVEL1
PCI_OPTIMIZATION_NW_CONSTRAINT:MAX5PCICHANGESONLY
PCI_OPTIMIZATION_PRIORITY: 2
PCI_OPTIMIZATION_TIME_CONSTRAINT:NONE

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#005

Test Case Name

VES (DMaaP) interaction with SON-Handler MS

Description

SON Handler Micro-service is able to successfully receive a message over DMaaP containing PM/FM data (from VES Collector).

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running, VES is online

(b) SON-Handler MS is online

(c) DMaaP is up

Testing Steps

Steps	Expected Result
Ensure relevant DMaaP topics are created (SEC_FAULT_OUTPUT, SEC_MEASUREMENT_OUTPUT), along with producer and consumers (using unauthenticated mode) Post a fault message (REST) to VES Collector. If feasible, use the RAN-Simulator to post the fault message. Post a performance measurements message (REST) to VES Collector. If feasible, use the RAN-Simulator to post the fault message.	VES Collector posts the received fault message on to DMaaP using the topic SEC_FAULT_OUTPUT. SON-Handler MS receives the DMaaP SEC_FAULT_OUTPUT message and processes it - verified via logs. VES Collector posts the received PM message on to DMaaP using the topic SEC_MEASUREMENT_OUTPUT. SON-Handler MS receives the DMaaP SEC_MEASUREMENT_OUTPUT message and processes it - verified via logs.

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#006

Test Case Name

DMaaP interaction with SON-Handler MS

Description

SON Handler Micro-service is able to successfully receive a message over DMaaP containing neighbor list update

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) DMaaP is up

Testing Steps

Steps	Expected Result
Ensure relevant DMaaP topics are created (PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT), along with producer and consumers (using unauthenticated mode). Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP manually	SON-Handler MS receives the DMaaP PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT message and processes it - verified via logs and DB contents

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#007

Test Case Name

SON-Handler MS fetching neighbor details

Description

SON Handler Micro-service is able to successfully fetch neighbor list details using a REST interface

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) IT#006 is completed

Testing Steps

Steps	Expected Result
Bring up ConfigDB using the installation steps. Pre-populate RAN topology data in ConfigDB using a configuration file. Post a neighbor-list change notification message (Topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006.	SON-Handler MS receives the DMaaP PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT message and invokes ConfigDB REST APIs to obtain neighbor list details for cluster formation and determination of collisions and confusions (verified via logs).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#008

Test Case Name

OOF trigger for PCI optimization

Description

SON Handler Micro-service invokes REST API of OOF for PCI optimization

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) OOF is online

(d) IT#004 is completed.

(e) IT#005 or IT#006 is completed

Testing Steps

Steps	Expected Result
Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.	SON-Handler MS receives the DMaaP message and processes it. After determining that the number of collisions/confusions is greater than the threshold for triggering OOF, SON-Handler triggers OOF for PCI optimization (verified by logs on SON-Handler MS and OOF sides).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#009

Test Case Name

Config DB interaction with OOF

Description

OOF is able to obtain relevant neighbor list information for optimization algorithm from ConfigDB (REST API)

Release

Dublin

Pre-conditions

(a) OOF is online

(b) ConfigDB is up and running

Testing Steps

Steps	Expected Result
Configure Config-DB with the RAN topology information using a configuration file. If IT#008 is completed, either post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005, else, invoke OOF's REST API for PCI optimization.	OOF receives PCI optimization request, and invokes ConfigDB APIs to obtain information about the neighbors for executing the PCI optimization algorithm (verified by logs).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#010

Test Case Name

OOF response for PCI optimization

Description

SON Handler Micro-service receives optimization result from OOF (via callback URL) and sends response

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) OOF is online

(d) IT#008 and IT#009 are completed.

Testing Steps

Steps	Expected Result
Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.	SON-Handler MS receives the DMaaP message and triggers OOF for PCI optimization (verified by logs on SON-Handler MS and OOF sides). OOF executes PCI optimization algorithm and provides the result to SON-Handler MS using the callback URL (verified by logs on OOF and SON-Handler MS sides). SON-Handler MS sends a response to OOF (verified by logs).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#011

Test Case Name

Config DB interaction with SON-Handler MS for PNF details

Description

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) OOF is online

(d) IT#010 is completed.

Testing Steps

Steps

Expected Result

Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.
Note: In case this test case has to be done independent of OOF (i.e., IT#009 and IT#010), then upon completion of IT#008, the callback URL of SON-Handler MS can be invoked manually.

SON-Handler MS receives PCI-optimization results via the callback URL (verified by logs on OOF and SON-Handler MS sides).
SON-Handler MS then invokes ConfigDB API for obtaining PNF names corresponding to the cells whose PCI values have been modified by OOF (verified by logs on Config DB and SON-Handler MS sides).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#012

Test Case Name

SON-Handler MS interaction with Policy (DMaaP) for PCI updates

Description

SON Handler Micro-service is able to send PCI/neighbor list updates via DMaaP to Policy

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) Policy is online

(d) IT#011 is completed.

Testing Steps

Steps

Expected Result

Ensure that the DMaaP topic DCAE_CL_OUTPUT is created, and producers/consumers are registered.
Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.

SON-Handler MS receives PCI optimization results via the callback URL.
SON-Handler MS then prepares and sends a DMaaP message (topic: DCAE_CL_OUTPUT) to Policy (verified by logs) for the required PCI changes.

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#013

Test Case Name

Policy interaction with SDN-R (DMaaP) for PCI updates

Description

Policy is able to successfully send a DMaaP message (to SDN-R) with PCI/neighbor list updates.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

Testing Steps

Steps	Expected Result
Ensure that the DMaaP topics DCAE_CL_OUTPUT and SDNR_CL are created, and producers/consumers are registered. Implement the steps for IT#012, if IT#012 is not completed, post a DMaaP message with topic DCAE_CL_OUTPUT containing details of PCI/neighbor list updates.	Policy receives the DMaaP message with topic DCAE_CL_OUTPUT and invokes a control loop (verified by by checking active control loops). Policy then sends a DMaaP message to SDN-R with topic SDNR-CL containing the relevant contents received in the DCAE_CL_OUTPUT message (verified by logs).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#014

Test Case Name

Policy interaction with DMaaP for PCI update response

Description

Policy is able to successfully receive a DMaaP message with PCI/neighbor list update response

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

(c) IT#013 is completed

Testing Steps

Steps	Expected Result
Ensure that the DMaaP topic SDNR-CL-RSP is created, and producers/consumers are registered. Post a DMaaP message with topic SDNR_CL_RSP containing details of success/failure of PCI/neighbor updates	Policy receives the DMaaP message with topic SDNR_CL_RSP and sends a DMaaP message to SON-Handler MS with topic DCAE_CL_RSP (verified by logs). The control loop is also no longer active (verified by checking active control loops).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#015

Test Case Name

Policy interaction with SON-Handler MS (DMaaP) for PCI update response

Description

SON Handler Micro-service is able to receive PCI/neighbor list update response via DMaaP from Policy, process it and logs the results.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

(c) IT#014 is completed

Testing Steps

Steps	Expected Result
Ensure that the DMaaP topic DCAE_CL_RSP is created, and producers/consumers are registered. Execute the steps in IT#014.	SON-Handler MS receives the DMaaP message with topic DCAE_CL_RESPONSE from Policy, processes it and logs the relevant information about the failure/success of the PCI/neighbor changes in the RAN.

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#016

Test Case Name

ANR update by SON-Handler MS

Description

SON Handler Micro-service is able to process PM KPI data, store as needed, and apply rules to recommend removal of a neighbor relationship.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

(c) IT#005, IT#007 and IT#011 are completed

Testing Steps

Steps	Expected Result
Ensure SEC_MEASUREMENT_OUTPUT and DCAE_CL_OUTPUT DMaaP topics are created, along with producer and consumers (using unauthenticated mode) Post a sequence of PM messages (REST) to VES Collector. If feasible, use the RAN-Simulator to post the PM message.	SON-Handler MS receives the DMaaP messages with topic SEC_MEASUREMENT_OUTPUT from VES Collector. SON-Handler MS stores the information as needed, processes and determines neighbors to be removed based on the handover metrics. SON-Handler MS then sends a DMaaP message with topic DCAE_CL_OUTPUT to Policy containing details of the neighbors to be removed (verified via logs).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#017

Test Case Name

Netconf interface of SDN-R (edit-config)

Description

Upon receiving message from Policy, SDN-R sends netconf message to RAN-Sim.

Release

Dublin

Pre-conditions

(a) SDN-R is up and running

(b) (RAN) devices are mounted on to SDN-R

Testing Steps

Steps	Expected Result
Send a DMaap message to SDN-R with topic SDNR-CL containing details of PCI/neighbor list updates. This can be done by executing the steps for IT#013, or by manually sending a DMaaP message (after ensuring DMaaP topic creation, and registration of producer/consumer).	SDN-R successfully receives the DMaaP message with topic SDNR-CL (verified via logs). SDN-R sends a netconf edit-config message to RAN-Sim (or netconf server) with details of the PCI/neighbor updates.

Conclusion (Pass /Fail)

Testing Lab

Windriver lab

Test Case ID

IT#018

Test Case Name

VES Collector interface with RAN

Description

VES Collector receives VES messages from RAN-Simulator

Release

Dublin

Pre-conditions

(a) DCAE Platform including VES-Collector is up and running.

(b) RAN-Simulator is up and running

Testing Steps

Steps	Expected Result
Modify the neighbor list via RAN-Sim GUI. This triggers a VES fault message (REST) from the RAN-Sim towards the VES collector. Initiate reporting of PM metrics via RAN-Sim GUI. This triggers periodic VES PM messages (REST) from the RAN-Sim towards the VES collector.	VES Collector successfully receives the fault message (verified via logs). VES Collector successfully receives the PM messages (verified via logs).

Conclusion (Pass /Fail)

Testing Lab

Windriver lab