Integration Testing

Note: The OOF-PCI use case

...

integration testing is being done at Windriver lab.

Note: DMaaP should be setup for most of the test cases below.

#

Component(s)

Test Case

Status

Remarks

IT#001

DCAE (SON Handler MS)

SON Handler Micro-service successfully on-boarded on to DCAE

Status

colour	Green
title	Complete

IT#002

DCAE (SON Handler MS)

SON Handler Micro-service's DB is up and the Micro-service is able to read/write data.

Status

colour	Green
title	Complete

IT#003

DCAE (SON Handler MS)

SON Handler Micro-service is able to successfully fetch config policies from Consul.

NOT YET TESTEDIT#00

Status

colour	Green
title	Complete

IT#004

OOF and Policy

OOF is able to successfully fetch config policy from Policy

NOT YET TESTED

Status


colour	Blue
title	NOT APPLICABLE

This test case is no longer applicable, as there is no policy-driven actions in OOF for this use case in Dublin.

IT#005

DCAE (SON Handler MS)

SON Handler Micro-service is able to successfully receive a message over DMaaP containing PM/FM data (from VES Collector)

Status

colour

Yellow

title
Green

Partially

Complete

IT#006

DCAE (SON Handler MS) and DMaaP

SON Handler Micro-service is able to successfully receive a message over DMaaP containing neighbor list update

NOT YET TESTED

Status

colour	Green
title	Complete

IT#007

DCAE (SON Handler MS) REST interface

SON Handler Micro-service is able to successfully fetch neighbor list details using REST interface (Config DB APIs)

NOT YET TESTEDIT#014Policy and DMaaPPolicy is able to successfully receive a DMaaP message with PCI/neighbor list update response

NOT YET TESTED

IT#015DCAE (SON Handler MS) and PolicySON Handler Micro-service is able to receive PCI/neighbor list update response via DMaaP from Policy, process it and logs the results.

NOT YET TESTED

IT#016

DCAE (SON Handler MS)SON Handler Micro-service is able to process PM KPI data, store as needed, and apply rules to recommend removal of a neighbor relationship.

NOT YET TESTED

IT#017SDN-RUpon receiving message from Policy, SDN-R sends netconf message to RAN-Sim.NOT YET TESTED

Status

colour	Green
title	Complete

IT#008

DCAE (SON Handler MS) and OOF

SON Handler Micro-service invokes REST API of OOF for PCI optimization

NOT YET TESTED

IT#009OOF and Config DB (REST API)OOF is able to obtain relevant neighbor list information for optimization algorithm from ConfigDB (REST API)

NOT YET TESTED

IT#010DCAE (SON Handler MS) and OOFSON Handler Micro-service receives optimization result from OOF (via callback URL) and sends response

NOT YET TESTED

IT#011DCAE (SON Handler MS) and Config DB (REST API)SON Handler Micro-service is able to successfully fetch PNF details from Config DB (using REST API call)

NOT YET TESTED

IT#012DCAE (SON Handler MS) and PolicySON Handler Micro-service is able to send PCI/neighbor list updates via DMaaP to Policy

NOT YET TESTED

IT#013Policy and SDN-RPolicy is able to successfully send a DMaaP message (to SDN-R) with PCI/neighbor list updates.

NOT YET TESTED

Status


colour	Green
title	COMPLETE

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	DCAEGEN2-1529

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	OPTFRA-505

IT#009

OOF and Config DB (REST API)

OOF is able to obtain relevant neighbor list information for optimization algorithm from ConfigDB (REST API)

Status


colour	Green
title	Complete

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	OPTFRA-508

IT#010

DCAE (SON Handler MS) and OOF

SON Handler Micro-service receives optimization result from OOF (via callback URL) and sends response

Status


colour	Green
title	Complete

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	OPTFRA-508

IT#011

DCAE (SON Handler MS) and Config DB (REST API)

SON Handler Micro-service is able to successfully fetch PNF details from Config DB (using REST API call)

Status

colour	Green
title	Complete

IT#012

DCAE (SON Handler MS) and Policy

SON Handler Micro-service is able to send PCI/ANR updates via DMaaP to Policy

Status


colour	Green
title	COMPLETE

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	DCAEGEN2-1529

IT#013

Policy and DMaaP

Policy is able to successfully send a DMaaP message (to SDN-R) with PCI/neighbor list updates.

Status

colour	Green
title	Complete

IT#014

Policy and DMaaP

Policy is able to successfully receive a DMaaP message with PCI/neighbor list update response

Status

colour	Green
title	Complete

IT#015

DCAE (SON Handler MS) and Policy

SON Handler Micro-service is able to receive PCI/neighbor list update response via DMaaP from Policy, process it and logs the results.

Status

colour	Green
title	Complete

IT#016

DCAE (SON Handler MS)

SON Handler Micro-service is able to process PM KPI data, store as needed, and apply rules to recommend removal of a neighbor relationship.

Status


colour	Green
title	COMPLETE

IT#017

SDN-R

Upon receiving message from Policy, SDN-R sends netconf message to RAN-Sim.

Status


colour	Blue
title	NOT APPLICABLE

pending completion of 2 known issues (Medium): CCSDK-1399 and CCSDK-1400

Will be completed when CCSDK-1399 and CCSDK-1400 are delivered (so not part of Dublin).

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	CCSDK-1364

Jira Legacy

server	System Jira
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	4733707d-2057-3a0f-ae5e-4fd8aff50176
key	SDNC-772

The DMaaP listener docker with the code fix is not yet deployed in SB-00 where the testing is done. The code, fix however, is already merged on to Dublin and Master branches. So, once the latest docker is deployed, we will continue and complete the test case at Windriver.

We submitted additional bug fixes and hope that the new Jenkins release would be deployed in the integration labs for testing

Jira: CCSDK-1399 https://jira.onap.org/browse/CCSDK-1399
Code commit: https://gerrit.onap.org/r/c/ccsdk/sli/northbound/+/89792

One bug/issue is still under investigation and has been reported onto the SDNR distribution list.

https://lists.onap.org/g/onap-SDNR/message/386
Jira: CCSDK-1400 (https://jira.onap.org/browse/CCSDK-1400)

IT#018

DCAE (VES Collector)

VES Collector receives VES messages from RAN-Simulator

Status

colour	Green
title	Complete

Test Case ID

IT#001

Test Case Name

SON Handler Micro-service on-boarding

Description

SON Handler Micro-service is successfully on-boarded on to DCAE

Release

Dublin

Pre-conditions

(a) DCAE up and running

(b) DCAE CLI available

Testing Steps

Steps	Expected Result
SON-Handler component spec validated. SON-Handler is brought up manually on DCAE. SON-Handler Blueprint validated successfully. DCAE is brought online.	SON-Handler comes up online and is stable.

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#002

Test Case Name

SON-Handler Micro-service DB on-boarding

Description

SON-Handler Micro-service's PG DB is up, and SON-Handler Micro-service is able to write data e.g., local configuration data and read it.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is up and running

Testing Steps

Steps	Expected Result
SON-Handler component and its DB is on-boarded on to DCAE. Local configurable attributes are written/read by SON-Handler MS (verified by viewing logs)	Verify DB working by viewing SON-Handler MS logs.

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#003

Test Case Name

Policy fetch by SON-Handler MS

Description

SON Handler Micro-service is able to successfully fetch config policies from Consul.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is up and running

(b) DCAE Platform including Consul is up and running

Testing Steps

Steps	Expected Result
Upload config policies for SON-Handler MS. SON-Handler MS should fetch the policies from Consul	Verify Policy fetch and store via logs and checking in PG DB

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#004

Test Case Name

Policy fetch by OOF

Description

OOF is able to successfully fetch the config policies from Policy

Release

Dublin

Pre-conditions

(a) Policy component is up and running

(b) Config policies related to OOF are provisioned in Policy component

Testing Steps

Steps

Expected Result

Bring up the OOF using the installation steps.
Check the OOF database to see the contents of the OOF config policy for PCI optimization use case (i.e., OOF should invoke Policy API to fetch the config policies).

OOF comes online and the database and local datastructures for PCI optimization use case are initialized correctly.
Config-policy is correctly fetched and stored in the OOF DB. Sample config policy contents are given below:
ALGO_CATEGORY:OOF-PCI-OPTIMIZATION
PCI_OPTMIZATION_ALGO_NAME:OOF-PCI-OPTIMIZATION-LEVEL1
PCI_OPTIMIZATION_NW_CONSTRAINT:MAX5PCICHANGESONLY
PCI_OPTIMIZATION_PRIORITY: 2
PCI_OPTIMIZATION_TIME_CONSTRAINT:NONE

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#005

Test Case Name

VES (DMaaP) interaction with SON-Handler MS

Description

SON Handler Micro-service is able to successfully receive a message over DMaaP containing PM/FM data (from VES Collector).

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running, VES is online

(b) SON-Handler MS is online

(c) DMaaP is up

Testing Steps

Steps	Expected Result
Ensure relevant DMaaP topics are created (SEC_FAULT_OUTPUT, SEC_MEASUREMENT_OUTPUT), along with producer and consumers (using unauthenticated mode) Post a fault message (REST) to VES Collector. If feasible, use the RAN-Simulator to post the fault message. Post a performance measurements message (REST) to VES Collector. If feasible, use the RAN-Simulator to post the fault message.	VES Collector posts the received fault message on to DMaaP using the topic SEC_FAULT_OUTPUT. SON-Handler MS receives the DMaaP SEC_FAULT_OUTPUT message and processes it - verified via logs. VES Collector posts the received PM message on to DMaaP using the topic SEC_MEASUREMENT_OUTPUT. SON-Handler MS receives the DMaaP SEC_MEASUREMENT_OUTPUT message and processes it - verified via logs.

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#006

Test Case Name

DMaaP interaction with SON-Handler MS

Description

SON Handler Micro-service is able to successfully receive a message over DMaaP containing neighbor list update

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) DMaaP is up

Testing Steps

Steps	Expected Result
Ensure relevant DMaaP topics are created (PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT), along with producer and consumers (using unauthenticated mode). Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP manually	SON-Handler MS receives the DMaaP PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT message and processes it - verified via logs and DB contents

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#007

Test Case Name

SON-Handler MS fetching neighbor details

Description

SON Handler Micro-service is able to successfully fetch neighbor list details using a REST interface

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) IT#006 is completed

Testing Steps

Steps	Expected Result
Bring up ConfigDB using the installation steps. Pre-populate RAN topology data in ConfigDB using a configuration file. Post a neighbor-list change notification message (Topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006.	SON-Handler MS receives the DMaaP PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT message and invokes ConfigDB REST APIs to obtain neighbor list details for cluster formation and determination of collisions and confusions (verified via logs).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#008

Test Case Name

OOF trigger for PCI optimization

Description

SON Handler Micro-service invokes REST API of OOF for PCI optimization

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) OOF is online

(d) IT#004 is completed.

(e) IT#005 or IT#006 is completed

Testing Steps

Steps	Expected Result
Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.	SON-Handler MS receives the DMaaP message and processes it. After determining that the number of collisions/confusions is greater than the threshold for triggering OOF, SON-Handler triggers OOF for PCI optimization (verified by logs on SON-Handler MS and OOF sides).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#009

Test Case Name

Config DB interaction with OOF

Description

OOF is able to obtain relevant neighbor list information for optimization algorithm from ConfigDB (REST API)

Release

Dublin

Pre-conditions

(a) OOF is online

(b) ConfigDB is up and running

Testing Steps

Steps	Expected Result
Configure Config-DB with the RAN topology information using a configuration file. If IT#008 is completed, either post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005, else, invoke OOF's REST API for PCI optimization.	OOF receives PCI optimization request, and invokes ConfigDB APIs to obtain information about the neighbors for executing the PCI optimization algorithm (verified by logs).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#010

Test Case Name

OOF response for PCI optimization

Description

SON Handler Micro-service receives optimization result from OOF (via callback URL) and sends response

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) OOF is online

(d) IT#008 and IT#009 are completed.

Testing Steps

Steps	Expected Result
Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.	SON-Handler MS receives the DMaaP message and triggers OOF for PCI optimization (verified by logs on SON-Handler MS and OOF sides). OOF executes PCI optimization algorithm and provides the result to SON-Handler MS using the callback URL (verified by logs on OOF and SON-Handler MS sides). SON-Handler MS sends a response to OOF (verified by logs).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#011

Test Case Name

Config DB interaction with SON-Handler MS for PNF details

Description

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) OOF is online

(d) IT#010 is completed.

Testing Steps

Steps

Expected Result

Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.
Note: In case this test case has to be done independent of OOF (i.e., IT#009 and IT#010), then upon completion of IT#008, the callback URL of SON-Handler MS can be invoked manually.

SON-Handler MS receives PCI-optimization results via the callback URL (verified by logs on OOF and SON-Handler MS sides).
SON-Handler MS then invokes ConfigDB API for obtaining PNF names corresponding to the cells whose PCI values have been modified by OOF (verified by logs on Config DB and SON-Handler MS sides).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#012

Test Case Name

SON-Handler MS interaction with Policy (DMaaP) for PCI updates

Description

SON Handler Micro-service is able to send PCI/neighbor list updates via DMaaP to Policy

Release

Dublin

Pre-conditions

(a) DCAE platform is up and running

(b) SON-Handler MS is online

(c) Policy is online

(d) IT#011 is completed.

Testing Steps

Steps

Expected Result

Ensure that the DMaaP topic DCAE_CL_OUTPUT is created, and producers/consumers are registered.
Post a neighbor-list change notification message (topic: PCI-NOTIF-TOPIC-NGHBR-LIST-CHANGE-INFO_OUTPUT) on DMaaP as done for IT#006 (or) post a DMaaP SEC_FAULT_OUTPUT message as done for IT#005. In either case, the message contents should be such that the number of resulting collisions/confusions should be more than the threshold for triggering OOF for PCI optimization.

SON-Handler MS receives PCI optimization results via the callback URL.
SON-Handler MS then prepares and sends a DMaaP message (topic: DCAE_CL_OUTPUT) to Policy (verified by logs) for the required PCI changes.

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#013

Test Case Name

Policy interaction with SDN-R (DMaaP) for PCI updates

Description

Policy is able to successfully send a DMaaP message (to SDN-R) with PCI/neighbor list updates.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

Testing Steps

Steps	Expected Result
Ensure that the DMaaP topics DCAE_CL_OUTPUT and SDNR_CL are created, and producers/consumers are registered. Implement the steps for IT#012, if IT#012 is not completed, post a DMaaP message with topic DCAE_CL_OUTPUT containing details of PCI/neighbor list updates.	Policy receives the DMaaP message with topic DCAE_CL_OUTPUT and invokes a control loop (verified by by checking active control loops). Policy then sends a DMaaP message to SDN-R with topic SDNR-CL containing the relevant contents received in the DCAE_CL_OUTPUT message (verified by logs).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#014

Test Case Name

Policy interaction with DMaaP for PCI update response

Description

Policy is able to successfully receive a DMaaP message with PCI/neighbor list update response

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

(c) IT#013 is completed

Testing Steps

Steps	Expected Result
Ensure that the DMaaP topic SDNR-CL-RSP is created, and producers/consumers are registered. Post a DMaaP message with topic SDNR_CL_RSP containing details of success/failure of PCI/neighbor updates	Policy receives the DMaaP message with topic SDNR_CL_RSP and sends a DMaaP message to SON-Handler MS with topic DCAE_CL_RSP (verified by logs). The control loop is also no longer active (verified by checking active control loops).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#015

Test Case Name

Policy interaction with SON-Handler MS (DMaaP) for PCI update response

Description

SON Handler Micro-service is able to receive PCI/neighbor list update response via DMaaP from Policy, process it and logs the results.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

(c) IT#014 is completed

Testing Steps

Steps	Expected Result
Ensure that the DMaaP topic DCAE_CL_RSP is created, and producers/consumers are registered. Execute the steps in IT#014.	SON-Handler MS receives the DMaaP message with topic DCAE_CL_RESPONSE from Policy, processes it and logs the relevant information about the failure/success of the PCI/neighbor changes in the RAN.

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#016

Test Case Name

ANR update by SON-Handler MS

Description

SON Handler Micro-service is able to process PM KPI data, store as needed, and apply rules to recommend removal of a neighbor relationship.

Release

Dublin

Pre-conditions

(a) SON-Handler MS is online

(b) Policy is online

(c) IT#005, IT#007 and IT#011 are completed

Testing Steps

Steps	Expected Result
Ensure SEC_MEASUREMENT_OUTPUT and DCAE_CL_OUTPUT DMaaP topics are created, along with producer and consumers (using unauthenticated mode) Post a sequence of PM messages (REST) to VES Collector. If feasible, use the RAN-Simulator to post the PM message.	SON-Handler MS receives the DMaaP messages with topic SEC_MEASUREMENT_OUTPUT from VES Collector. SON-Handler MS stores the information as needed, processes and determines neighbors to be removed based on the handover metrics. SON-Handler MS then sends a DMaaP message with topic DCAE_CL_OUTPUT to Policy containing details of the neighbors to be removed (verified via logs).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#017

Test Case Name

Netconf interface of SDN-R (edit-config)

Description

Upon receiving message from Policy, SDN-R sends netconf message to RAN-Sim.

Release

Dublin

Pre-conditions

(a) SDN-R is up and running

(b) (RAN) devices are mounted on to SDN-R

Testing Steps

Steps	Expected Result
Send a DMaap message to SDN-R with topic SDNR-CL containing details of PCI/neighbor list updates. This can be done by executing the steps for IT#013, or by manually sending a DMaaP message (after ensuring DMaaP topic creation, and registration of producer/consumer).	SDN-R successfully receives the DMaaP message with topic SDNR-CL (verified via logs). SDN-R sends a netconf edit-config message to RAN-Sim (or netconf server) with details of the PCI/neighbor updates.

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Test Case ID

IT#018

Test Case Name

VES Collector interface with RAN

Description

VES Collector receives VES messages from RAN-Simulator

Release

Dublin

Pre-conditions

(a) DCAE Platform including VES-Collector is up and running.

(b) RAN-Simulator is up and running

Testing Steps

Steps	Expected Result
Modify the neighbor list via RAN-Sim GUI. This triggers a VES fault message (REST) from the RAN-Sim towards the VES collector. Initiate reporting of PM metrics via RAN-Sim GUI. This triggers periodic VES PM messages (REST) from the RAN-Sim towards the VES collector.	VES Collector successfully receives the fault message (verified via logs). VES Collector successfully receives the PM messages (verified via logs).

Conclusion (Pass /Fail)

Testing Lab

WinlabWindriver lab

Versions Compared

Old Version 21

New Version Current

Key

Integration Testing

Note: The OOF-PCI use case