Page Comparison

...

Test data used complies with Open ROADM YANG model. Specifically, openroadm-device nodes consisting of 86 fragments are created. For example, a test that creates 1000 device nodes will result in 86,000 fragments in the database. Some tests use up to 3000 device nodes (258,000 fragments - equivalent to around 20,000 CM-handles in NCMP), with four anchors replicating the data, meaning that the system has been tested up to 1 million fragments. Not all results are displayed on this page, but are included in the attached spreadsheet.

Storing data nodes

A varying number of Open ROADM device nodes will be stored using CpsDataService::saveData.

Observations

• Storing data nodes has linear time complexity (as expected).
• Raw performance is roughly 3000 fragments per second for the given test setup.
• Performance can be improved by enabling write batching (CPS-1795)
• There are edge cases with exponential complexity.

...

In this scenario, 1000 Open ROADM device nodes are already defined. A number of these existing data nodes will be updated using CpsDataService::updateDataNodeAndDescendants.

Observations

• Updating data nodes has linear time complexity (as expected).
• Raw performance is roughly 600 fragments per second for the given model and test setup.
• Updating data nodes is 5 times slower than storing data nodes.

...

{
    'openroadm-device': [
        {'device-id':'C201-7-1A-1', 'status':'fail', 'ne-state':'jeopardy'}
    ]
}

Test Results

Observations

• Updating data leaves has linear time complexity.
• Raw performance is about 3000 fragments per second.
• This is very fast compared to updating whole data nodes. I recommend that NCMP use this API for updating CM-handle state.

...

In this scenario, 300 Open ROADM device nodes are already defined. A number of these data nodes will be deleted using CpsDataService::deleteDataNodes. The types of nodes will be varied, for example, deleting container nodes, list elements, or whole lists.

Test results

Observations

• Delete performance is linear on the amount of data being deleted (as expected).
• Raw performance of deleting containers of list elements is around 40,000 fragments per second. (So we can delete data nodes around 10x faster than creating them.)
• Deleting lists is much slower than deleting the parent container of the list (which can be easily improved).
• Of note, attempting to delete non-existing data nodes takes longer than actually deleting the equivalent amount of nodes with descendants - it is a slow operation.

...

Reading the top-level container node with no descendants:

The above data clearly indicates constant time.

Reading the top-level container node with all descendants:

Observations

• Reading a single top-level container node with no descendants has constant time (as expected).
• Reading a single top-level container node with all descendants has linear time (as expected).
• Raw performance of reading with all descendants is roughly 100,000 fragments per second.

...

This test uses CpsDataService::getDataNodesForMultipleXpaths with all descendants to retrieve a varying number of Open ROADM device nodes.

Test results

Special case: attempting to read multiple non-existing data nodes

...

As can be seen below, CPU usage never reached 100% during the tests of the current version.

CM-handle

...

de-registration

...