Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The purpose of the page is to have content for review with the ARCHCOM before the end of July 2020:

  1. Design the Model Driven CPS PoC
    Jira Legacy
    serverSystem Jira
    serverId4733707d-2057-3a0f-ae5e-4fd8aff50176
    keyREQ-384
  2. Start the design of a stand-alone CPS
  3. Engage with the CPS community on the design of the stand-alone CPS
  4. Review with the ARCHCOM to ensure broad ONAP community awareness and agreed direction

Table of Contents
maxLevel3

...

Drawio
bordertrue
diagramNameCPS Major participants
simpleViewerfalse
linksauto
tbstyletop
lboxtrue
diagramWidth1181
revision1116

Participant NameFunctional purposePoC handling
YANG Model parserConverts YANG models into java objectsRequired
YANG Data parserConverts data compliant with a YANG model into java primitives, assists the CPS coreRequired
Model storeKeeps a local cache of the structures and relationships (generated from the YANG models) required by the CPSRequired
SDC Model loaderSubscribes to DMaaP for CSAR publication. Parses and extracts the models from the CSAR and gives them to the mirror servicexNF Model Data ProxyNot required
Topology syncDoes initial sync and maintains an up to date view of the active topology state. Informs the mirror service of any changesNot requiredMirror service
xNF Model Data ProxyEncapsulation of all model and data access that relates to xNFNot required
Mirror xNF Proxy Data accessREST interface that provides access to xNF dataNot required
CPS CoreProvides validation of and access to data. Is agnostic to the DB technology and schemaRequired
CPS Data accessProvides REST CRUD access to the dataRequired
CPS notificationProvides a DMaaP notification in the event that (1) data is changed and (2) it has been tagged/marked for notifcationRequired for stretch
PostgreSQL pluginMaps the java primitive representation of the YANG data objects to their DB technology (PostgreSQL) and schema specifics (SQL)Required
PostgreSQLDatabase Management System for current dataRequired
Temporal [Stack]Placeholder for logical components representing the temporal handling of dataNot required
Temporal DBDatabase Management System for temporal dataNot required

...

The pink software encapsulates the integration with other data stores. This is being taken by the temporal DB PoC. This PoC needs to provide a feed of change notifications for any joint demonstrations. Configuring the parts of the model that will emit change notifications is part of the stretch goal.

Data lake and access control

The CPS will make it possible to share data across components. This is beneficial where it is expensive to source data or access latency is a concern.

The ability to share data raises concerns w.r.t. uncontrolled coupling between components. This makes system maintenance difficult.

The CPS will provide a balance between these competing needs.

For more discussion, see: Common information model, Data lake and Access control

Solution

To be useful to an application, data must be accessible and possible to interpret.

Access is provided by the CPS. Access may be qualified as read only or write.

The ability to interpret data requires access to the model that defines the structure and constraints of the data.

Consider the following scenario.

image2020-7-21_16-46-16.pngImage Added

Svc A and Svc B are considered to own their data, A and B respecively.

Data A and Data B are stored in a Common Data Service (e.g. DBMS). This is important, but not relevant to this discussion.

Data A and Data B may represent multiple instances of the same model – for example there may be multiple instances of the same xNF in the scope of the ONAP system, each would need to be in a separate tree, all compliant to the same model.

Svc C would like to access Data A and Data B, without going through the public interfaces exposed by Svc A and Svc B.

Svc C must petition Svc A and Svc B for access rights (this may be a design or run-time activity) to their data.

Once access has be provided by the owners (Svc A and Svc B) Svc C may use this permission to access data directly.

Svc C cannot completely interpret the data without access to the models. The model is the responsibility of the owning service, and is provided to Svc C together with the permission and any relevant instances of data trees.

This is also the case for the CPS. It cannot persist or validate data without access to the model. It is the responsibility of the data owner to provide the model to the CPS prior to data access.

The run-time aspect of the granting of permission has some potential to also address some race conditions.

The key benefits are: (1) Data lake use cases are enabled; (2) Coupling between components at the level of 'shared' data are explicitly declared/published.

PoC implemetaion

The PoC will start with a compatible subset of the above implementation. Nothing in the PoC implementation will preclude the development of the above solution.

image2020-7-21_16-40-28.pngImage Added

All access to data will be via the owning service.

Model handling

Below a simplified view of model handling artifacts and their relationships.

...

Backward compatibility

See decision for more information. Compatibility will be provided by either a dedicated component, or ONAP applications depending on legacy interfaces will be updated. This will depend on resources.

...

Adapt change notification behavior as described in model

T.B.D. Show how the model spec is used to control interactions between CPS Core and CPS Notification

Interfaces

Style and payload structure

The payload will be represented as Generic objects/documents. For more reasoning and discussion, see Data Representation

In java, the CPSDataObject will be a 'glorified' map -- not a basic map, but one level more concrete (in terms of abstractions). I.e. at the language level
The sample java code below illustrates ubiquitous use of the CPSDataObject.

Code Block
languagejava
public class CPSDataObject extends Map {...}

CPSDataObject tree = cps.getTree(obj, level, filter)
List<CPSDataObject> refs = cps.getRefs(obj)
CPSDataObject parent = cps.getParent(obj)
List<CPSDataObject> children = cps.getChildren(obj)

Filters will be encoded as XPath expressions.

YANG <-> Java type mapping will be inherited from the selection of YANG parser

For REST, the payload will be a document complying to a generic JSON schema.

The style of the interface will be such that access methods are associated with the CPS. Data is provided as context. For more reasoning and discussion, see Interface style.

The pseudo Java code below illustrates this style by showing how complementary relationships could be navigated.

Code Block
languagejava
CPSDataObject cpsRootObject = CPS.getRootObject(uuid_to_identify_inventory_item);
System.out.println(cpsRootObject);
List<CPSDataObject> kids = CPS.getChildren(cpsRootObject);
for (CPSDataObject kid: kids) {
	System.out.println(CPS.getParent(kid));
}
REST payload structure
Java payload structure
Access methods
NameInterfaceDescriptionParametersReturn
Add modelCPS-E-01Add a YANG model to the CPS so that compliant data may be stored & retrieved
  • YANG model
Some identifier
Remove modelCPS-E-01Mark a model for removal, and remove when no data remains that relies on the model
  • Model identifier
Success/Number of model instances still using model
Upgrade modelCPS-E-01T.B.D
  • YANG model
  • Model identifier
-
Create model instanceCPS-E-02Prepare the underlying DB for the storage of data compliant to the model.
  • Model identifier
  • Instance unique identifier
Model instance identifier
Delete model instanceCPS-E-02Remove the data and any artifacts from the underlying DB
  • Instance unique identifier
Success/Failure
Create dataCPS-E-02[Validate &] Store data compliant to a model
  • Instance unique identifier
  • Data structure/document
Success/Failure
Replace dataCPS-E-02[Validate &] Store data compliant to a model in place of existing data
  • Instance unique identifier
  • Data structure/document
Success/Failure
Update dataCPS-E-02[Validate &] Update data that is (1) compliant to a model and (2) already stored
  • Instance unique identifier
  • Data structure/document
Success/Failure
Delete dataCPS-E-02Remove data from a model instance
  • Instance unique identifier
  • Data structure/document
Success/Failure
Data change notificationCPS-E-03Notify interested parties that data (marked as interesting by the model) has been modified
  • Instance unique identifier
  • Data structure/document
  • Previous data structure/document
-
Service Provider Interface (SPI)

T.B.D. An internal interface that defines how the CPS Core will interact with the DBMS. The design of the SPI will be informed by agreed CPS behavior and minimum requirements for the DBMS.

The SPI is required to ensure that different DB technologies can be used in different deployments.

Behavior

T.B.D.: Eventual consistency; Data integrity guarantees; Fast failure; Progress; Cancellation;

REST

Will be specified as part of the PoC execution, and updated/referenced from here

This will follow REST rules and guidelines as defined by ONAP documentation (TODO: add link)

Java

Will be specified as part of the PoC execution, and updated/referenced from here

Decision reference

Issues decisions and assumptions