High Level Design (PoC)
- Tony Finnerty
- Ciaran Johnston
- Benjamin Cheung
The purpose of the page is to:
Design the Model Driven CPS PoCREQ-384: Define and prove the concept of model driven in the context of CPSDone
Start the design of a stand-alone CPS
Engage with the CPS community on the design of the stand-alone CPS
Review with the ARCHCOM to ensure broad ONAP community awareness and agreed direction
- 1 PoC Overview Diagram
- 2 Scope
- 2.1 Base scope
- 2.2 Stretch target
- 2.3 Proof points
- 2.4 Out of scope
- 3 High Level Design
- 4 Logical view of the CPS as a project
- 5 Data lake and access control
- 5.1 Solution
- 5.2 PoC implemetaion
- 6 Model handling
- 7 Detailed design
- 7.1 Key sequences/flows
- 7.2 Interfaces
- 8 Decision reference
PoC Overview Diagram
Topic Areas
Topic Area | Description |
|---|---|
 | MODEL DRIVEN SCHEMA: Relates to Design time activities. SDC operation. Distribution of the CSAR and the Model-driven Schema coming from Design time |
UPDATE: Takes incoming Standards Defined VES event from DCAE (PNF) and is published on DMAAP | |
SYNCHRONIZATION: C&PS synchronizes to A&AI, which allows it to get the latest view of what is in A&AI and align it with the C&PS data records. | |
SERVICE PROVIDER INTERFACE: The C&PS interface to the DBMS | |
C&PS API: The API that allows other micro-services and other components to access C&PS DB with CRUD operations. |
Scope
Base scope
Read/write persisted Configuration Management data:
defined by xNF (simple YANG model), published as YANG
Models deployed @ runtime with no ONAP platform impacts or LCM events
Show the benefits in terms of constraint validation, access and upgrade
Stretch target
Adapt behavior of CPS on read/write based on information in the model:
Change notification emissions driven by the model
Interaction with temporal data store, synergize with State Management PoC
Volatile/prolific xNF data read-through (e.g. state data) – may be used to support histogram
Model made available to CPS (Configuration and Persistency Service) using existing onboarding via SDC
Provide some indication of dimension and user access characteristics that will be provided by CPS
Proof points
Demonstrate Create/read operations using YANG fragments against a CPS backed by very simple schema / schema-less repository
Demonstrate ability to deploy / upgrade YANG fragments at run-time
Demonstrate CPS behavior driven by YANG model
Provide architecture vision and roadmap for a target architecture, supported use cases, non-functional requirements towards an ONAP Project
Out of scope
Automating the ingestion of models from SDC or controller (unless we get to it in the stretch target). The model LCM interface will be exercised manually during the PoC.
Syncing with A&AI. The PoC will work on a fixed topology
High Level Design
Context of CPS
Interface Id | Purpose | ||
|---|---|---|---|
AAIE-1 | Read and maintain relevant parts of the topology | ||
SDCE-6 | React to CSAR packages (on-boarded in SDC) that contain models of interest | ||
CPS-E-01 | Model lifecycle (add and remove models) | ||
CPS-E-02 | Data access interface (CRUD) | ||
CPS-E-03 | Notification of data change (configured by model) |
C4 diagram of main components
Logical view of the CPS as a project
Participant Name | Functional purpose | PoC handling |
|---|---|---|
YANG Model parser | Converts YANG models into java objects | Required |
YANG Data parser | Converts data compliant with a YANG model into java primitives, assists the CPS core | Required |
Model store | Keeps a local cache of the structures and relationships (generated from the YANG models) required by the CPS | Required |
SDC Model loader | Subscribes to DMaaP for CSAR publication. Parses and extracts the models from the CSAR and gives them to the xNF Model Data Proxy | Not required |
Topology sync | Does initial sync and maintains an up to date view of the active topology state. Informs the mirror service of any changes | Not required |
xNF Model Data Proxy | Encapsulation of all model and data access that relates to xNF | Not required |
xNF Proxy Data access | REST interface that provides access to xNF data | Not required |
CPS Core | Provides validation of and access to data. Is agnostic to the DB technology and schema | Required |
CPS Data access | Provides REST CRUD access to the data | Required |
CPS notification | Provides a DMaaP notification in the event that (1) data is changed and (2) it has been tagged/marked for notifcation | Required for stretch |
PostgreSQL plugin | Maps the java primitive representation of the YANG data objects to their DB technology (PostgreSQL) and schema specifics (SQL) | Required |
PostgreSQL | Database Management System for current data | Required |
Temporal [Stack] | Placeholder for logical components representing the temporal handling of data | Not required |
Temporal DB | Database Management System for temporal data | Not required |
The blue software encapsulates the handling of models and data related to xNF (mirrored data). This implementation is not required to prove the model driven aspects of the CPS.
The pink software encapsulates the integration with other data stores. This is being taken by the temporal DB PoC. This PoC needs to provide a feed of change notifications for any joint demonstrations. Configuring the parts of the model that will emit change notifications is part of the stretch goal.
Data lake and access control
The CPS will make it possible to share data across components. This is beneficial where it is expensive to source data or access latency is a concern.
The ability to share data raises concerns w.r.t. uncontrolled coupling between components. This makes system maintenance difficult.
The CPS will provide a balance between these competing needs.
For more discussion, see: Common information model, Data lake and Access control
Solution
To be useful to an application, data must be accessible and possible to interpret.
Access is provided by the CPS. Access may be qualified as read only or write.
The ability to interpret data requires access to the model that defines the structure and constraints of the data.
Consider the following scenario.
Svc A and Svc B are considered to own their data, A and B respecively.
Data A and Data B are stored in a Common Data Service (e.g. DBMS). This is important, but not relevant to this discussion.
Data A and Data B may represent multiple instances of the same model – for example there may be multiple instances of the same xNF in the scope of the ONAP system, each would need to be in a separate tree, all compliant to the same model.
Svc C would like to access Data A and Data B, without going through the public interfaces exposed by Svc A and Svc B.
Svc C must petition Svc A and Svc B for access rights (this may be a design or run-time activity) to their data.
Once access has be provided by the owners (Svc A and Svc B) Svc C may use this permission to access data directly.
Svc C cannot completely interpret the data without access to the models. The model is the responsibility of the owning service, and is provided to Svc C together with the permission and any relevant instances of data trees.
This is also the case for the CPS. It cannot persist or validate data without access to the model. It is the responsibility of the data owner to provide the model to the CPS prior to data access.
The run-time aspect of the granting of permission has some potential to also address some race conditions.
The key benefits are: (1) Data lake use cases are enabled; (2) Coupling between components at the level of 'shared' data are explicitly declared/published.
PoC implemetaion
The PoC will start with a compatible subset of the above implementation. Nothing in the PoC implementation will preclude the development of the above solution.
All access to data will be via the owning service.
Model handling
Below a simplified view of model handling artifacts and their relationships.
The run-time artifacts are shaded (orange). A typical simplified flow:
A document representing xNF data is given to the CPS.
It will parse (and validate) the document,
Map data to the generic DB schema and store data.
In order for this to work:
The document must be compliant to an xNF schema model. This schema model describes the structure of the data and any constraints that need to be checked to ensure integrity.
The xNF schema model must be written in YANG. This language describes syntax and capabilities (basic concepts).
A Yang parser is required to interpret documents compliant with the YANG language. The parser is dependent on the language.
To store data in a DBMS, typically a schema is required. This is denoted above as, 'Generic schema model'.
Obviously the xNF schema model and the Generic schema model are not the same. Therefor a mapping is required. Depending on how simple/generic/abstract the generic schema model, the mapping may be coupled with either the language or the xNF schema model. One of the aims of the PoC is to be agnostic to the xNF model at run-time, thus the mapping is made to the YANG language.
To support multiple container run-times (Java, Python, Go, Rust, ...) for the CPS, the mapping is described in a 'Language Capability Map'.
The model mapper depends on the language capability map and the Yang parser (in particular the java representation of the language features).
Model ingestion
Model ingestion is subject to a design discussion. See below. This section will be updated when that discussion has been concluded and a decision made.
Note that for the purposes of the PoC, model ingestion will be via a model life-cycle REST interface.
Model upgrade
T.B.D. See Upgrade of models for discussion and decision
Backward compatibility
See decision for more information. Compatibility will be provided by either a dedicated component, or ONAP applications depending on legacy interfaces will be updated. This will depend on resources.
Detailed design
Key sequences/flows
Add a model
Store data (including validation)
Read data
Reading data is very similar to storing data. The uuid of the model instance is assumed. Validation is not needed. Queries will need to be processed and mapped to DB technology specifics.
Upgrade a model
T.B.D. Needs to wait until upgrade behavior is specified.
Adapt change notification behavior as described in model
T.B.D. Show how the model spec is used to control interactions between CPS Core and CPS Notification
Interfaces
Style and payload structure
The payload will be represented as Generic objects/documents. For more reasoning and discussion, see Data Representation
In java, the CPSDataObject will be a 'glorified' map -- not a basic map, but one level more concrete (in terms of abstractions). I.e. at the language level
The sample java code below illustrates ubiquitous use of the CPSDataObject.
public class CPSDataObject extends Map {...}
CPSDataObject tree = cps.getTree(obj, level, filter)
List<CPSDataObject> refs = cps.getRefs(obj)
CPSDataObject parent = cps.getParent(obj)
List<CPSDataObject> children = cps.getChildren(obj)Filters will be encoded as XPath expressions.
YANG <-> Java type mapping will be inherited from the selection of YANG parser
For REST, the payload will be a document complying to a generic JSON schema.
The style of the interface will be such that access methods are associated with the CPS. Data is provided as context. For more reasoning and discussion, see Interface style.
The pseudo Java code below illustrates this style by showing how complementary relationships could be navigated.
CPSDataObject cpsRootObject = CPS.getRootObject(uuid_to_identify_inventory_item);
System.out.println(cpsRootObject);
List<CPSDataObject> kids = CPS.getChildren(cpsRootObject);
for (CPSDataObject kid: kids) {
System.out.println(CPS.getParent(kid));
}
REST payload structure
Java payload structure
Access methods
Name | Interface | Description | Parameters | Return |
|---|---|---|---|---|
Add model | CPS-E-01 | Add a YANG model to the CPS so that compliant data may be stored & retrieved |
| Some identifier |
Remove model | CPS-E-01 | Mark a model for removal, and remove when no data remains that relies on the model |
| Success/Number of model instances still using model |
Upgrade model | CPS-E-01 | T.B.D |
| - |
Create model instance | CPS-E-02 | Prepare the underlying DB for the storage of data compliant to the model. |
| Model instance identifier |
Delete model instance | CPS-E-02 | Remove the data and any artifacts from the underlying DB |
| Success/Failure |
Create data | CPS-E-02 | [Validate &] Store data compliant to a model |
| Success/Failure |
Replace data | CPS-E-02 | [Validate &] Store data compliant to a model in place of existing data |
| Success/Failure |
Update data | CPS-E-02 | [Validate &] Update data that is (1) compliant to a model and (2) already stored |
| Success/Failure |
Delete data | CPS-E-02 | Remove data from a model instance |
| Success/Failure |
Data change notification | CPS-E-03 | Notify interested parties that data (marked as interesting by the model) has been modified |
| - |
Service Provider Interface (SPI)
T.B.D. An internal interface that defines how the CPS Core will interact with the DBMS. The design of the SPI will be informed by agreed CPS behavior and minimum requirements for the DBMS.
The SPI is required to ensure that different DB technologies can be used in different deployments.
Behavior
T.B.D.: Eventual consistency; Data integrity guarantees; Fast failure; Progress; Cancellation;
REST
Will be specified as part of the PoC execution, and updated/referenced from here
This will follow REST rules and guidelines as defined by ONAP documentation (TODO: add link)
Java
Will be specified as part of the PoC execution, and updated/referenced from here