| Table of Contents | ||
|---|---|---|
|
Study the existing SPI
| Table of Contents | ||
|---|---|---|
|
Study the existing SPI used in ENM to migrate from Versant to Graph DB in ENM implementation. (TBA link to new GitHub repo with shared)
Learning from this SPI can be used to propose an SPI for the CPS SPI that also allows to 'plug-in' different DBs
- Interface with Tony Finnerty for direction
- ONAP Wiki Page describing
- High level structure of ENM SPI
- Already implemented SPI methods (from PoC)
- Suggestions for future structure & functionality
- Discuss with Team and agree proposal
Jira Ticket:
...
Jira Ticket:
| Jira Legacy | ||||||||
|---|---|---|---|---|---|---|---|---|
|
Jira Backlog:
...
Decisions
# | Description | Details | Decisions |
|---|---|---|---|
| 1 | Assume flat attributes (no complex data types) but data could be stored as json object in SPI impl. for convenience/PoC | ||
| 2 | SPI Imp NOT using Model (service) current ENM SPI does! |
Open Issues
Definition
Capabilities
Interface QueryService
Service responsible for dealing with query related features of Data Persistence Service.
- execute
Executes the query and presents the result as an iterator of objects queried for. Will always return the full object from the database.
- executeProjection
Executes the query and presents the result as a list. Returns a list, the type of which is based on the Projection used.
- executeProjection
Executes the query and presents the result as a list. Will return a projection with an List on which the type of data will be an Object array.
- executeCount
Executes the query and presents the result as a count of how many instances are matched against the query.
Note: because of database limitations, at this moment this method is not optimized.
- executeDeletion
Deletes all matching objects and where appropriate deletes any related managed object.
Note: because of database limitations, at this moment this method is not optimized.
Interface QueryPathService
# | Description | Details | Decisions |
|---|---|---|---|
| 1 |
CPS Persistence SPI
https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi
High level structure of ENM SPI
Query SPI
Path - cps-persistence-spi-spi-docs\docs\dps-core-spi\com\ericsson\oss\itpf\datalayer\dps\query\spi
| Are we going to have XPath/query builder? | We should decide based on what the users want to use. | ||
| 2 | Having a fluent interface in my opinion would be way more intuitive and practical. E.g. //AND of two restrictions Restriction and(Restriction firstRestriction, Restriction secondRestriction) //OR of two restrictions Restriction or(Restriction firstRestriction, Restriction secondRestriction) //NOT of a restriction Restriction not(Restriction restriction) This will give any user of the API an intuitive way to build a complex query and the developer of the API a tree to navigate and translate to SQL (or any other query language). A developer could do something the likes of myQuery.setRestriction(restriction1.and(restrcition2).and(restriction3)) | ||
| 3 | SPI should be type safe (the SPI impl. has no access to models)
| We can do validation but still store as a string (JSON object). |
Link to ENM SPI
Link to the SPI can be found here: https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi
High level structure of ENM SPI
This SPI implementation is towards a NoSQL database. The SPI depends on the DPS API and the model service API.
It can be used as a source of ‘inspiration’ however it is important to realize that the SPI implementation was ‘model’ aware - used models heavily to optimize the queries. The SPI implementation also makes a choice of breaking down models to nodes based on MO type and flatten CDTs (complex data types) as properties within the node. There is no concept of MO type or CDTs in Yang so we cannot reuse the same idea.
Query SPI
...
Interface DpsObjectQuery
Specifies methods that are used internally by DPS but are not to be exposed externally to clients.
- isTargetTypeHierarchical
Is the target type hierarchical, i.e. is it able to form containment relationships.
...
Interface DpsTypeContainmentQuery
Aggregates common functionality to type queries and containment queries.
...
Inherits methods from
- DpsQuery
- DpsObjectQuery
- DpsContainmentQuery
- DpsTypeQuery
Query Restriction
cps-persistence-spi-spi-docs\docs\dps-core-spi\com\ericsson\oss\itpf\datalayer\dps\query\restriction\spi
Name
Capabilities
Class AttributeInModelRestriction
Restriction to filter instances of specific versions (model version) only, where the given attribute name is defined.
- negate
Method that is inverting the given restriction
Returns - newly created restriction that is a negation of the base restriction
- getRestrictionType
Indicates the type of restriction type that this restriction represents.
- getAttributeName
Is the restriction negated?
Class AttributePathRestriction
A restriction to compare modeled attributes against path values.
- negate
- getRestrictionType
- getAttribute
- toString
Class AttributeRestriction
Simple restriction involving attribute values.
- getRestrictionType
Class BetweenRestriction
Class to resolve between and not between restrictions.
- negate
- getRestrictionType
- getAttributeName
- getLowerAttributeValue
- getHigherAttributeValue
- isNegated
- toString
Class ContainmentRestriction
Creates a containment restriction applicable for change logs (e.g. get change logs for all managed objects under supplied FDN of MIB root)
- negate
- getRestrictionType
- getStringToMatch
Class DescendantsRestriction
Class to resolve descendants and not descendants restrictions.
- negate
- getRestrictionType
- getDistance
- getDescendantsScope
- isNegated
Interface DpsRestriction
DPS Internal contract for Restriction. All restrictions implementation of DPS should implement DpsRestriction instead of Restriction.
- negate
- getRestrictionType
- isCdtRestriction
- canUseWithOrRestrictions
Class HasMemberRestriction
Creates a HasMember restriction applicable for group entities (e.g. get groups that have a persistence object as member)
- HasMemberRestriction
- negate
- getRestrictionType
- getAttributeName
- getMemberValue
Class InRestriction
Class that creates an IN restriction type, restricting an attribute to a list of values.
- negate
- getRestrictionType
- getAttributeName
- getAllowedValues
- isNegated
- toString
- getString
Class InternalRestriction
Creates an internal restriction applicable for group entities.
- getRestrictionType
Class ListMemberRestriction
A restriction which verifies if a list attribute has members matching specific restrictions.
- matchAllMembers
- negate
- getRestrictionType
- toString
Class ListRestriction
Common functionality of a List restriction.
- getListAttributeName
- getRestrictionToApply
- isNegated
Class LogicalRestriction
Implement logical restriction (AND/OR) and the negation of this logical restriction.
- negate
- getRestrictionType
- getRestrictionList
- getLogicalOperator
- isNegated
- toString
Class NullValueRestriction
Implementation of the null value restriction and not null value restriction.
- negate
- getRestrictionType
- getAttributeName
- isNegated
- toString
Class ObjectFieldBetweenRestriction
Class to resolve between and not between restrictions for ObjectFields.
- getRestrictionType
- getObjectField
Gets the object field that the restriction applies to.
- toString
Class ObjectFieldInRestriction
Class that creates an IN Restriction type, restricting a Object Field to a list of values.
- getRestrictionType
- getObjectField
- toString
Class ObjectFieldPartialMatchRestriction
Implementation of the partial match restriction and not match restriction.
- getRestrictionType
- getObjectField
- toString
Class ObjectFieldPathRestriction
A restriction to compare object fields against path values.
negate
- getRestrictionType
- getObjectField
- toString
Interface ObjectFieldRestriction
A restriction that applies to an object field as opposed to a modelled field.
- getObjectField
Class ObjectFieldRetriever
Some object field restrictions serialized as part of a saved query by earlier versions of the relevant restriction class may be missing the object field when deserialized. This utility class allows the missing object field to be determined from the 'attributeName' that will have been stored instead.
TODO This workaround can be removed once we have given teams time to recreate any object field saved queries they may have.
- retrieve
ObjectField.getFieldName() method.Class ObjectFieldSimpleRestriction
Creates a simple restriction (e.g. compare one field to one value) for object fields.
- getRestrictionType
- getObjectField
- toString
| Name | Definition | Capabilities | |||||
|---|---|---|---|---|---|---|---|
| 1 | Interface QueryService | Service responsible for dealing with query related features of Data Persistence Service. | getTargetObjects
Executes the query | ands gets and presents the result as an | iterable iterator of | 'full' objects queried | .
Query Criterion
cps-persistence-spi-spi-docs\docs\dps-core-spi\com\ericsson\oss\itpf\datalayer\dps\query\criterion\spi
...
Name
...
Definition
...
Capabilities
...
Defines a sorting order which can be applied to a query to order its result set.
- getDpsSortingElement
Retrieves the sorting element used in this sorting order.
- getOrderingDirection
Retrieves the sorting direction used in this sorting order.
- getType
Retrieves the sorting type used in this sorting order.
...
Interface DpsTypeQuery
Specifies common functionality used by type queries.
- getNamespace
Get the namespace of the type of object the type query is to target.
- getType
Get the type of object that the type query is to target.
...
Interface DpsQuery
Specifies methods that are used internally by DPS but are not to be exposed externally to clients.
- getAllSortingOrders
Gets the sorting orders used in this query criteria.
- getQueryCategory
Gets the category of query that this query criteria represents.
- getRestriction
Gets the restriction used in these query criteria, including any DPS internal restrictions added in addition to those set by the user.
- isConversionNeeded
Indicates if conversion is needed for the returned results.
- isDepthFirst
Are the query results to be ordered in a depth first manner?
...
Interface DpsContainmentQuery
Contains common functionality used by containment queries.
- getBaseMoFdn
Retrieve the FDN of the base MO of this query. The base MO is the managed object under which we will look for MOs matching the other criteria of this query.
for. Will always return the full object from the database.
Executes the query and presents the result as a list. Returns a list of the type of which is based on the Projection used.
Executes the query and presents the result as a list. Will return a list of type Object array.
Executes the query and presents the result as a count of how many instances are matched against the query.
Deletes all matching objects and where appropriate deletes any related managed object (children). This method parameters are;
| |||
| 2 | Interface QueryPathService | Service responsible for dealing with path query related features of Data Persistence Service. |
Executes the query ands gets the result as an iterable of 'full' objects queried.
Project the results from points in the query path. Projections allow the user project a single data element from the objects which were found along a query path - these elements can be modelled (i.e. persistence object attributes) or non-modelled (i.e. object fields). |
Query Criterion
Name | Definition | Capabilities | |
|---|---|---|---|
| 1 | Interface SortingOrder | Defines a sorting order which can be applied to a query to sort its result set. |
Retrieves the sorting element used in this sorting order.
Retrieves the sorting direction used in this sorting order.
Retrieves the sorting type used in this sorting order. |
| 2 | Interface DpsTypeQuery | Specifies common functionality used by type queries. |
Retrieves the namespace of the type of object the type query is to target.
Retrieves the type of object that the type query is to target. |
| 3 | Interface DpsQuery | Specifies methods that are used internally by DPS but are not to be exposed externally to clients. |
Gets the sorting orders used in this query criteria.
Gets the category of query that this query criteria represents.
Gets the restriction used in the query criteria, including any DPS internal restrictions added in addition to those set by the user.
Indicates if conversion is needed for the returned results.
Indicates if the query results are to be ordered in a depth first manner. |
| 4 | Interface DpsContainmentQuery | Contains common functionality used by containment queries. |
Retrieve the FDN of the base MO of this query. |
| 5 | Interface DpsObjectQuery | Specifies methods that are used internally by DPS but are not to be exposed externally to clients. |
Indicates if the target type is hierarchical, i.e. is it able to form containment relationships. |
| 6 | Interface DpsTypeContainmentQuery | Aggregates common functionality to type queries and containment queries. | Inherits methods from
|
Query Restriction
Definition
Capabilities
Interface PersistenceLayerPersistenceObject
Exposes functionality only available on the persistence layer objects.
setVersion
Sets the new version of this persistence object.
removeAllAssociations
Remove all associations of this persistence object.
Interface PersistenceLayerManagedObject
Exposes functionality only available on the persistence layer objects.
loadAllChildrenIntoMemory
Used to pre-load children from the database into memory. This method is provided to optimise use cases which will be doing multiple calls to ManagedObject.getChild(String) - it pre-loads all children from the database so that subsequent calls to ManagedObject.getChild(String) will not need round trips to the database.
Class MinimalObjectData
Provides the basic information needed to access an object.
getDbId
Get the native database ID of the object.
getType
Get the type of the object.
getVersion
Get the version of the object.
equals
hashCode
Interface DecoratedPersistenceObject
getNativeDbId
Get the native database identity of the database object backing this persistence object.
Use of this must be done with great care, as it will vary in its behaviour with respect to consistency, volatility, etc. depending on the database being used.
It should never be exposed to applications and is for use within the DPS layer only.
getBucketName
Gets the data bucket name to which this persistence object belongs.
getPersistenceLayerObject
Gets the persistence layer object without any layers of decoration.
If the innermost object is not a persistence layer object then a suitable exception will be thrown.
getDecoratedObject
Gets the object being decorated by the layer of decoration specified.
checkUserSuppliedAttributesAtCreation
Passes the attributes supplied by the user for a new persistence object so that they may be verified as required.
This method will be invoked when creating a new persistence object but should not be used to set the attribute values, that is the purpose of initializeAttributes(java.util.Map<java.lang.String, java.lang.Object>).
The purpose of this method is to allow decorators perform actions on the specific attributes supplied by the user, as opposed to the final set of attributes to be applied on the new object, which may contain merged in default values.
The implementation of this method may not modify the supplied attribute map as they are passed through the layers of decorators. And if the original map is required for later use it should be copied as it may get changed subsequently when initializeAttributes(java.util.Map<java.lang.String, java.lang.Object>) is called.
initializeAttributes
Initializes the attributes for a new persistence object. This method should be invoked when creating a new persistence object rather than using setAttributes() as it will have different behavior, namely:
it verifies all mandatory attributes have been supplied
it initializes all non-supplied attributes to their default value where relevant
no notifications will be sent as a result of this method being called
Name | Definition | Capabilities | |||||||
|---|---|---|---|---|---|---|---|---|---|
| 1 | Class AttributeInModelRestriction | Restriction to filter instances of specific versions (model version) only, where the given attribute name is defined. |
Method that is inverting the given restriction Returns - newly created restriction that is a negation of the base restriction
Retrieves the type of restriction type that this underlying restriction (DpsRestriction) represents.
Retrieves the name of the attribute that the restriction is. | ||||||
| 2 | Class AttributePathRestriction | A restriction to compare modeled attributes against path values. |
| ||||||
| 3 | Class AttributeRestriction | Simple restriction involving attribute values. |
| ||||||
| 4 | Class BetweenRestriction | Class to resolve between and not between restrictions. |
| ||||||
| 5 | Class ContainmentRestriction | Creates a containment restriction applicable for change logs (e.g. get change logs for all managed objects under supplied FDN of MIB root) |
| ||||||
| 6 | Class DescendantsRestriction | Class to resolve descendants and not descendants restrictions. |
| ||||||
| 7 | Interface DpsRestriction | DPS Internal contract for |
| ||||||
| 8 | Class HasMemberRestriction | Creates a HasMember restriction applicable for group entities (e.g. get groups that have a persistence object as member) |
| ||||||
| 9 | Class InRestriction | Class that creates an IN restriction type, restricting an attribute to a list of values. |
| ||||||
| 10 | Class InternalRestriction | Creates an internal restriction applicable for group entities. |
| ||||||
| 11 | Class ListMemberRestriction | A restriction which verifies if a list attribute has members matching specific restrictions. |
| ||||||
| 12 | Class ListRestriction | Common functionality of a List restriction. |
| ||||||
| 13 | Class LogicalRestriction | Implement logical restriction (AND/OR) and the negation of this logical restriction. |
| ||||||
| 14 | Class NullValueRestriction | Implementation of the null value restriction and not null value restriction. |
| ||||||
| 15 | Class ObjectFieldBetweenRestriction | Class to resolve between and not between restrictions for ObjectFields. |
Gets the object field that the restriction applies to.
| ||||||
| 16 | Class ObjectFieldInRestriction | Class that creates an IN Restriction type, restricting a Object Field to a list of values. |
| ||||||
| 17 | Class ObjectFieldPartialMatchRestriction | Implementation of the partial match restriction and not match restriction. |
| ||||||
| 18 | Class ObjectFieldPathRestriction | A restriction to compare object fields against path values. |
| ||||||
| 19 isNegated getStringToMatch toString | 23 | Class PathRestriction | 24 | Class RedundantRestriction |
Gets the flag value of a query, which is using only one, AND-ed redundant restriction or not.
Sets a flag of a query, which is using only one, AND-ed redundant restriction or not. | 25 | Class SimpleRestriction Interface ObjectFieldRestriction | A restriction that applies to an object field as opposed to a modelled field. |
|
| 20 | Class ObjectFieldRetriever | Some object field restrictions serialized as part of a saved query by earlier versions of the relevant restriction class may be missing the object field when deserialized. This utility class allows the missing object field to be determined from the 'attributeName' that will have been stored instead. TODO This workaround can be removed once we have given teams time to recreate any object field saved queries they may have. |
ObjectField.getFieldName() method. | ||||||
| 21 | Class ObjectFieldSimpleRestriction | Creates a simple restriction (e.g. compare one field to one value) for object fields. |
| ||||||
| 22 | Class PartialMatchRestriction | Implementation of the partial match restriction and not match restriction. |
|
Object SPI
Path - cps-persistence-spi-spi-docs\docs\dps-core-spi\com\ericsson\oss\itpf\datalayer\dps\object\spi
| |||
| 23 | Class PathRestriction | Base class for path restrictions. |
|
| 24 | Class RedundantRestriction | Abstract class to add property to check for redundant restriction in query. |
Gets the flag value of a query, which is using only one, AND-ed redundant restriction or not.
Sets a flag of a query, which is using only one, AND-ed redundant restriction or not. |
| 25 | Class SimpleRestriction | Creates a simple restriction (e.g. compare one field to one value). |
|
Object SPI
...
GroupAlreadyDefinedException
...
Class GroupNotAccessibleException
...
GroupNotAccessibleException
...
An exception thrown when a group requested from DPS method does not exist.
...
GroupNotFoundException
Existing SPI in CPS POC
...
| Name | Definition | Capabilities | |
|---|---|---|---|
| 1 | Interface PersistenceLayerPersistenceObject | Exposes functionality only available on the persistence layer objects. |
Sets the new version of this persistence object.
Remove all associations of this persistence object. |
| 2 | Interface PersistenceLayerManagedObject | Exposes functionality only available on the persistence layer objects. |
Used to pre-load children from the database into memory. This method is provided to optimise use cases which will be doing multiple calls to |
| 3 | Class MinimalObjectData | Provides the basic information needed to access an object. |
Retrieves the native database ID of the object.
Retrieves the type of the object.
Retrieves the version of the object.
|
| 4 | Interface DecoratedPersistenceObject | Extends the public persistence object interface with methods that are used internally by DPS but are not to be exposed externally to clients. |
Retrieves the native database identity of the database object backing this persistence object. Use of this must be done with great care, as it will vary in its behaviour with respect to consistency, volatility, etc. depending on the database being used. It should never be exposed to applications and is for use within the DPS layer only.
Gets the data bucket name to which this persistence object belongs.
Gets the persistence layer object without any layers of decoration. If the innermost object is not a persistence layer object then a suitable exception will be thrown.
Gets the object being decorated by the layer of decoration specified.
Passes the attributes supplied by the user for a new persistence object so that they may be verified as required. This method will be invoked when creating a new persistence object but should not be used to set the attribute values, that is the purpose of The purpose of this method is to allow decorators perform actions on the specific attributes supplied by the user, as opposed to the final set of attributes to be applied on the new object, which may contain merged in default values. The implementation of this method may not modify the supplied attribute map as they are passed through the layers of decorators.
Removes backpointer from this persistence object to the persistence object which is referencing it by uni-directional with backpointer association.
Sets backpointer from this persistence object to the persistence object which is referencing it by uni-directional with backpointer association.
Retrieves referencing persistence objects for backpointer names for all uni-directional with backpointer associations referencing this persistence object.
Gets the values for the attributes which have been specifically set on this object. This includes attributes which:
Note that an attribute may have been specifically set to null so the values returned in the map may contain nulls.
Determines if the supplied attribute has been specifically set (includes attributes which were set to null).
Gets the primary type specification for this object.
Gets change identifier for this persistence object.
Gets all of the attribute names available in this persistence object.
Unsets the values of the supplied attributes. |
| 5 | Class BasicManagedObjectData | Used to efficiently transfer basic information about a Managed Object - without the full Managed Object and without the bloat of the existing Data Transfer Objects (DTOs). |
| 6 | Extends the public managed object interface with methods that are used internally by DPS but are not to be exposed externally to clients. |
Object Exception
Path - cps-persistence-spi-spi-docs\docs\dps-core-spi\com\ericsson\oss\itpf\datalayer\dps\object\spi\exception
...
Definition
...
Capabilities
...
Class AssociationAlreadyDefinedException
Exception thrown when the user tries to create an association that already exists.
...
AssociationAlreadyDefinedException
...
Class ChildAlreadyDefinedException
...
Exception thrown when a new child object is attempted to be created under a parent which already has a child with that type and name.
...
ChildAlreadyDefinedException
And if the original map is required for later use it should be copied as it may get changed subsequently when
Initializes the attributes for a new persistence object. This method should be invoked when creating a new persistence object rather than using setAttributes() as it will have different behavior, namely:
The implementation of this method may modify the supplied attribute map as they are passed through the layers of decorators.
Removes backpointer from this persistence object to the persistence object which is referencing it by uni-directional with backpointer association.
Sets backpointer from this persistence object to the persistence object which is referencing it by uni-directional with backpointer association.
Retrieves referencing persistence objects for backpointer names for all uni-directional with backpointer associations referencing this persistence object.
Gets the values for the attributes which have been specifically set on this object. This includes attributes which:
Note that an attribute may have been specifically set to null so the values returned in the map may contain nulls.
Determines if the supplied attribute has been specifically set (includes attributes which were set to null).
Gets the primary type specification for this object.
Gets change identifier for this persistence object.
Gets all of the attribute names available in this persistence object.
Unsets the values of the supplied attributes. | |||
| 5 | Class BasicManagedObjectData | Used to efficiently transfer basic information about a Managed Object - without the full Managed Object and without the bloat of the existing Data Transfer Objects (DTOs). |
|
| 6 | Interface DecoratedManagedObject | Extends the public managed object interface with methods that are used internally by DPS but are not to be exposed externally to clients. |
Retrieves the RDN of the managed object.
Returns the existing amount of the managed object of the type referenced by type namespace and name.
Sets the active choice case for the given choice.
Retrieves the active choice case for the given choice.
Retrieves all the children managed object of a given type. |
Object Exception
| Name | Definition | Capabilities | |
|---|---|---|---|
| 1 |
Store the JSON structure in the database.
Get the JSON structure from the database using the entity identifier.
Delete the JSON structure from the database using the entity identifier. |
Model SPI
...
Definition
...
Capabilities
...
Interface ModelPersistencyService
Responsible for accessing and manipulating module data using the chosen database solution
...
- storeModule
Store the module from a yang model in the database.
...
Class AssociationAlreadyDefinedException | Exception thrown when the user tries to create an association that already exists. |
| |
| 2 | Class ChildAlreadyDefinedException | Exception thrown when a new child object is attempted to be created under a parent which already has a child with that type and name. |
|
| 3 | Class GroupAlreadyDefinedException | An attempt was made to create a Group with non-unique namespace and name. |
|
| 4 | Class GroupNotAccessibleException | An exception thrown when the requested group is not accessible to current user. |
|
| 5 | Class GroupNotFoundException | An exception thrown when a group requested from DPS method does not exist. |
|