CCSDK-2752: DM: [Spike] Examine DPS SPI

Study the existing SPI used in ENM to migrate from Versant to Graph DB in ENM implementation. 

Learning from this SPI can be used to propose an SPI for the CPS SPI that also allows to 'plug-in' different DBs

Jira Ticket:

CPS-9: DM: [Spike] Examine DPS SPI, propose and agree CPS SPIClosed

Decisions

#

Description

Details

Decisions

#

Description

Details

Decisions

1

Assume flat attributes (no complex data types) but data could be stored as json object in SPI impl. for convenience/PoC





2

SPI Imp NOT using Model (service) current ENM SPI does!





Open Issues

#

Description

Details

Decisions

#

Description

Details

Decisions

1

Are we going to have XPath/query builder?

We should decide based on what the users want to use.



2

Having a fluent interface in my opinion would be way more intuitive and practical. E.g.

//AND of two restrictions

Restriction and(Restriction firstRestriction,  Restriction secondRestriction)

//OR of two restrictions

Restriction or(Restriction firstRestriction,  Restriction secondRestriction)

//NOT of a restriction

Restriction not(Restriction restriction)

This will give any user of the API an intuitive way to build a complex query and the developer of the API a tree to navigate and translate to SQL (or any other query language). A developer could do something the likes of myQuery.setRestriction(restriction1.and(restrcition2).and(restriction3))





3

SPI should be type safe (the SPI impl. has no access to models)

  1.  

    1. All numeric types could be passed as ‘long’ in Java

We can do validation but still store as a string (JSON object).



Link to ENM SPI

Link to the SPI can be found here: https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi



High level structure of ENM SPI

This SPI implementation is towards a NoSQL database. The SPI depends on the DPS API and the model service API. 

It can be used as a source of ‘inspiration’ however it is important to realize that the SPI implementation was ‘model’ aware - used models heavily to optimize the queries. The SPI implementation also makes a choice of breaking down models to nodes based on MO type and flatten CDTs (complex data types) as properties within the node. There is no concept of MO type or CDTs in Yang so we cannot reuse the same idea. 

Query SPI 

https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi/com/ericsson/oss/itpf/datalayer/dps/query/spi



Name



Definition

 Capabilities



Name



Definition

 Capabilities

1

Interface QueryService 

Service responsible for dealing with query related features of Data Persistence Service.



  • execute

Executes the query and presents the result as an iterator of objects queried for. Will always return the full object from the database.

  • executeProjection

Executes the query and presents the result as a list. Returns a list of  the type of which is based on the Projection used.

  • executeProjection

Executes the query and presents the result as a list. Will return a list of type Object array.

  • executeCount

Executes the query and presents the result as a count of how many instances are matched against the query.
Note: because of database limitations, at this moment this method is not optimized.

  • executeDeletion

Deletes all matching objects and where appropriate deletes any related managed object (children).
Note: because of database limitations, at this moment this method is not optimized.

This method parameters are;

  1. query - query that identifies the objects to be deleted.

  2. bucketName - the name of the data bucket in which the query will be executed against.

2

Interface QueryPathService

Service responsible for dealing with path query related features of Data Persistence Service.

  • getTargetObjects

Executes the query ands gets the result as an iterable of 'full' objects queried. 

  • projectResults

Project the results from points in the query path. Projections allow the user project a single data element from the objects which were found along a query path - these elements can be modelled (i.e. persistence object attributes) or non-modelled (i.e. object fields).



Query Criterion

https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi/com/ericsson/oss/itpf/datalayer/dps/query/criterion/spi



Name

Definition

 Capabilities



Name

Definition

 Capabilities

1

Interface SortingOrder

Defines a sorting order which can be applied to a query to sort its result set.



  • getDpsSortingElement

Retrieves the sorting element used in this sorting order.

  • getOrderingDirection

Retrieves the sorting direction used in this sorting order.

  • getType

Retrieves the sorting type used in this sorting order.

2

Interface DpsTypeQuery

Specifies common functionality used by type queries.



  • getNamespace

Retrieves  the namespace of the type of object the type query is to target.

  • getType

Retrieves the type of object that the type query is to target.

3

Interface DpsQuery

Specifies methods that are used internally by DPS but are not to be exposed externally to clients.



  • getAllSortingOrders

Gets the sorting orders used in this query criteria.

  • getQueryCategory

Gets the category of query that this query criteria represents.

  • getRestriction

Gets the restriction used in the query criteria, including any DPS internal restrictions added in addition to those set by the user.

  • isConversionNeeded

Indicates if conversion is needed for the returned results.

  • isDepthFirst

Indicates if the query results are to be  ordered in a depth first manner.

4

Interface DpsContainmentQuery

Contains common functionality used by containment queries.



  • getBaseMoFdn

Retrieve the FDN of the base MO of this query.



5

Interface DpsObjectQuery

Specifies methods that are used internally by DPS but are not to be exposed externally to clients.



  • isTargetTypeHierarchical

Indicates if the target type is hierarchical, i.e. is it able to form containment relationships.



6

 Interface DpsTypeContainmentQuery

Aggregates common functionality to type queries and containment queries.



Inherits methods from

  • DpsQuery

  • DpsObjectQuery

  • DpsContainmentQuery

  • DpsTypeQuery



Query Restriction

https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi/com/ericsson/oss/itpf/datalayer/dps/query/restriction/spi



Name

Definition

 Capabilities



Name

Definition

 Capabilities

1

Class AttributeInModelRestriction

Restriction to filter instances of specific versions (model version) only, where the given attribute name is defined.

  • negate

Method that is inverting the given restriction

Returns - newly created restriction that is a negation of the base restriction

  • getRestrictionType

Retrieves the type of restriction type that this underlying restriction (DpsRestriction) represents.

  • getAttributeName

Retrieves the name of the attribute that the restriction is.





2

Class AttributePathRestriction

A restriction to compare modeled attributes against path values.

  • negate

  • getRestrictionType

  • getAttribute

  • toString

3

Class AttributeRestriction

Simple restriction involving attribute values.

  • getRestrictionType

4

Class BetweenRestriction

Class to resolve between and not between restrictions.

  • negate

  • getRestrictionType

  • getAttributeName

  • getLowerAttributeValue

  • getHigherAttributeValue

  • isNegated

  • toString

5

Class ContainmentRestriction

Creates a containment restriction applicable for change logs (e.g. get change logs for all managed objects under supplied FDN of MIB root)

  • negate

  • getRestrictionType

  • getStringToMatch

6

Class DescendantsRestriction

Class to resolve descendants and not descendants restrictions.

  • negate

  • getRestrictionType

  • getDistance

  • getDescendantsScope

  • isNegated

7

Interface DpsRestriction

DPS Internal contract for RestrictionAll restrictions implementation of DPS should implement DpsRestriction instead of Restriction.

  • negate

  • getRestrictionType

  • isCdtRestriction

  • canUseWithOrRestrictions

8

Class HasMemberRestriction

Creates a HasMember restriction applicable for group entities (e.g. get groups that have a persistence object as member)

  • HasMemberRestriction

  • negate

  • getRestrictionType

  • getAttributeName

  • getMemberValue

9

Class InRestriction

Class that creates an IN restriction type, restricting an attribute to a list of values.

  • negate

  • getRestrictionType

  • getAttributeName

  • getAllowedValues

  • isNegated

  • toString

  • getString

10

Class InternalRestriction

Creates an internal restriction applicable for group entities.

  • getRestrictionType

11

Class ListMemberRestriction

A restriction which verifies if a list attribute has members matching specific restrictions.

  • matchAllMembers

  • negate

  • getRestrictionType

  • toString

12

Class ListRestriction

Common functionality of a List restriction.

  • getListAttributeName

  • getRestrictionToApply

  • isNegated

13

Class LogicalRestriction

Implement logical restriction (AND/OR) and the negation of this logical restriction.

  • negate

  • getRestrictionType

  • getRestrictionList

  • getLogicalOperator

  • isNegated

  • toString

14

Class NullValueRestriction

Implementation of the null value restriction and not null value restriction.

  • negate

  • getRestrictionType

  • getAttributeName

  • isNegated

  • toString

15

Class ObjectFieldBetweenRestriction

Class to resolve between and not between restrictions for ObjectFields.

  • getRestrictionType

  • getObjectField

Gets the object field that the restriction applies to.

  • toString

16

Class ObjectFieldInRestriction

Class that creates an IN Restriction type, restricting a Object Field to a list of values.



  • getRestrictionType

  • getObjectField

  • toString

17

Class ObjectFieldPartialMatchRestriction

Implementation of the partial match restriction and not match restriction.



  • getRestrictionType

  • getObjectField

  • toString

18

Class ObjectFieldPathRestriction

A restriction to compare object fields against path values.



  • negate

  • getRestrictionType

  • getObjectField

  • toString

19

Interface ObjectFieldRestriction

A restriction that applies to an object field as opposed to a modelled field.

  • getObjectField

20

Class ObjectFieldRetriever

Some object field restrictions serialized as part of a saved query by earlier versions of the relevant restriction class may be missing the object field when deserialized. This utility class allows the missing object field to be determined from the 'attributeName' that will have been stored instead.

TODO This workaround can be removed once we have given teams time to recreate any object field saved queries they may have.

  • retrieve

Retrieves the object field associated with the supplied field name.This method performs the reverse of the now deprecated ObjectField.getFieldName() method.

21

Class ObjectFieldSimpleRestriction

Creates a simple restriction (e.g. compare one field to one value) for object fields.



  • getRestrictionType

  • getObjectField

  • toString

22



Class PartialMatchRestriction



Implementation of the partial match restriction and not match restriction.



  • negate

  • getRestrictionType

  • getMatchConditionType

  • getAttributeName

  • isNegated

  • getStringToMatch

  • toString

23

Class PathRestriction

Base class for path restrictions.



  • getValueToMatch

  • getOperator

  • getString



24

Class RedundantRestriction

Abstract class to add property to check for redundant restriction in query.



  • isRedundant

Gets the flag value of a query, which is using only one, AND-ed redundant restriction or not.

  • setRedundant

Sets a flag of a query, which is using only one, AND-ed redundant restriction or not.

25

Class SimpleRestriction

Creates a simple restriction (e.g. compare one field to one value).



  • negate

  • getAttributeName

  • getAttributeValue

  • getSimpleExpressionOperator

  • toString



Object SPI 

https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi/com/ericsson/oss/itpf/datalayer/dps/object/spi



Name



Definition

 Capabilities



Name



Definition

 Capabilities

1

Interface PersistenceLayerPersistenceObject

Exposes functionality only available on the persistence layer objects.



  • setVersion

Sets the new version of this persistence object.

  • removeAllAssociations

Remove all associations of this persistence object.



2

Interface PersistenceLayerManagedObject

Exposes functionality only available on the persistence layer objects.



  • loadAllChildrenIntoMemory

Used to pre-load children from the database into memory. This method is provided to optimise use cases which will be doing multiple calls to ManagedObject.getChild(String) - it pre-loads all children from the database so that subsequent calls to ManagedObject.getChild(String) will not need round trips to the database.

3

Class MinimalObjectData



Provides the basic information needed to access an object.

  • getDbId

Retrieves the native database ID of the object.

  • getType

Retrieves the type of the object.

  • getVersion

Retrieves the version of the object.

  • equals

  • hashCode

4

Interface DecoratedPersistenceObject



Extends the public persistence object interface with methods that are used internally by DPS but are not to be exposed externally to clients.

  • getNativeDbId

Retrieves the native database identity of the database object backing this persistence object.

Use of this must be done with great care, as it will vary in its behaviour with respect to consistency, volatility, etc. depending on the database being used.

It should never be exposed to applications and is for use within the DPS layer only.

  • getBucketName

Gets the data bucket name to which this persistence object belongs.

  • getPersistenceLayerObject

Gets the persistence layer object without any layers of decoration.

If the innermost object is not a persistence layer object then a suitable exception will be thrown.

  • getDecoratedObject

Gets the object being decorated by the layer of decoration specified.

  • checkUserSuppliedAttributesAtCreation

Passes the attributes supplied by the user for a new persistence object so that they may be verified as required.

This method will be invoked when creating a new persistence object but should not be used to set the attribute values, that is the purpose of initializeAttributes(java.util.Map<java.lang.String, java.lang.Object>).

The purpose of this method is to allow decorators perform actions on the specific attributes supplied by the user, as opposed to the final set of attributes to be applied on the new object, which may contain merged in default values.

The implementation of this method may not modify the supplied attribute map as they are passed through the layers of decorators. And if the original map is required for later use it should be copied as it may get changed subsequently when initializeAttributes(java.util.Map<java.lang.String, java.lang.Object>) is called.

  • initializeAttributes

Initializes the attributes for a new persistence object. This method should be invoked when creating a new persistence object rather than using setAttributes() as it will have different behavior, namely:

  1. it verifies all mandatory attributes have been supplied

  2. it initializes all non-supplied attributes to their default value where relevant

  3. no notifications will be sent as a result of this method being called

The implementation of this method may modify the supplied attribute map as they are passed through the layers of decorators.

  • removeBackpointer

Removes backpointer from this persistence object to the persistence object which is referencing it by uni-directional with backpointer association.

  • setBackpointer

Sets backpointer from this persistence object to the persistence object which is referencing it by uni-directional with backpointer association.

  • getBackpointerAssociations

Retrieves referencing persistence objects for backpointer names for all uni-directional with backpointer associations referencing this persistence object.

  • getAttributesSpecificallySet

Gets the values for the attributes which have been specifically set on this object.

This includes attributes which:

  1. were supplied at creation of the object or

  2. had default values applied during creation or

  3. were specifically set after creation by invoking one of the setAttribute(s) methods.

Note that an attribute may have been specifically set to null so the values returned in the map may contain nulls.

  • isAttributeSet

Determines if the supplied attribute has been specifically set (includes attributes which were set to null).

  • getPrimaryTypeSpecification

Gets the primary type specification for this object.

  • getChangeIdentifier

Gets change identifier for this persistence object.

  • getAttributeNames

Gets all of the attribute names available in this persistence object.

  • unsetAttributes

Unsets the values of the supplied attributes.



5

Class BasicManagedObjectData

Used to efficiently transfer basic information about a Managed Object - without the full Managed Object and without the bloat of the existing Data Transfer Objects (DTOs).

  • getFdn

  • getNamespace

  • getType

  • getName

  • getVersion

  • getPoId

  • equals

  • hashCode

6

Interface DecoratedManagedObject



Extends the public managed object interface with methods that are used internally by DPS but are not to be exposed externally to clients.

  • getRdn

Retrieves the RDN of the managed object.

  • getChildrenCardinality

Returns the existing amount of the managed object of the type referenced by type namespace and name.

  • setActiveChoiceCase

Sets the active choice case for the given choice.

  • getActiveChoiceCase

Retrieves the active choice case for the given choice.

  • getChildrenByType

Retrieves all the children managed object of a given type.





Object Exception

https://github.com/Ericsson/cps-persistence-spi/tree/spi-docs/docs/dps-core-spi/com/ericsson/oss/itpf/datalayer/dps/object/spi/exception



Name



Definition

 Capabilities



Name



Definition

 Capabilities

1

Class AssociationAlreadyDefinedException

Exception thrown when the user tries to create an association that already exists.



  • AssociationAlreadyDefinedException

2

Class ChildAlreadyDefinedException



Exception thrown when a new child object is attempted to be created under a parent which already has a child with that type and name.

  • ChildAlreadyDefinedException

3

Class GroupAlreadyDefinedException

An attempt was made to create a Group with non-unique namespace and name.

  • GroupAlreadyDefinedException

4

Class GroupNotAccessibleException



An exception thrown when the requested group is not accessible to current user.

  • GroupNotAccessibleException

5

Class GroupNotFoundException

An exception thrown when a group requested from DPS method does not exist.



  • GroupNotFoundException