Please note that while Dublin release is in development, the tutorial steps may be out-of-sync with the current code.
TBC NEED STEPS FOR NEW SCHEMA-SERVICE COMPONENT HERE
- Set up a development environment as described in AAI Developer Environment Setup - Dublin
- This document will show editing schema files with Eclipse, but Eclipse is not required.
- In this example, we will add 2 new attributes to the cloud-region node type and a new node type called "new-widget" which will be a child node of "cloud-region"
- Adding a new attribute to "cloud-region"
- Open aai/schema-service, aai-schema/src/main/resources/oxm/aai_oxm_v11.xml
Add the following attributes between "complexName" and "resourceVersion":
<xml-element java-attribute="complexName" name="complex-name" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="complex name for cloud-region instance. NOTE - THIS FIELD IS NOT CORRECTLY POPULATED." />
</xml-properties>
</xml-element>
<xml-element java-attribute="newAttributeForDemo" name="new-attribute-for-demo" required="true" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="Example new attribute for cloud-region instance. " />
</xml-properties>
</xml-element>
<xml-element java-attribute="numberAttributeForDemo" name="number-attribute-for-demo" required="true" type="java.lang.Integer">
<xml-properties>
<xml-property name="description" value="Example number attribute for cloud-region instance. " />
</xml-properties>
</xml-element>
<xml-element java-attribute="resourceVersion" name="resource-version" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="Used for optimistic concurrency. Must be empty on create, valid on update and delete." />
</xml-properties>
</xml-element>Do not try to add a new attribute with ‘xml-key=”true”’ because it creates complications when the key value is already set!
Existing classes in the schema will already have an xml-key set.
Compare with the "NewWidget" example below. As a new class in the schema, NewWidget must define a new attribute with the xml-key set.
Save the file, and rebuild the libraries and microservices:
here's my example file: aai_oxm_v11.xml- Rebuild schema-service first:
- $ cd ~/LF/AAI/schema-service
$ mvn versions:set -DnewVersion=0.0.1-TEST-SNAPSHOT
$ mvn clean install
Should result in BUILD SUCCESS - $ cd ~/LF/AAI/graphadmin
- $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should result in BUILD SUCCESS - $ cd ~/LF/AAI/resources
- $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should result in BUILD SUCCESS - $ cd ~/LF/AAI/traversal
mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT Should result in BUILD SUCCESS
Sometimes you may be tempted to use "-DskipTests" in the mvn command line to save some time, but it is known that some kinds of schema mistakes are detected only at run-time, not compile-time, so running the tests is a good indicator.
Run GenTester, using the target dir under aai-resources:
$ (cd graphadmin/ && mvn -PrunAjsc -Dstart-class=org.onap.aai.schema.GenTester -Daai.schema.version=0.0.1-TEST-SNAPSHOT -DskipTests -Dcheckstyle.skip=true -DAJSC_HOME=$HOME/src/aai/graphadmin -DBUNDLECONFIG_DIR=src/main/resources)
You should see the following output:
---- NOTE --- about to open graph (takes a little while)--------;
-- loading schema into JanusGraph
-- loading schema into JanusGraph
-- Loading new schema elements into JanusGraph --
-- graph commit
-- graph shutdown
You can check the logs to see if the graph elements were created successfully and grep for your property
cd graphadmin/logs/createDBSchema/;
grep 'complex-name' metrics.log
2018-12-06T04:43:18.061+0000|2018-12-06T04:43:42.960+0000|449b8583-54c1-42bf-8ada-5b75c52d7a13||main ||AAI|AAI-TOOLS|AAI|main|COMPLETE|0|||INFO||127.0.1.1|24899|localhost||org.onap.a
ai.dbgen.SchemaGenerator||||||||co=DBGenTester:Creating PropertyKey: [complex-name], [String], [SINGLE]
2018-12-06T04:43:18.061+0000|2018-12-06T04:43:42.960+0000|449b8583-54c1-42bf-8ada-5b75c52d7a13||main ||AAI|AAI-TOOLS|AAI|main|COMPLETE|0|||INFO||127.0.1.1|24899|localhost||org.onap.a
ai.dbgen.SchemaGenerator||||||||co=DBGenTester:Add index for PropertyKey: [complex-name]- Start the "resources" microservice
- Resources runs on port 8447. Go to the resources directory
$ cd ~/LF/AAI/resources - Set the debug port to 9447
$ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n" - Start the microservice
$ mvn -pl aai-resources -P runAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
- Resources runs on port 8447. Go to the resources directory
- Start the "traversal" microservice
- Traversal runs on port 8446. Go to the traversal directory
$ cd ~/LF/AAI/traversal - Set the debug port to 9446
$ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9446,server=y,suspend=n" - Start the microservice
$ mvn -pl aai-traversal -PrunAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should see something like this: Traversal Microservice Started
- Traversal runs on port 8446. Go to the traversal directory
- Start the "graphadmin" microservice
- Graphadmin runs on port 8449. Go to the graphadmin directory
$ cd ~/LF/AAI/graphadmin - Set the debug port to 9449
$ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9449,server=y,suspend=n" - Start the microservice
$ mvn -PrunAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should see something like this: GraphAdmin Microservice Started
- Graphadmin runs on port 8449. Go to the graphadmin directory
Get an example cloud-region object, postman: Cloud-Region Example.postman_collection.json
You should see the new attributes on the example object, as highlighted in the red rectangle above
- Copy and paste the example object. Note that the following attributes need to be removed from your new object, if present in the example data:
- model-invariant-id (this is a foreign key to SDC model data, which will not exist in this tutorial)
- model-customization-id (this is a foreign key to SDC model data, which will not exist in this tutorial)
- model-version-id (this is a foreign key to SDC model data, which will not exist in this tutorial)
- resource-version (this value must be empty when creating a new object)
PUT your new object data to persist the new attributes. Use this postman collection during the next 2 steps as well.
- Check the object by doing a GET: GET Cloud-Region (no depth param) in postman collection
- Add the depth parameter: GET Cloud-Region (depth = all) in postman collection
- This indicates that the schema has been updated with the new attributes and they are making it to the database
- This indicates that the schema has been updated with the new attributes and they are making it to the database
- Clear the cloud region and then delete it by using "Clear Cloud-Region" and "Delete Cloud-Region"
- note: you will need to perform a GET each time and update the resource version of the cloud-region in the XML payload for the clear, and do the GET again and update the resource-version QueryParam to perform
the delete
- note: you will need to perform a GET each time and update the resource version of the cloud-region in the XML payload for the clear, and do the GET again and update the resource-version QueryParam to perform
Run PUT Cloud-Region - missing attr. This will try to PUT the object without the required attribute we defined in the schema, and the response will look like this:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Fault>
<requestError>
<serviceException>
<messageId>SVC3000</messageId>
<text>Invalid input performing %1 on %2 (msg=%3) (ec=%4)</text>
<variables>
<variable>PUT</variable>
<variable>v11/cloud-infrastructure/cloud-regions/cloud-region/example-cloud-owner-val-44086/example-cloud-region-id-val-67393</variable>
<variable>Invalid input performing %1 on %2:Missing required property: new-attribute-for-demo</variable>
<variable>ERR.5.2.3000</variable>
</variables>
</serviceException>
</requestError>
</Fault>Adding a new node type. First, add the container under the top-level inventory category. Adding "newWidgets" to "Network" so the URI will be "aai/v11/network/new-widgets/new-widget/{new-widget-name}"
<java-type name="Network">
<xml-properties>
<xml-property name="description" value="Namespace for network inventory resources." />
</xml-properties>
<xml-root-element name="network" />
<java-attributes>
<xml-element java-attribute="logicalLinks" name="logical-links" type="inventory.aai.onap.org.v11.LogicalLinks" />
<xml-element java-attribute="sitePairSets" name="site-pair-sets" type="inventory.aai.onap.org.v11.SitePairSets" />
<xml-element java-attribute="vpnBindings" name="vpn-bindings" type="inventory.aai.onap.org.v11.VpnBindings" />
<xml-element java-attribute="vplsPes" name="vpls-pes" type="inventory.aai.onap.org.v11.VplsPes" />
<xml-element java-attribute="multicastConfigurations" name="multicast-configurations" type="inventory.aai.onap.org.v11.MulticastConfigurations" />
<xml-element java-attribute="vces" name="vces" type="inventory.aai.onap.org.v11.Vces" />
<xml-element java-attribute="vpes" name="vpes" type="inventory.aai.onap.org.v11.Vpes" />
<xml-element java-attribute="vnfcs" name="vnfcs" type="inventory.aai.onap.org.v11.Vnfcs" />
<xml-element java-attribute="l3Networks" name="l3-networks" type="inventory.aai.onap.org.v11.L3Networks" />
<xml-element java-attribute="networkPolicies" name="network-policies" type="inventory.aai.onap.org.v11.NetworkPolicies" />
<xml-element java-attribute="genericVnfs" name="generic-vnfs" type="inventory.aai.onap.org.v11.GenericVnfs" />
<xml-element java-attribute="lagLinks" name="lag-links" type="inventory.aai.onap.org.v11.LagLinks" />
<xml-element java-attribute="newvces" name="newvces" type="inventory.aai.onap.org.v11.Newvces" />
<xml-element java-attribute="pnfs" name="pnfs" type="inventory.aai.onap.org.v11.Pnfs" />
<xml-element java-attribute="physicalLinks" name="physical-links" type="inventory.aai.onap.org.v11.PhysicalLinks" />
<xml-element java-attribute="newWidgets" name="new-widgets" type="inventory.aai.onap.org.v11.NewWidgets" />
<xml-element java-attribute="ipsecConfigurations" name="ipsec-configurations" type="inventory.aai.onap.org.v11.IpsecConfigurations" />
<xml-element java-attribute="routeTableReferences" name="route-table-references" type="inventory.aai.onap.org.v11.RouteTableReferences" />
<xml-element java-attribute="instanceGroups" name="instance-groups" type="inventory.aai.onap.org.v11.InstanceGroups" />
<xml-element java-attribute="zones" name="zones" type="inventory.aai.onap.org.v11.Zones" />
</java-attributes>
</java-type>Set up the "NewWidgets" java type:
<java-type name="NewWidgets">
<xml-properties>
<xml-property name="description" value="Collection of new Widgets" />
</xml-properties>
<xml-root-element name="new-widgets" />
<java-attributes>
<xml-element container-type="java.util.ArrayList" java-attribute="newWidget" name="new-widget" type="inventory.aai.onap.org.v11.NewWidget" />
</java-attributes>
</java-type>Set up the NewWidget java type:
<java-type name="NewWidget">
<xml-root-element name="new-widget" />
<java-attributes>
<xml-element java-attribute="newWidgetName" name="new-widget-name" required="true" type="java.lang.String" xml-key="true">
<xml-properties>
<xml-property name="description" value="e.g.,awesome-new-widget, terrific-new-widget" />
</xml-properties>
</xml-element>
<xml-element java-attribute="newWidgetType" name="new-widget-type" required="true" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="Type of new Widget, e.g., fantastic, amazing" />
</xml-properties>
</xml-element>
<xml-element java-attribute="resourceVersion" name="resource-version" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="Used for optimistic concurrency. Must be empty on create, valid on update and delete." />
</xml-properties>
</xml-element>
<xml-element java-attribute="modelInvariantId" name="model-invariant-id" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="the ASDC model id for this resource or service model." />
<xml-property name="visibility" value="deployment" />
<xml-property name="requires" value="model-version-id" />
<xml-property name="dbAlias" value="model-invariant-id-local" />
</xml-properties>
</xml-element>
<xml-element java-attribute="modelVersionId" name="model-version-id" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="the ASDC model version for this resource or service model." />
<xml-property name="visibility" value="deployment" />
<xml-property name="requires" value="model-invariant-id" />
<xml-property name="dbAlias" value="model-version-id-local" />
</xml-properties>
</xml-element>
<xml-element java-attribute="newWidgetId" name="new-widget-id" type="java.lang.String">
<xml-properties>
<xml-property name="description" value="ID of the newWidget" />
</xml-properties>
</xml-element>
<xml-element java-attribute="relationshipList" name="relationship-list" type="inventory.aai.onap.org.v11.RelationshipList" />
</java-attributes>
<xml-properties>
<xml-property name="description" value="New widgets are the best widgets. Our new widgets are really, really great."/>
<xml-property name="indexedProps" value="new-widget-id,new-widget-name,model-invariant-id,model-version-id" />
<xml-property name="uniqueProps" value="new-widget-id" />
<xml-property name="container" value="new-widgets" />
<xml-property name="namespace" value="network" />
<xml-property name="searchable" value="new-widget-name,new-widget-id" />
</xml-properties>
</java-type>Save the file, and rebuild the libraries and microservices:
here's my example file: aai_oxm_v11.xml- Rebuild schema-service first:
- $ cd ~/LF/AAI/schema-service
$ mvn versions:set -DnewVersion=0.0.1-TEST-SNAPSHOT
$ mvn clean install
Should result in BUILD SUCCESS - $ cd ~/LF/AAI/graphadmin
- $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should result in BUILD SUCCESS - $ cd ~/LF/AAI/resources
- $ mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should result in BUILD SUCCESS - $ cd ~/LF/AAI/traversal
mvn clean install -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT - Should result in BUILD SUCCESS
Run GenTester, using the target dir under aai-resources:
$ cd ~/LF/AAI/graphadmin;
java -DAJSC_HOME=. -DBUNDLECONFIG_DIR=src/main/resources/ -Dloader.main=org.onap.aai.schema.GenTester -Dloader.path=./src/main/resources -Dschema.ingest.file=src/main/resources/application.properties -jar target/aai-graphadmin-*.jar
You should see the following output:
---- NOTE --- about to open graph (takes a little while)--------;
-- loading schema into JanusGraph
-- loading schema into JanusGraph
-- Loading new schema elements into JanusGraph --
-- graph commit
-- graph shutdown
You can check the logs to see if the graph elements were created successfully and grep for your propertycd logs/createDBSchema/;
grep 'new-widget-name' metrics.logCreating PropertyKey: [new-widget-name], [String], [SINGLE]
[DEV: 2017-Jul-28 08:44:12,287][INFO ][main ]Creating PropertyKey: [new-widget-name], [String], [SINGLE]
Add index for PropertyKey: [new-widget-name]
[DEV: 2017-Jul-28 08:44:12,289][INFO ][main ]Add index for PropertyKey: [new-widget-name]
Creating PropertyKey: [new-widget-type], [String], [SINGLE]
[DEV: 2017-Jul-28 08:44:12,291][INFO ][main ]Creating PropertyKey: [new-widget-type], [String], [SINGLE]
No index added for PropertyKey: [new-widget-type]
[DEV: 2017-Jul-28 08:44:12,501][INFO ][main ]No index added for PropertyKey: [new-widget-type]
PropertyKey [resource-version] already existed in the DB.
PropertyKey [model-invariant-id] already existed in the DB.
PropertyKey [model-version-id] already existed in the DB.
Creating PropertyKey: [new-widget-id], [String], [SINGLE]
[DEV: 2017-Jul-28 08:44:12,501][INFO ][main ]Creating PropertyKey: [new-widget-id], [String], [SINGLE]- Start the "resources" microservice
- Resources runs on port 8447. Go to the resources directory
$ cd ~/LF/AAI/resources - Set the debug port to 9447
$ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n" - Start the microservice
$ mvn -pl aai-resources -P runAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should see something like this: Resources Microservice Started
- Resources runs on port 8447. Go to the resources directory
- Start the "traversal" microservice
- Traversal runs on port 8446. Go to the traversal directory
$ cd ~/LF/AAI/traversal - Set the debug port to 9446
$ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9447,server=y,suspend=n" - Start the microservice
$ mvn -pl aai-traversal -P runAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should see something like this: Traversal Microservice Started
- Traversal runs on port 8446. Go to the traversal directory
- Start the "graphadmin" microservice
- Graphadmin runs on port 8449. Go to the graphadmin directory
$ cd ~/LF/AAI/graphadmin - Set the debug port to 9449
$ export MAVEN_OPTS="-Xms1024m -Xmx5120m -XX:PermSize=2024m -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9449,server=y,suspend=n" - Start the microservice
$ mvn -PrunAjsc -Daai.core.version=0.0.1-TEST-SNAPSHOT -Daai.schema.version=0.0.1-TEST-SNAPSHOT -Daai.schema.ingest.version=0.0.1-TEST-SNAPSHOT
Should see something like this: GraphAdmin Microservice Started
- Graphadmin runs on port 8449. Go to the graphadmin directory
- Get an example newWidget using "New Widget Example" (find these examples in this postman collection: New-Widget Example.postman_collection.json)
- Copy the new widget and PUT it: (PUT New Widget)
- Success! We have updated an existing object type and added a new object type. Next, we will set up an edge between the CloudRegion and the NewWidget
- Open DbEdgeRules_v11.json
File is aai-common/aai-core/src/main/resources/dbedgerules/DbEdgeRules_v11.json
Add a new EdgeRule, allowing an edge between the CloudRegion and NewWidget:
{
"from": "cloud-region",
"to": "tenant",
"label": "has",
"direction": "OUT",
"multiplicity": "One2Many",
"contains-other-v": "${direction}",
"delete-other-v": "NONE",
"SVC-INFRA": "!${direction}",
"prevent-delete": "${direction}"
},
{
"from": "cloud-region",
"to": "new-widget",
"label": "has",
"direction": "OUT",
"multiplicity": "One2Many",
"contains-other-v": "NONE",
"delete-other-v": "NONE",
"SVC-INFRA": "NONE",
"prevent-delete": "NONE"
},
This means cloud-region connects to new-widget, edge label is "has", it's an OUT edge, one cloud region can have multiple edges to new-widgets,
cloud-region does not contain new-widget, new-widget will not be deleted when the cloud-region connected to it is deleted, it is not SVC_INFRA type, and having an edge to a new-widget will not prevent deletion of the cloud-region.- Save the file, and rebuild aai-common, resources, and traversal. Start resources and traversal microservices.
- Create a cloud region with a relationship to the new-widget that was created earlier: (PUT Cloud-Region related to New-Widget)
- GET the new-widget object: (GET New Widget)
- Notice that when GETting the new-widget object it shows the relationship to the cloud-region.
We have successfully modified the schema and edge rules to update an existing type, create a new type, and define an edge relationship which allows for a new-widget type to be connected to a cloud-region.