CPS Delta between 2 Anchors API
- Arpit Singh
- Toine Siebelink
- 1.1 Issues & Decisions
- 1.2 CPS Delta feature Exceptions
- 1.3 HTTP response codes for Delta between 2 Anchors API
- 1.3.1 Request parameters:
- 1.4 Response Body/Delta Report Format
- 1.5 Mechanism for Delta generation between Anchors
- 1.6 Additional Details
- 1.6.1 Format/Conventions to be used for Delta Report
- 1.6.2 RFC 6902 JSON Patch
- 1.6.2.1 Sample JSON Patch
- 1.6.3 RFC 9144 Data Model
- 1.7 Comparison of RFC 9144 Data Model and RFC6902 JSON Patch format
- 1.8 CPS Delta Report Format
- 1.9 Proposed Algorithm
- 1.9.1 Algorithm:
- 2 References/Future Changes
Issues & Decisions
Issue | Notes | Decision | |
|---|---|---|---|
| 1 | Add or Delete leaves handle as UPDATE or ADD/DELETE? | The delta report proposed follows the Json Patch format of representing the differences between 2 JSON data. Apart from this the delta is dependent on individual data nodes of the JSON data, this is because the delta report will contain the action, xpath and the source/target data. And since the xpaths are unique to data nodes and not leaf data, every delta entity in delta report will be between 2 data nodes. Going by the general convention and referring RFC-6902:
| Updated as per notes. |
| 2 | How to handle multiple changes at different levels? | Since the delta report will contain the xpaths of data nodes changed, so for changes at multiple levels i.e., parent and child data nodes, the approach will be to handle each data node individually and report them in the delta report. |
|
| 3 | More scenarios need to be explored and documented in detail. Such as handling arrays within a json, handling child/grandchild changes. |
| scenarios such as data nodes at multiple levels, arrays and lists are covered. |
CPS Delta feature Exceptions
Wherever possible the Delta Feature will throw the same exceptions as defined in CPS core. If any new exception for the delta feature is required, the following will be updated here.
HTTP response codes for Delta between 2 Anchors API
The proposed API will be part of the CPS Data Interface. The following response codes will be returned by the API:
# | Sub Interface | Method | Scenario | HTTP Response codes to be implemented | Notes |
|---|---|---|---|---|---|
1 | Data | Proposed API: GET-/v2/dataspaces/{dataspace-name}/anchors/{source-anchor-name}/delta?target-anchor-name={target-anchor-name}?xpath={xpath}&descendants={descendants} Proposed method name: getDeltaByDataspaceAndAnchors() | Generate a delta report between 2 anchors in a given dataspace. |
| AnchorNotFoundException should provide the name of missing anchor from the given two anchor names. |
Request parameters:
Parameter name | In | Required | Description |
|---|---|---|---|
dataspace-name | Path | Yes | Dataspace name |
source-anchor-name | Path | Yes | First Anchor Name/Reference Anchor |
target-anchor-name | Query | Yes | Second Anchor Name/Comparand Anchor |
xpath | Query | Yes | xpath of the node |
descendants | Query | No | Level of descendants for delta comparison. |
Response Body/Delta Report Format
Response body should contain anchors delta report (added/deleted/modified configuration) as below.
[
{
"action": "create",
"xpath": "/bookstore/categories/[@code=3]",
"target-data": {
"code": 3,
"name": "kidz"
}
},
{
"action": "remove",
"xpath": "/bookstore/categories/[@code=1]",
"source-data": {
"code": 1,
"name": "Fiction"
}
},
{
"action": "replace",
"xpath": "/bookstore/categories/[@code=2]",
"source-data": {
"name": "Funny"
},
"target-data": {
"name": "Comic"
}
}
]Mechanism for Delta generation between Anchors
Additional Details
Format/Conventions to be used for Delta Report
The delta report format is based on two RFCs namely RFC 6902 and RFC 9144 . A detailed comparison of the RFCs can be found here.
RFC 6902 JSON Patch
JSON Patch defines a JSON document structure for expressing a sequence of operations to apply to a JavaScript Object Notation(JSON) document; it is suitable for use with the HTTP PATCH method.
But in it can also be used to get the differences between two JSON as the JSON patch document represents an array of objects. where each object represents a single operation(op), path and value.
The notation of "operation" is used because the same JSON Patch document can be used to perform HTTP Patch operations. But the operations field can be used as a reference to decide how the "action" field should function in the implementation of Delta report.
The operation "op" field in a JSON patch document can have following values:
Operation | Description | Delta report equivalent |
|---|---|---|
Adds the value at the target location; if the value exists in the given location, it’s replaced | if the value is not present in the source json, but was found in the target json, then it should be considered a "create" action. | |
Removes the value at the target location | if a value was present at the source json, but was not found in the target json, then it should be considered as "remove" action. | |
Replaces the value at the target location | if a value is present in the source json, but an updated value is present in the target json, then it will be considered as "replace" action. | |
Removes the value at a specified location and adds it to the target location | N/A | |
Copies the value at a specified location to the target location | N/A | |
Tests that a value at the target location is equal to a specified value | N/A |
Sample JSON Patch
[
{ "op": "test", "path": "/a/b/c", "value": "foo" },
{ "op": "remove", "path": "/a/b/c" },
{ "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
{ "op": "replace", "path": "/a/b/c", "value": 42 },
{ "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
{ "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
]RFC 9144 Data Model
RFC 9144 YANG data model that defines RPCs intended to be used in conjunction with NETCONF [RFC6241] or RESTCONF [RFC8040]. These RFCs allow a client to request a server to compare two NMDA datastores and report any differences.
Comparison of RFC 9144 Data Model and RFC6902 JSON Patch format
The core of the RFC 9144 solution is a new operator, <compare>, that compares the data tree contents of two datastores. The operation checks whether there are any differences in values or in data nodes that are contained in either datastore and then returns any differences as output. The output is returned in the format specified below.
The RFC6902 approach to Delta report generation follows the JSON patch format. A JSON Patch document represents an array of objects, where each object contains exactly one operation, path and associated values. The operation can have following values: add, remove, replace, move, copy and test. The path represents the JSON patch format and the values contain the difference in source and target values.
Sample delta as per RFC9144{ "ietf-nmda-compare:output" : {
"differences" : {
"ietf-yang-patch:yang-patch" : {
"patch-id" : "interface status",
"comment" : "diff between intended (source) and operational",
"edit" : [
{
"edit-id" : "1",
"operation" : "replace",
"target" : "/ietf-interfaces:interface=eth0/enabled",
"value" : {
"ietf-interfaces:interface/enabled" : "false"
},
"source-value" : {
"ietf-interfaces:interface/enabled" : "true",
"@ietf-interfaces:interface/enabled" : {
"ietf-origin:origin" : "ietf-origin:learned"
}
}
},
{
"edit-id" : "2",
"operation" : "create",
"target" : "/ietf-interfaces:interface=eth0/description",
"value" : {
"ietf-interface:interface/description" : "ip interface"
}
}
]
}
}
}
}
|
Sample JSON Patch document as per RFC 6902PATCH /my/data HTTP/1.1
Host: example.org
Content-Length: 326
Content-Type: application/json-patch+json
If-Match: "abc123"
[
{ "op": "test", "path": "/a/b/c", "value": "foo" },
{ "op": "remove", "path": "/a/b/c" },
{ "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
{ "op": "replace", "path": "/a/b/c", "value": 42 },
{ "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
{ "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
]
|
|---|
CPS Delta Report Format
Based on the RFC documentations mentioned above, CPS Delta Report is designed in the format specified in the following code block. This Delta Report takes the features from both RFCs that are best suited for CPS and fulfill its requirements.
The format of Delta report has following key take aways from the above-mentioned RFCs:
The "op" field from RFC 6902 is replaced with "action" field because in CPS we want to report the action that was performed on the data. And this field can have 3 predefined values: create, remove and replace.
The xpath is used in CPS to uniquely identify individual data nodes and is used in place of JSON path as defined in the two RFCs
The source-data and target-data fields are used from RFC9144 to report the data that has been added, removed or updated. This approach properly categorizes the data instead of grouping it under the one "value" field as in RFC6902.
Delta Report Format for CPS
[
{
"action": "create",
"xpath": "/bookstore/categories/[@code=3]",
"target-data": {
"code": "3,",
"name": "kidz"
}
},
{
"action": "remove",
"xpath": "/bookstore/categories/[@code=1]",
"source-data": {
"code": "1,",
"name": "Fiction"
}
},
{
"action": "replace",
"xpath": "/bookstore/categories/[@code=2]",
"source-data": {
"name": "Funny"
},
"target-data": {
"name": "Comic"
}
}
]Proposed Algorithm
The JSON data stored in CPS can be retrieved as Maps, And using this we can find the delta between two anchors retrieved as two separate Maps. The result can be stored in an JSON array with appropriate notations for action, xpath and payload. The following algorithm can be used to find the difference between the two maps and also recursively finds the difference between the child data nodes. The response is in form of a JSON Array which closely represents the JSON Patch format as described above.
Algorithm:
Create an empty JSON Array, to store the result
The JSON array will contain the following: action, xpath, payload as individual JSON objects each object representing a singular operation.
Fetch the data from two anchors and store in two separate Maps
Iterate over the entries of first map
For each entry of first map, check if the key is present in second map.
If the key is not found in second map, it means that the key-value pair has been removed from the second map. Create a JSON Object with 'remove' action, the xpath to deleted node and payload. Add the Object to the JSON Array.
If the key is present in the second map, compare the values associated with the keys in both maps.
If the values are instance of Map, recursively call the comparision algorithm to compare all nested maps. Add all the necessary fields into a JSON object and put the object into the JSON Array
If the values are not equal, it means the key-value pair was updated. Create a JSON Object with 'replace' action, the xpath to updated nodes and payload. Add the Object to the JSON Array.
Iterate over the keys of second map and find the keys not present in first map. These keys were added as new entries. Create a JSON Object with 'create' action, the xpath to added node and payload. Add the Object to the JSON Array.
Return the JSON Array and the updates to Kafka events