CPS Delta between 2 Anchors API design

Issues & Decisions

CPS Delta feature Exceptions

Where ever possible the Delta Feature will throw the same exceptions as defined in CPS core. If any new exception for the delta feature are required the following will be updated here.

HTTP response codes for Delta between 2 Anchors API

The proposed API will be part of the CPS Data Interface. The following response codes will be returned by the API:

Request parameters:

Response Body/Delta Report Format

[
  {
    "action": "ADD",
    "xpath": "/bookstore/categories/[@code=3]",
    "target": {
      "code": 3,
      "name": "kidz"
    }
  },
  {
    "action": "DELETE",
    "xpath": "/bookstore/categories/[@code=1]",
    "source": {
      "code": 1,
      "name": "Fiction"
    }
  },
  {
    "action": "UPDATE",
    "xpath": "/bookstore/categories/[@code=2]",
    "source": {
      "name": "Funny"
    },
    "target": {
      "name": "Comic"
    }
  }
]

Mechanism for Delta generation between Anchors

Additional Details

Format/Conventions to be used for Delta Report

There are several ways of representing the differences between JSON but here we discuss the JSON Patch format of representing these differences because the proposed Delta report closely represents JSON patch with a few differences. This approach focuses in producing another JSON document that represents the differences between the two JSON's that have been compared.

RFC 6902 JSON Patch

JSON Patch defines a JSON document structure for expressing a sequence of operations to apply to a JavaScript Object Notation(JSON) document; it is suitable for use with the HTTP PATCH method. The "application/json-patch+json" media type is used to identify such patch documents.

But in it can also be used to get the differences between two JSON as the JSON patch document represents an array of objects. where each object represents a single operation(op), path and value.

Here the notation of operation is used as the same JSON Patch document can be used to perform HTTP Patch operations. But The operations field can be used as a reference to decide how the "action" field should function in the implementation of Delta report.

The operation "op" field in a JSON patch document can have following values:

[
     { "op": "test", "path": "/a/b/c", "value": "foo" },
     { "op": "remove", "path": "/a/b/c" },
     { "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
     { "op": "replace", "path": "/a/b/c", "value": 42 },
     { "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
     { "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
   ]

RFC 9144 Data Model

RFC 9144 YANG data model that defines RPCs intended to be used in conjunction with NETCONF [RFC6241] or RESTCONF [RFC8040]. These RPCs allow a client to request a server to compare two NMDA datastores and report any differences.

Comparison of RFC 9144 Data Model and RFC6902 JSON Patch format

• The core of the RFC 9144 solution is a new management operator, <compare>, that compares the data tree contents of two datastores. The operation checks whether there are any differences in values or in data nodes that are contained in either datastore and returns any differences as output. The output is returned in the format specified below.
• The RFC6902 approach to Delta report generation follows the JSON patch format. A JSON Patch document represents an array of objects, where each object contains exactly one operation, path and associated values. The operation can have following values: add, remove, replace, move, copy and test. The path represents the JSON patch and the values contain the difference in source and target values.

{ "ietf-nmda-compare:output" : {
    "differences" : {
      "ietf-yang-patch:yang-patch" : {
        "patch-id" : "interface status",
        "comment" : "diff between intended (source) and operational",
        "edit" : [
          {
            "edit-id" : "1",
            "operation" : "replace",
            "target" : "/ietf-interfaces:interface=eth0/enabled",
            "value" : {
               "ietf-interfaces:interface/enabled" : "false"
            },
            "source-value" : {
               "ietf-interfaces:interface/enabled" : "true",
               "@ietf-interfaces:interface/enabled" : {
                 "ietf-origin:origin" : "ietf-origin:learned"
               }
             }
          },
          {
            "edit-id" : "2",
            "operation" : "create",
            "target" : "/ietf-interfaces:interface=eth0/description",
            "value" : {
              "ietf-interface:interface/description" : "ip interface"
            }
          }
        ]
      }
    }
  }
}

PATCH /my/data HTTP/1.1
   Host: example.org
   Content-Length: 326
   Content-Type: application/json-patch+json
   If-Match: "abc123"
 
   [
     { "op": "test", "path": "/a/b/c", "value": "foo" },
     { "op": "remove", "path": "/a/b/c" },
     { "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
     { "op": "replace", "path": "/a/b/c", "value": 42 },
     { "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
     { "op": "copy", "from": "/a/b/d", "path": "/a/b/e" }
   ]

CPS Delta Report Format

Based on the RFC documentations mentioned above, CPS Delta Report is designed in the format specified in the following code block. This Delta Report takes the features from both RFCs that are best suited for CPS and fulfill it's requirements.

[
  {
    "action": "ADD",
    "xpath": "/bookstore/categories/[@code=3]",
    "target-data": {
      "code": "3,",
      "name": "kidz"
    }
  },
  {
    "action": "REMOVE",
    "xpath": "/bookstore/categories/[@code=1]",
    "source-data": {
      "code": "1,",
      "name": "Fiction"
    }
  },
  {
    "action": "UPDATE",
    "xpath": "/bookstore/categories/[@code=2]",
    "source-data": {
      "name": "Funny"
    },
    "target-data": {
      "name": "Comic"
    }
  }
]

Proposed Algorithm

The JSON data stored in CPS can be retrieved as Maps, And using this we can find the delta between two anchors retrieved as two separate Maps. The result can be stored in an JSON array with appropriate notations for action, xpath and payload. The following algorithm can be used to find the difference between the two maps and also recursively finds the difference between the child data nodes. The response is in form of a JSON Array which closely represents the JSON Patch format as described above.

Algorithm:

1. Create an empty JSON Array, to store the result
   • The JSON array will contain the following: action, xpath, payload as individual JSON objects each object representing a singular operation.
2. Fetch the data from two anchors and store in two separate Maps
3. Iterate over the entries of first map
4. For each entry of first map, check if the key is present in second map.
5. If the key is not found in second map, it means that the key-value pair has been removed from the second map. Create a JSON Object with DELETE action, the xpath to deleted node and payload. Add the Object to the JSON Array.
6. If the key is present in the second map, compare the values associated with the keys in both maps.
7. If the values are instance of Map, recursively call the comparision algorithm to compare all nested maps. Add all the necessary fields into a JSON object and put the object into the JSON Array
8. If the values are not equal, it means the key-value pair was updated. Create a JSON Object with UPDATE action, the xpath to updated nodes and payload. Add the Object to the JSON Array.
9. Iterate over the keys of second map and find the keys not present in first map. These keys were added as new entries. Create a JSON Object with ADD action, the xpath to added node and payload. Add the Object to the JSON Array.
10. Return the JSON Array and the updates to Kafka events