{"type":"doc","content":[{"type":"extension","attrs":{"extensionType":"com.atlassian.confluence.macro.core","extensionKey":"toc","parameters":{"macroParams":{},"macroMetadata":{"macroId":{"value":"a30ca7c7-85f6-4a24-9c0a-8a505d189035"},"schemaVersion":{"value":"1"},"title":"Table of Contents"}}}},{"type":"heading","attrs":{"level":1},"content":[{"text":"Issues and Decisions","type":"text"}]},{"type":"table","content":[{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Date","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Issue/Questions","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Decisions","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"type":"date","attrs":{"timestamp":"1720051200000"}},{"text":" ","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Should dry-run be a path parameter or a query parameter? ","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"dry-run flag should be an optional query parameter, making it part of path parameter will result in change in multiple endpoints using this feature. List of these APIs can be found below. And changing the path/endpoint by using dry-run as path parameter may impact backwards compatibility of these APIs","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Overview","type":"text"}]},{"type":"paragraph","content":[{"text":"Currently when persisting any data in CPS the data validation is performed using the yang schema and upon successful validation the data is stored in CPS DB. But in case the data validation fails a 500 response is returned by CPS.","type":"text"}]},{"type":"mediaSingle","attrs":{"layout":"align-start","width":30.0},"content":[{"type":"media","attrs":{"width":1457,"id":"d2ba7913-c379-4590-9206-2c96f3ebb8be","collection":"contentId-16552206","type":"file","height":947}}]},{"type":"paragraph","content":[{"text":"So, in case the data fails the validation check there is no way for the user to know the exact cause of failure. The failure is logged and can be seen in the container logs:","type":"text"}]},{"type":"heading","attrs":{"level":5},"content":[{"text":"Logged error","type":"text","marks":[{"type":"strong"}]}]},{"type":"codeBlock","content":[{"text":"{\"logTimeStamp\":\"2024-06-19T09:28:15.909Z\",\"logLevel\":\"ERROR\",\"traceId\":\"a0360e58c95bc63f6ff9f873c796e51c\",\n\"principalId\":\"cpsuser\",\n\"serviceName\":\"cps-application\",\n\"message\":\"Exception occurred\",\n\"spanId\":\"81b44b7a14f8fecb\",\n\"processId\":\"1\",\n\"threadName\":\"qtp1504320796-24\",\n\"class\":\"o.o.c.r.e.CpsRestExceptionHandler\",\n\"exception\":\"\\norg.onap.cps.spi.exceptions.DataValidationException: Failed to parse json data. Unsupported xpath or json data:{}\nat org.onap.cps.utils.YangParserHelper.parseJsonData(YangParserHelper.java:129)\nat org.onap.cps.utils.YangParserHelper.parseData(YangParserHelper.java:88)\n /....\n a lot of logged data \n ...../\nat java.base/java.lang.Thread.run(Unknown Source)\\nCaused by: java.lang.IllegalStateException: Schema node with name books was not found under (urn:ietf:params:xml:ns:netconf:base:1.0)data.\nat org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.resolveNamespace(JsonParserStream.java:410)\nat org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.read(JsonParserStream.java:308)\nat org.opendaylight.yangtools.yang.data.codec.gson.JsonParserStream.parse(JsonParserStream.java:193)\nat org.onap.cps.utils.YangParserHelper.parseJsonData(YangParserHelper.java:122)\n... 175 common frames omitted\\n\\n\"}","type":"text"}]},{"type":"paragraph","content":[{"text":"This is not very helpful for the user as it does not provide proper information about the root cause of failure.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Proposal","type":"text"}]},{"type":"paragraph","content":[{"text":"The idea here is to have a mechanism to validate the data and throw an appropriate response with message, without the data being persisted in the database. This validation check is not meant to be limited to new data but should also extend to sub-sets of a larger data set, such as data used in case of an update operation. So based on this the following approaches are proposed to perform a validation on data:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"A new endpoint to validate the JSON payload against an existing schema in CPS DB, this endpoint will only validate the data and never persist it.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"A dry-run flag in the existing CPS API's which would validate the data and return a response containing the details of validation failure. This flag can be implemented to multiple CPS APIs such as POST (Create a Node), PATCH (Update a Node) and PUT (Replace a node), where we can validate the entire data or parts of data and return the exact point of failure in the validation as part of response.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Implementation Ideas","type":"text"}]},{"type":"paragraph","content":[{"text":"In either case the endpoint should return a RESTful response with appropriate response code and message. ","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Response Code: ","type":"text","marks":[{"type":"strong"}]},{"text":"400","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Response Body: ","type":"text","marks":[{"type":"strong"}]},{"text":"a JSON response providing details about the failure. (response body format to be finalized)","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"List of impacted APIs if dry-run flag is introduced","type":"text"}]},{"type":"table","attrs":{"isNumberColumnEnabled":true},"content":[{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Name","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Endpoint","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Issue","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Required Action","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Create a Node","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"/{apiVersion}/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Returns 500 status on data validation failure","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":5},"content":[{"type":"paragraph","content":[{"text":"Return 400 status with updated and detailed error message.","type":"text"}]},{"type":"paragraph","content":[{"text":"Updated Message:","type":"text","marks":[{"type":"strong"}]}]},{"type":"codeBlock","content":[{"text":"{\n\"status\": \"400 BAD_REQUEST\",\n\"message\": \"Data Validation Failed\",\n\"Schema node with name \"Node\" was not found \nunder (urn:ietf:params:xml:ns:netconf:base:1.0)data.\" \n}","type":"text"}]},{"type":"paragraph","content":[{"type":"hardBreak"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Update Node Leaves","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"/{apiVersion}/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Returns 400 status with entire JSON payload in error message","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Replace Nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"/{apiVersion}/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Returns 400 status with entire JSON payload in error message","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Add List Element","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"/{apiVersion}/dataspaces/{dataspace-name}/anchors/{anchor-name}/list-nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Returns 500 status on data validation failure","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Replace List element","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"/{apiVersion}/dataspaces/{dataspace-name}/anchors/{anchor-name}/list-nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Returns 400 status with entire JSON payload in error message","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Example","type":"text"}]},{"type":"paragraph","content":[{"text":"The following example shows the expected changes to existing APIs to implement a dry-run flag:","type":"text"}]},{"type":"table","content":[{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"#","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"API","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"heading","attrs":{"level":3},"content":[{"text":"Request parameters","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Response Codes","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Scenarios","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"1","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Create a Node:","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"/v2/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"type":"hardBreak"}]},{"type":"extension","attrs":{"extensionType":"com.atlassian.confluence.migration","extensionKey":"__confluenceADFMigrationUnsupportedContentInternalExtension__","text":"","parameters":{"cxhtml":"

Parameter Name	In	Required	Description
dataspace-name	Path	true	dataspace name
anchor-name	Path	true	anchor name
xpath	Query	true	xpath of node
dry-run	Query	false	flag to enable or disable data validation (set to false by default)

","nestedContent":{"type":"doc","content":[{"type":"table","content":[{"type":"tableRow","content":[{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Parameter Name","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"In","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Required","type":"text"}]}]},{"type":"tableHeader","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Description","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"dataspace-name","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Path","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"true","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"dataspace name","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"anchor-name","type":"text","marks":[{"type":"textColor","attrs":{"color":"#172b4d"}}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Path","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"true","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"anchor name","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"xpath","type":"text","marks":[{"type":"textColor","attrs":{"color":"#172b4d"}}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Query","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"true","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"xpath of node","type":"text"}]}]}]},{"type":"tableRow","content":[{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"dry-run","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Query","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"false","type":"text"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"flag to enable or disable data validation","type":"text"}]},{"type":"paragraph","content":[{"text":"(set to false by default)","type":"text"}]}]}]}]}],"version":1},"reason":"A table in a table cell can't be created or edited in the new editor."}}},{"type":"paragraph","content":[{"type":"hardBreak"}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"200 (OK)","type":"text"},{"type":"hardBreak"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"success","type":"text"}]}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"400","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"dataspace not found","type":"text"},{"type":"hardBreak"},{"text":"DataspaceNotFoundException","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"anchor not found","type":"text"},{"type":"hardBreak"},{"text":"AnchorNotFoundException","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Data node not found","type":"text"},{"type":"hardBreak"},{"text":"DataNodeNotFoundException","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"invalid xpath","type":"text"},{"type":"hardBreak"},{"text":"CpsPathException","type":"text"}]}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"500","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"unexpected error","type":"text"}]}]}]}]}]}]},{"type":"tableCell","attrs":{"colspan":1,"rowspan":1},"content":[{"type":"paragraph","content":[{"text":"Scenario 1: dry run set to false","type":"text","marks":[{"type":"strong"}]}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In this case there will be no functional change in the APIs, and they would return appropriate response codes with messages in the following scenarios:","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In case of successful validation, the data will be persisted, and a 201-response code is returned with nothing in response body","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In case of validation failure, the data validation error is returned with appropriate message and a 400-response code.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Scenario 2: dry run set to true","type":"text","marks":[{"type":"strong"}]}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In this case the APIs will return the same response codes. The only difference will be:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"in case data validation passes, then it would return a 200-response code without any response body, and the data will not be persisted in case of POST operation or modified in case of PUT/PATCH operations.","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"In case of validation failure, the data validation error is returned with appropriate message and a 400-response code.","type":"text"}]}]}]}]}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Sub-tasks","type":"text"}]},{"type":"paragraph","content":[{"text":"The following APIs:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Create a Node","type":"text"}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"Add List Element","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"would return a 500-response code in case of Data Validation failure. This is in clear contradiction to the details mentioned in ","type":"text"},{"text":"CPS Exceptions and REST APIs HTTP Response Codes ","type":"text","marks":[{"type":"link","attrs":{"href":"https://lf-onap.atlassian.net/wiki/display/DW/CPS+Exceptions+and+REST+APIs+HTTP+Response+Codes"}}]},{"text":"documentation. This has been identified as a bug and was fixed in the following patch ","type":"text"},{"text":"Fix \"Create a node\" and \"Add List Elements\" APIs response code · Gerrit Code Review","type":"text","marks":[{"type":"link","attrs":{"href":"https://gerrit.onap.org/r/c/cps/+/138268"}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"References:","type":"text"}]},{"type":"bulletList","content":[{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"CPS Exceptions and REST APIs HTTP Response Codes","type":"text","marks":[{"type":"link","attrs":{"href":"https://lf-onap.atlassian.net/wiki/spaces/DW/pages/16447685"}}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"400 Bad Request - HTTP | MDN (mozilla.org)","type":"text","marks":[{"type":"link","attrs":{"href":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400"}}]}]}]},{"type":"listItem","content":[{"type":"paragraph","content":[{"text":"A good article explaining concept of dry-run: ","type":"text"},{"text":"https://dx.apiture.com/docs/api/concepts/dry-runs/","type":"text","marks":[{"type":"link","attrs":{"href":"https://dx.apiture.com/docs/api/concepts/dry-runs/"}}]},{"text":" ","type":"text"},{"type":"emoji","attrs":{"id":"1f642","text":"🙂","shortName":":slight_smile:"}}]}]}]},{"type":"paragraph","content":[{"type":"hardBreak"}]}],"version":1}

Browser not supported