CPS Exceptions and REST APIs HTTP Response Codes

ONAP Guidelines

Finally found general ONAP Advise:

"The API specification should describe the right HTTP status code to return the client.

Status codes should align with IETF's HTTP Status Code Registry: https://www.ietf.org/assignments/http-status-codes/http-status-codes.xml"

Agreed CPS API(SPI) Exception Hierarchy

(see CCSDK-2871 DP: Long-term Java API(s)

CpsException
- CpsAdminException
  - DataspaceNotFoundException
  - DataspaceAlreadyDefinedException
  - SchemaSetNotFoundException
  - SchemaSetValidationException
  - SchemaSetAlreadyDefinedException (this is separate from SchemaSetValidationException as it could be ignored by the client)
  - AnchorAlreadyDefinedException
  - AnchorNotFoundException
- ModelValidationException (mainly applicable to schema set validation)
- DataValidationException
  - SpecificException3
  - SpecificException4
- PathValidationException
  - SpecificException5

*Note we might have to move these 'common' exceptions classes to the SPI just like we did with the common data objects

Issues & Decisions

#

Slogan

Remarks

Decision

Who &When

1

treatment of incorrect dataspace, schemaset (names)

These are core concept of CPS and should be correct/exists (except in create use cases of course)

Introduce specific exception for each extending new CPSAdminException class
re-use same exception in API (pass through)
Treat as bad URL in REST interface

CPS Team Meeting 3 Dec

2

Using different HTTP Response code (400 v 404) depending on type of operation, adding data/requesting data

It might make sense to use 404 Not Found in a get scenario (ie like #3 scenario in the table below)
But this would be wrong in a create/add data scenario. The API throws the same Exception so the REST Impl would need additional code to handle this distinction

Use 400 Bad requests in both scenarios for the following reasons

Consistency
Simplicity of handling in REST Layer
HTTP error codes are always ambiguous, we will add documentation for each REST call anyway to specify which errors are used in which scenario

CPS Team Meeting 3 Dec

Vertical slice view of error scenarios

CPS-Core

#	Use case	Scenario	DB (Spring) Response	SPI Response	Java API	REST API Response Code	Response Body	Error Object Message
1	Add dataspace	dataspace already exists	DataIntegrityViolationException	DataspaceAlreadyDefinedException	DataspaceAlreadyDefinedException	400 (bad request)	json error object	dataspace xyz already defined
2	Add schema set to dataspace	schema set with same name already exists	DataIntegrityViolationException	SchemaSetAlreadyDefinedException	SchemaSetAlreadyDefinedException	400 (bad request)	json error object	schema set xyz already defined for dataspace abc
3	Add schema set to dataspace	dataspacename does not exist	No records (Optional, not present)	DataspaceNotFoundException	DataspaceNotFoundException	400 (bad request)	json error object	TBD
4	Add schema set to dataspace	the schema set throws a validation exception when parsed with ODL Yang Parser e.g. No files (in zip) Missing (import) files Incorrect yang in a module Duplicate namespaces	N/A	N/A	ModelValidationException (e casue) details should contain cause message	400 (bad request)	json error object	message and details from the original exception.
5	List anchors for dataspace name	dataspacename does not exist	No records (Optional, not present)	DataspaceNotFoundException	DataspaceNotFoundException	400 (bad request)	json error object	dataspace xyz not found
6	List anchors for dataspace name	no anchor records in DB	0 records	empty collection	empty collection	200 (ok)	empty list	N/A
7	Add a fragment	Insert a fragment that violates the unique constraint e.g. when trying to insert a fragment with the same property values(dataspace_id,anchor_id,xpath) as an existing fragment.	DataIntegrityViolationException
8	Remove schema set	Schema set removal is requested from REST (or via API/SPI with explicitly defined option prohibiting removal of associated anchors and data if found) and there is (are) associated anchor record(s) in database.	N/A	SchemaSetInUseException	SchemaSetInUseException	409 (Conflict)	json error object	schama set abc in dataspace xyz is having anchor records associated.
9	Get a fragment that does not exist	get fragment by xpath that does not exist	FragmentNotFoundException	DataNodeNotFoundException	DataNodeNotFoundException	400 (bad request)	json error object	datanode with xpath xyz not found in dataspace xyz
10	Invalid cps path	a cpsPath that cannot be parsed i.e. not recognized as a valid query 2021-03-08T14:57:19.343Z\|main\|\| o.onap.cps.rest.exceptions.CpsRestExceptionHandler - An error has occurred : Invalid cps path. Status: 400 BAD_REQUEST Details: Cannot interpret or parse cps path. 2021-03-08 15:01:44.441 ERROR 21676 --- [ main] o.o.c.r.e.CpsRestExceptionHandler : An error has occurred : Unsupported leaf value. Status: 400 BAD_REQUEST Details: Unsupported leaf value in cps path.	CpsPathException	CpsPathException	CpsPathException	400 (bad request)	json error object	message and details from the original exception.
11	Data node already exists	Create the same data node twice	DataIntegrityViolationException	DataNodeAlreadyDefined	DataNodeAlreadyDefined	409 (Conflict)	json error object	Data node already defined for dataspace abc
12	Unsupported update node leaves	Unsupported json data when Conversion of a normalized node to a datanode within the datanode builder class. private DataNode buildFromNormalizedNodeTree() { final Collection<DataNode> dataNodeCollection = buildCollectionFromNormalizedNodeTree(); return dataNodeCollection.iterator().next(); } 2. Parsing list json data	NoSuchElementException IllegalStateException	DataValidationException	DataValidationException	400	json error object	"Unsupported json data: " + jsonData
13	Delete dataspace with anchor/s	Delete dataspace that still contains 1 or more anchor		DataspaceInUseException	DataspaceInUseException	409 (Conflict)	json error object	dataspace contains anchor(s)
14	Delete dataspace with schemaset/s	Delete dataspace that still contains 1 or more schemaset		DataspaceInUseException	DataspaceInUseException	409 (Conflict)	json error object	dataspace contains schemaset(s)

Spike CPS-728: Inventory of HTTP Response Codes for all CPS interfaces

CPS-782 - Getting issue details... STATUS

CPS-Core

(commit id 8c9b4a24b4841154a2d08f506e2f7beca4f6fc31)

#	Sub interface	Method	Scenario	Specified HTTP Response Code	Implemented HTTP Response Code	Changes required
1	Admin	POST - /v1/dataspaces CpsAdminApi.createDataspace()	Create a new dataspace	201 400 401 403	201 (created) for success 409 (conflict) for data integrity violation when persisting (AlreadyDefinedException) 500 (internal server error) for unexpected system errors	Specification: Remove 400 Add 409
2	Admin	DELETE - /v1/dataspaces CpsAdminApi.deleteDataspace()	Delete a dataspace	204 400 401 403 409	204 (no content) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) 409 (conflict) for dataspace having anchors or schema sets (DataspaceInUseException) 500 (internal server error) for unexpected system errors
3	Admin	GET - /v1/dataspaces/{dataspace-name}/anchors CpsAdminApi.getAnchors()	Read all anchors, given a dataspace	200 400 401 403 404	200 (ok) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) 500 (internal server error) for unexpected system errors	Specification: Remove 404
4	Admin	POST - /v1/dataspaces/{dataspace-name}/anchors CpsAdminApi.createAnchor()	Create a new anchor in the given dataspace	201 400 401 403	201 (created) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for schema set not found (SchemaSetNotFoundException) 409 (conflict) for data integrity violation when persisting (AlreadyDefinedException) 500 (internal server error) for unexpected system errors	Specification: Add 409
5	Admin	GET - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name} CpsAdminApi.getAnchor()	Read an anchor given an anchor name and a dataspace	200 400 401 403 404	200 (ok) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) 500 (internal server error) for unexpected system errors	Specification: Remove 404
6	Admin	DELETE - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name} CpsAdminApi.deleteAnchor()	Delete an anchor given an anchor name and a dataspace	204 400 401 403	204 (no content) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) 500 (internal server error) for unexpected system errors
7	Admin	POST - /v1/dataspaces/{dataspace-name}/schema-sets CpsAdminApi.createSchemaSet()	Create a new schema set in the given dataspace	201 400 401 403	201 (created) for success 400 (bad request) for different errors related to input schema set file reading (CpsException) for zip schema set file size exceeding defined limit (ModelValidationException) for zip schema set file number entries exceeding defined limit (ModelValidationException) for zip schema set file containing no yang files (ModelValidationException) for multiple resources with same name contained in zip schema set file (ModelValidationException) for schema set file name extension different from yang or zip (ModelValidationException) for invalid Yang syntax (ModelValidationException) for dataspace not found (DataspaceNotFoundException) 409 (conflict) for data integrity violation when persisting (AlreadyDefinedException) 500 (internal server error) for concurrency error when several clients are requesting to create the exact same resource at the same time and that the automatic retry keeps failing (DuplicatedYangResourceException) for unexpected system errors	Specification: Add 409 Implementation Concurrency error is 500, but there is not much that can be done on the server side, the client would have to resend ... Should it be 400 instead ? => Decision made to keep 500
8	Admin	GET - /v1/dataspaces/{dataspace-name}/schema-sets/{schema-set-name} CpsAdminApi.getSchemaSet()	Read a schema set given a schema set name and a dataspace	200 400 401 403 404	200 (ok) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for schema set not found (SchemaSetNotFoundException) 500 (internal server error) for unexpected system errors	Specification: Remove 404
9	Admin	DELETE - /v1/dataspaces/{dataspace-name}/schema-sets/{schema-set-name} CpsAdminApi.deleteSchemaSet()	Delete a schema set given a schema set name and a dataspace	204 400 401 403 409	204 (no content) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for schema set not found (SchemaSetNotFoundException) 409 (conflict) for existence of anchors using the schema set (SchemaSetInUseException) 500 (internal server error) for unexpected system errors
10	Data	GET - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/node CpsDataApi.getNodeByDataspaceAndAnchor()	Get a node with an option to retrieve all the children for a given anchor and dataspace	200 400 401 403 404	200 (ok) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) 500 (internal server error) for unexpected system errors	Specification: Remove 404
11	Data	PUT - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes CpsDataApi.replaceNode()	Replace a node with descendants for a given dataspace, anchor and a parent node xpath	200 400 401 403	200 (ok) for success 400 (bad request) for error while parsing json data or not supported xpath (DataValidationException) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) for data node not found (DataNodeNotFoundException) 500 (internal server error) for concurrent transactions making data to be updated stale (ConcurrencyException) for unexpected system errors	Implementation: Concurrency error is 500, but there is not much that can be done on the server side, the client would have to resend ... Should it be 400 instead ? => Decision made to keep 500
12	Data	POST - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes CpsDataApi.createNode()	Create a node for a given anchor and dataspace	201 400 401 403	201 (created) for success 400 (bad request) for error while parsing json data or not supported xpath (DataValidationException) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) for data node not found (DataNodeNotFoundException) 409 (conflict) for data integrity violation when persisting (AlreadyDefinedException) 500 (internal server error) for unexpected system errors	Specification: Add 409
13	Data	DELETE - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes CpsDataApi.deleteDataNode()	Delete a datanode for a given dataspace and anchor given a node xpath.	204 400 401 403	204 (no content) for success 400 (bad request) for data node not found (DataNodeNotFoundException) 500 (internal server error) for unexpected system errors
14	Data	PATCH - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes CpsDataApi.updateNodeLeaves()	Update a data node leaves for a given dataspace and anchor and a parent node xpath	200 400 401 403	200 (ok) for success 400 (bad request) for error while parsing json data or not supported xpath (DataValidationException) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) for data node not found (DataNodeNotFoundException) 500 (internal server error) for unexpected system errors
15	Data	PUT - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/list-nodes CpsDataApi.replaceListContent()	Replace list content under a given parent, anchor and dataspace	200 400 401 403	200 (ok) for success 400 (bad request) for error while parsing json data or not supported xpath (DataValidationException) for no data nodes provided (DataValidationException) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) for data node not found (DataNodeNotFoundException) 500 (internal server error) for unexpected system errors
16	Data	POST - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/list-nodes CpsDataApi.addListElements()	Add list element(s) to a list for a given anchor and dataspace	201 400 401 403	201 (created) for success 400 (bad request) for error while parsing json data or not supported xpath (DataValidationException) for no data nodes provided (DataValidationException) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) for data node not found (DataNodeNotFoundException) 500 (internal server error) for unexpected system errors
17	~~Data~~	~~DELETE - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/list-nodes~~ ~~CpsDataApi.deleteListOrListElement()~~	~~Delete one or all list element(s) for a given anchor and dataspace~~	~~204~~ ~~400~~ ~~401~~ ~~403~~	~~204 (no content)~~ ~~for success~~ ~~400 (bad request)~~ ~~for targetprobably not a list (DataNodeNotFoundException)~~ ~~500 (internal server error)~~ ~~for unexpected system errors~~	~~Deprecated, ignored~~
18	Query	GET - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/nodes/query CpsQueryApi.getNodesByDataspaceAndAnchorAndCpsPath()	Query data nodes for the given dataspace and anchor using CPS path	200 400 401 403 404	200 (ok) for success 400 (bad request) for dataspace not found (DataspaceNotFoundException) for anchor not found (AnchorNotFoundException) for failure when parsing provided cps path (CpsPathException) for failure when parsing retrieved fragment entity attributes (DataValidationException) 500 (internal server error) for unexpected system errors	Implementation: Failure to parse fragment entity should not be a client error ? 404 not needed

CPS-NCMP

(commit id 8c9b4a24b4841154a2d08f506e2f7beca4f6fc31)

#	Sub interface	Method	Scenario	Specified HTTP Response Code	Implemented HTTP Response Code	Comments
1	Data	HTTP: GET - /v1/ch/{cm-handle}/data/ds/ncmp-datastore:passthrough-operational NetworkCmProxyApi.getResourceDataOperationalForCmHandle()	Get resource data from pass-through operational for given cm handle	200 400 401 403 404	200 (ok) for success 500 (internal server error) for error when parsing provided dmi request to json (NcmpException) for dmi response code different from 2xx (NcmpException) for unexpected system errors	Specification: Remove 404 Implementation: Parsing error for dmi request should be a 400 client error
2	Data	GET - /v1/ch/{cm-handle}/data/ds/ncmp-datastore:passthrough-running NetworkCmProxyApi.getResourceDataRunningForCmHandle()	Get resource data from pass-through running for given cm handle	200 400 401 403 404	200 (ok) for success 500 (internal server error) for error when parsing provided dmi request to json (NcmpException) for dmi response code different from 2xx (NcmpException) for unexpected system errors	Specification: Remove 404 Implementation: Parsing error for dmi request should be a 400 client error
3	Data	PUT- /v1/ch/{cm-handle}/data/ds/ncmp-datastore:passthrough-running NetworkCmProxyApi.updateResourceDataRunningForCmHandle()	Update resource data from pass-through running for the given cm handle	200 400 401 403 404	200 (ok) for success 500 (internal server error) for error when parsing provided dmi request to json (NcmpException) for dmi response code different from 2xx (NcmpException) for unexpected system errors	Specification: Remove 404 Implementation: Parsing error for dmi request should be a 400 client error
4	Data	POST - /v1/ch/{cm-handle}/data/ds/ncmp-datastore:passthrough-running NetworkCmProxyApi.createResourceDataRunningForCmHandle()	Create resource data from pass-through running for given cm handle	201 400 401 403 404	201 (created) for success 500 (internal server error) for error when parsing provided dmi request to json (NcmpException) for dmi response code different from 2xx (NcmpException) for unexpected system errors	Specification: Remove 404 Implementation: Parsing error for dmi request should be a 400 client error
5	Data	PATCH - /v1/ch/{cm-handle}/data/ds/ncmp-datastore:passthrough-running NetworkCmProxyApi.patchResourceDataRunningForCmHandle()	Patch resource data from pass-through running for the given cm handle	200 400 401 403 404	200 (ok) for success 500 (internal server error) for error when parsing provided dmi request to json (NcmpException) for dmi response code different from 2xx (NcmpException) for unexpected system errors	Specification: Remove 404 Implementation: Parsing error for dmi request should be a 400 client error
6	Model	GET - /v1/ch/{cm-handle}/modules NetworkCmProxyApi.getModuleReferencesByCmHandle	Fetch all module references (name and revision) for a given cm handle	200 400 401 403 404	200 (ok) for success 500 (internal server error) for unexpected system errors	Specification: Remove 400, 404
7	Model	POST - /v1/ch/searches NetworkCmProxyApi.executeCmHandleSearch	Execute cm handle searches using 'hasAllModules' condition to get all cm handles for the given module names	200 400 401 403	200 (ok) for success 500 (internal server error) for no yang resources found for provided module (ModuleNamesNotFoundException) for unexpected system errors	Implementation: Currently, 500 is returned when no matches found. This should be changed to a 4xx response Invalid body payload struture format returns 500 with the system error message, not really useful for the client. Should also returns 400.
8	Inventory	POST /v1/ch NetworkCmProxyInventoryApi.updateDmiPluginRegistration()	Register, update or remove cm handles	201 400 401 403	201 (created) for success 500 (internal server error) for dmi plugin service name not provided (NcmpException) for invalid combination of dmi plugin service names (NcmpException) for parsing error while processing dmi plugin registration (DataValidationException) for unexpected system errors	Specification: Should success always be 200 (general ok) instead of 201 (created) that might look confusing for updates and removals ? Implementation: Invalid requests should return 4xx with an explanation message instead of 500 without information (ex: Invalid combination of plugin service names, URI is not absolute, ...) What should be the error code related to dmi connectivity issues ? Could be either 400 (the client provided an invalid url) or 500 (the url provided by the client is valid, but the connection failed because of a system error) Current information logged when dmi plugin service is not reachable is not enough to support and investigate.

DMI-Plugin

(commit id 1954daa5d31b537a8838d1e11cb41cd6f40b58d9)

#	Sub interface	Method	Scenario	Specified HTTP Response Code	Implemented HTTP Response Code	Comments
1	DMI Plugin Internal	POST - /v1/inventory/cmHandles DmiPluginInternalApi.registerCmHandles()	Register given list of cm handles (internal use only)	201 400 401 403	201 (created) for success 400 (bad request) for empty cm handles list from input 500 (internal server error) for parsing error when converting provided cm handles into JSON (DmiException) for any response code from NCMP that is different form 201 (CmHandleRegistrationException) for unexpected system errors	Implementation Input cm handle conversion error should return a 4xx client error
2	DMI Plugin	POST - /v1/ch/{cmHandle}/modules DmiPluginApi.getModuleReferences()	Get all modules for given cm handle	200 400 401 403	200 (ok) for success 404 (not found) for SDNC returning no modules for given cm handle (ModulesNotFoundException) 500 (internal server error) for SDNC response JSON format or processing failure (SdncException) for unexpected system errors	Specification: Remove 400 Add 404
3	DMI Plugin	POST - /v1/ch/{cmHandle}/moduleResources DmiPluginApi.retrieveModuleResources()	Retrieve module resources for one or more modules	200 400 401 403	200 (ok) for success 404 (not found) for module resource not found from SDNC for the given cm handle (ModuleResourceNotFoundException) for error occurred when trying to parse the response body from SDNC (ModuleResourceNotFoundException) 500 (internal server error) for JSON parsing error when creating the module request from the given module reference (DmiException) for any SDNC response code that is different from 200 or 404 for unexpected system errors	Implementation: SDNC errors are not really consistent (404 and 500) Error related to the given module reference should be 4xx ?
4	DMI Plugin	POST - /v1/ch/{cmHandle}/data/ds/ncmp-datastore:passthrough-operational DmiPluginApi.dataAccessPassthroughOperational()	Get resource data from passthrough-operational for cm handle. Will support read operations only.	200 400 401 403	200 (ok) for success 400 (bad request) for input operation that is different from read 500 (internal server error) for any SDNC response code that is different from 200 (ResourceDataNotFound) for unexpected system errors	Implementation: The exception ResourceDataNotFound is missing the Exception suffix
5	DMI Plugin	POST - /v1/ch/{cmHandle}/data/ds/ncmp-datastore:passthrough-running DmiPluginApi.dataAccessPassthroughRunning()	Post request to Get, Create or to Update resource data for a cm-handle. Since all requests need to include additional information in a request body HTTP Post is used for all use cases and the actual operation is defined in the request body instead.	201 400 401 403	200 (ok) for success with read or update requested operation. Response body is SDNC body response. 201 (created) for success with create requested operation. Response body is SDNC body response. 204 (no content) for success with delete requested operation. Response body is SDNC body response. 400 (bad request) for requested operation that is different from create, read, update or delete. Response body is SDNC body response. 500 (internal server error) for any SDNC response code that is different from 2xx when writing data (DmiException) for any SDNC response code that is different from 200 when getting data (ResourceDataNotFound) for unexpected system errors	Specification Should success always be 200 (general ok) instead of 201 (created) ? Implementation It does not seem consistent to return 404 with SDNC body response when operation is different from create, read update or delete. SDNC response codes are handled differently for getting and writing data

CPS-Temporal

(commit id 13483eacf395cae83239dd0f17114986b9476d19)

#

Sub interface

Method

Scenario

Specified HTTP Response Code

Implemented HTTP Response Code

Comments

1

Query

GET - /v1/dataspaces/{dataspace-name}/anchors/{anchor-name}/history

CpsTemporalQueryApi.getAnchorDataByName()

Get anchor data by name

200

400

401

403

200 (ok)

for success

400 (bad request)

for input data validation errors

500 (internal server error)

for unexpected system errors

2

Query

GET - /v1/dataspaces/{dataspace-name}/anchors/history

CpsTemporalQueryApi.getAnchorsDataByFilter()

Get anchors data based on filter criteria

200

400

401

403

200 (ok)

for success

400 (bad request)

for input data validation errors

500 (internal server error)

for unexpected system errors

General comments for all endpoints

Specification

Refer to https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml#http-status-codes-1

2xx: Success - The action was successfully received, understood, and accepted
4xx: Client Error - The request contains bad syntax or cannot be fulfilled
5xx: Server Error - The server failed to fulfill an apparently valid request

Always ensure that client request related errors return 4xx along with a clear explanation provided to the client about the reason for the error. When 4xx errors occur, the system itself is behaving as expected.
500 response codes should only correspond to unexpected errors that require support investigation and effort in order to be fixed (ex: network communication issue, dependent component or system down, corrupted data, application bug, ...). For 500 errors no details are really expected in the response for the client, all the information to investigate the problem should be available in the logs.
Review 401 and 403 responses code from specification that are not implemented yet.
500 needs to be added to the specification for all endpoints

Implementation

Is NotFoundInDataspaceException used ? If not, could it be removed ?
Handling of DataNodeNotFoundException in CpsRestExceptionHandler supports a different return code (404) when http method is get. Is this implementation used ? If not, could it be removed ?