Background
To query large outputs or large number of fragments across all anchors, we need to provide pagination for Query API( query data nodes across all anchors). it will help client to fetch limited set of data per request. below is two possible solutions for further discussions to decide appropriate choice.
Pagination over fragments
Implementation
API : GET http://<IP>:<PORT>/cps/api/v2/dataspaces/{dataspace-name}/nodes/query?cps-path={cps-path}&descendants={descendants}&pageIndex={pageIndex}&pageSize={pageSize}
Request Parameters:
| Parameter | isRequired | Description |
|---|---|---|
| cps-path | yes | node path to be queried |
| descendants | no | Number of descendants to be queried. default is none |
| pageIndex | no | page index starting from 1. |
| resultSize | no | number of fragments per page starting from 10 (TBD). |
Query
| Query | Description |
|---|---|
| SELECT id, anchor_id AS anchorId, xpath, parent_id AS parentId, CAST(attributes AS TEXT) AS attributes FROM FRAGMENT WHERE xpath ~ :xpathRegex ORDERED BY id LIMIT :resultSize OFFSET (:resultIndex - 1) * :resultSize | Query limited fragments in order of fragment id |
Pagination over anchors
we can provide pagination over anchors where client can choose number of anchors per request to fetch fragments from.
Implementation
API : GET http://<IP>:<PORT>/cps/api/v2/dataspaces/{dataspace-name}/nodes/query?cps-path={cps-path}&descendants={descendants}&pageIndex={pageIndex}&pageSize={pageSize}
Request Parameters:
| Parameter | isRequired | Description |
|---|---|---|
| cps-path | yes | node path to be queried |
| descendants | no | Number of descendants to be queried. default is none |
| pageIndex | no | page index starting from 1. |
| pageSize | no | number of records per request. It is nothing but number of anchors to be queried. Default value is 5(TBD). |
Response Headers
| Header | description |
|---|---|
| totalResults | total number of available records (achors) |
| totalPages | total nulber of pages. |
Query
| Query | Description |
|---|---|
| SELECT id FROM anchor order by id LIMIT : pageSize OFFSET : (pageIndex -1) * pageSize | query list of anchors |
| SELECT id, anchor_id AS anchorId, xpath, parent_id AS parentId, CAST(attributes AS TEXT) AS attributes FROM FRAGMENT WHERE anchor_id IN :anchorList AND xpath ~ :xpathRegex | query fragments for list of anchors from above query |