Support for Async datajobs

Not just for reading specific fdn, but rather QUERY Group of FDN , it's just a broadcast to every DMI plugin. The response should mimic sending a broadcast to 2 or more CM Handles 

CPS Team will only do java interface. REST Interface is done in DCM

NEW interface aligning with 3GPP i.e FDN instead of CM-HandleIds

(Read use case can re-use existing dataOperationz impl. after mapping FDNs to CMHandleIds for input and back for output!)

Read, Create, Update, Delete and Action support. I.e for passthrough only
Note: Q(uery) is  SUBNETWORK-wide read and should be done separately using a different endpoint for clarity and separation!

1. NCMP to introduce a qualifier to be used along with the DMI plugin so NCMP can break the request with multiple cmhandle into batches based on the DMI plugin and the Qualifier (where qualifier should be EMS name / id).

2. NCMP shall create one or multiple EMS job ids depending on FDNs requested

3. NCMP shall provide an interface to get status of an EMS job id (forward request)

4. NCMP shall provide an interface to get results of an EMS job id

5. NCMP shall send the results to Kafka topic

6. Synchronous

26 Jun 2024 

Add as part of this epic

• Status

• Result

No new epic for write req.

Update existing/legacy NCMP APIs to support FDN / alternateId

• CPS-NCMP

• Seán Beirne Gerard Nugent

• Ericsson Team 2

• Q4 24

Update remaining existing/legacy NCMP APIs to support alternateId (FDN)

• (EST) Q1 25 (ESH)

CM-handle search that returns 200k Cells (50k CM Handles)

https://lf-onap.atlassian.net/issues/?jql=parent%20%3D%20CPS-21698

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-

Now includes 

• 2010

• CPS-NCMP

• Sourabh Sourabh

• Ericsson Team 1

• Q1 25

• Async datajob Read to S3 (Minio impl)
  Csaba Kocsis / kieran mccarthy working on requirement, date tbc

NCMP to introduce a qualifier to be used along with the DMI plugin so NCMP can break the request with multiple cmhandle into batches based on the DMI plugin and the Qualifier (where qualifier should be EMS name / id).

• NCMP shall create one or multiple EMS job ids depending on FDNs requested

• NCMP shall provide an interface to get status of an EMS job id (forward request)

• NCMP shall provide an interface to get results of an EMS job id

• NCMP shall send the results to S3

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-

• 1992

• CPS-NCMP

• Priyank Maheshwari

• Ericsson Team 2

• IDUN-46098

• Q1 25

• NCMP to support the 3GPP ProvMnS CRUD interfaces.

• NCMP to Support new 3GPP sync single FDN request

Implement new NCMP CRUDAQ sync interface aligned with 3GPP API (Read and write use cases) - Wrapper on existing/legacy API

10

• CPS-NCMP

• Lee Anjella Macabuhay

• Ericsson

CM-handle search that returns 200k Cells (50k CM Handles)

• Team 2

• Q2 25'

Forwarding CM Data Notifications to Topic in Subscription.

250 million CM notifications per day that need to be processed and distributed to 100-200 different topics

• Topic used for forwarding notification should be based on the client-id which can be extracted from the subscription-id - Yet to be clarified 03 Apr 2024 

• Client application shall ONLY receive notification they subscribed on

• Client application shall not see notification they did not subscribed on

Interdependent on -

Async datajob Read to S3 (Minio impl)
Requirement not yet clarified

NCMP to introduce a qualifier to be used along with the DMI plugin so NCMP can break the request with multiple cmhandle into batches based on the DMI plugin and the Qualifier (where qualifier should be EMS name / id).

• NCMP shall create one or multiple EMS job ids depending on FDNs requested

• NCMP shall provide an interface to get status of an EMS job id (forward request)

• NCMP shall provide an interface to get results of an EMS job id

• NCMP shall send the results to S3

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1992

• CPS-NCMP

• TBC

• TBC

• IDUN-46098

• Q1 25

NCMP to support the 3GPP ProvMnS CRUD interfaces. NCMP to Support new 3GPP sync single FDN request

Implement new NCMP CRUDAQ sync interface aligned with 3GPP API (Read and write use cases) - Wrapper on existing/legacy API

Async datajob Read (Write output to Kafka)

TBC: Stakeholders needs to check validity of this requirement.

Dropdown from #7 due to the whole CM Data Notification Subscription is estimated for Q2 25 by //  therefore deprioritized

Invoke YANG modelled synchronous action

Sync Actions

Invoke YANG modelled action

Invoke YANG modelled RPC, Specification required. Reboot/Reset type of actions on node. Include to the sync one

Always on operational datastore.  Supported for nmcp:passthrough-operational and if executed against ncmp:operational then it is always forwarded to dmi plugin.  Is there another story for forwarding to be included as a dependency? Always run as sync request.  Is this dependent on CPS-1127  - see spin-off user stories table below this on.

KMC : Can we deprioritize - this can be run against passthrough-operational for now.  Just have to agree on the API / URL for the action to progress at this stage so that the passthrough-operational form is aligned with final operational form.

(S) - for passthrough. 

*Spec out before Sept'23. No implementation. 

09 Nov 2023 

can datajob cover this ?, currently no support for 'actions'. Action name at the end of resourceid. split ticket into, action with and without responses. 

12

CPS-NCMP Test & documentation Improvement

• CPS-NCMP

• Toine Siebelink

• Ericsson Team 1

• Q1 25'

Retro action: Make test, documentation (examples) more realistic

1. use of ‘bookstore example' where it’s a problem

2. add or replace with telecoms examples

3. Additional scope, consider;

   1. unit test

   2. acceptance test (groovy, semi-integration)

   3. CPS integration test

   4. documentation

4. Sync with internal stakeholders

13

• CPS-NCMP

• Ericsson Team 

• Q2 25'

Forwarding CM Data Notifications to Topic in Subscription

• Topic used for forwarding notification should be based on the client-id which can be extracted from the subscription-id - Yet to be clarified 03 Apr 2024 

• Client application shall ONLY receive notification they subscribed on

• Client application shall not see notification they did not subscribed on

Interdependent on - Subscription with DME interface

• This new epic is driven by the possible changes from Event based subscription to HTTP rest interface and potential changes to the schema.

• The whole CM Data Notification Subscription is estimated for Q2 25 by //  therefore deprioritized

• Doc still in study (2/3)

• CPS to study the document when study is approved

Newly Added

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1915

• CPS-

Dropdown from #7 due to the whole CM Data Notification Subscription is estimated for Q2 25 by //  therefore deprioritized

• CPS-NCMP

• Ericsson Team 

• Q2 25'

CM Subscription with DME interface

• This new epic is driven by the possible changes from Event based subscription to HTTP rest interface and potential changes to the schema.

• The whole CM Data Notification Subscription is estimated for Q2 25 by //  therefore deprioritized

• Doc still in study (2/3)

• CPS to study the document when study is approved

Newly Added

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1915

• CPS-NCMP

• TBD

• Ericsson  Team 2

• TBD

• Q3 25

• NCMP

• TBD

• Ericsson  Team 2

• TBD

• Q3 25

TBAC - Access Control for resources to ensure that operators can restrict access control to only those users (human/machines) that are authorized to execute CRUD operations on those resources. 

TBAC Study still ongoing, schedule an internal meeting to go through study doc, until sidecar is well define and implemented cps can't do nothing. Sidecar should specify the interfaces.

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1781

• CPS-NCMP

• TBD

• TBD

CM data subscriptions from application to DMI [Part 2].

For all cmhandle (general)

This epic was created to take on additional scope which got added to CPS-1616. 

https://lf-onap.atlassian.net/issues/?jql=parent%20%3D%20CPS-178116

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1965

• CPS-NMCP

• TBC

• TBC

AVC Subscription, advance filter. Part 2 of cmhandles

It includes creating subscription with patternmatch cmhandles. 

Filter on 'Type' instead of list of CM Handle IDs → 'Type' could be defined as the yang module set containing a specific module (name and version)

1617

• TBC

• CPS-NMCP

• TBC

• TBC

Event Digest

Additional field to help clients filter CM AVC Events
(S)

1718

TBC

Support NCMP-CPS upgrade

Currently only custom upgrade is supported. (upon request)

Requirement: It shall be possible to upgrade NCMP-CPS from release N-1 to N (without requiring manual intervention/workarounds). N is defined as any release requested by ESH

• Note. Need to agree version strategy: use current ONAP x.y.z. numbering. Ericsson to communicate when a version is to be 'delivered' and 'y' increased

Technical Debt to be addressed: Liquibase is used in CPS to manage data(upgrades) in CPS

Study: Resolve technical debt (mixed data). NCMP Data upgrade. CPS Core need to support model upgrade so that NCMP can use it,

08 Nov 2023 Liquibase is used in CPS to manage data(upgrades) - Now available.

• Still need to discuss the 'backward incompatible' → What interfaces should NOT be impacted Kieran mention NBI - northbound interfaces

• Propose workshop, Spike needed from CPS

(XL) - Scope needs to be defined. Risk is scope not identified, efforts might increase.

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1796

• CPS-CORE & NCMP

• TBC

• TBC

Spike for documenting Kafka interfaces using AsyncAPI

• Documentation Generation

• Interface Naming

• Cloud Events specifics asyncapi-cloud-events - Roll out for legacy events

• Code Generation (contract first, stubs)

• . numbering. Ericsson to communicate when a version is to be 'delivered' and 'y' increased

Technical Debt to be addressed: Liquibase is used in CPS to manage data(upgrades) in CPS

Study: Resolve technical debt (mixed data). NCMP Data upgrade. CPS Core need to support model upgrade so that NCMP can use it,

08 Nov 2023 Liquibase is used in CPS to manage data(upgrades) - Now available.

• Still need to discuss the 'backward incompatible' → What interfaces should NOT be impacted Kieran mention NBI - northbound interfaces

• Propose workshop, Spike needed from CPS

(XL) - Scope needs to be defined. Risk is scope not identified, efforts might increase.

Refactor legacy NCMP ASync Response Events to use Cloud Events format

(M) TBC 

can datajob cover this ?, currently no support for 'actions'. Action name at the end of resourceid. split ticket into, action with and without responses. 

• Jira Legacy
  server System Jira

• serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key

Access control for topics which are created by NCMP.

Spike needs to be conducted. Dependent of TBAC implementations.

Invoke YANG modelled sync action

Invoke YANG modelled action

Invoke YANG modelled RPC, Specification required. Rebbot/Reset type of actions on node. Include to the sync one

Always on operational datastore.  Supported for nmcp:passthrough-operational and if executed against ncmp:operational then it is always forwarded to dmi plugin.  Is there another story for forwarding to be included as a dependency? Always run as sync request.  Is this dependent on CPS-1127  - see spin-off user stories table below this on.

KMC : Can we deprioritize - this can be run against passthrough-operational for now.  Just have to agree on the API / URL for the action to progress at this stage so that the passthrough-operational form is aligned with final operational form.

(S) - for passthrough. 

*Spec out before Sept'23. No implementation. 

09 Nov 2023 

• CPS-1796

• CPS-CORE & NCMP

• TBC

• TBC

Spike for documenting Kafka interfaces using AsyncAPI

• Documentation Generation

• Interface Naming

• Cloud Events specifics asyncapi-cloud-events - Roll out for legacy events

• Code Generation (contract first, stubs)

Add label of techdebt Kolawole Adebisi-Adeolokun not an immediate req for

20

• Jira Legacy
  server System Jira serverId 4733707d-2057-3a0f-ae5e-4fd8aff50176 key CPS-1704

• CPS-NMCP

• TBC

• TBC

Refactor legacy NCMP ASync Response Events to use Cloud Events format

(M) TBC 

Access control for topics which are created by NCMP.

Spike needs to be conducted. Dependent of TBAC implementations.

22

Invoke YANG modelled Asynchronous action

Split from [CPS-2521] Invoke YANG modelled synchronous action - Jira

23

Enhanced query support (fields)

Currently the passthrough has an 'fields' parameter to do a scoped query.  Propose to support this in non-passthrough so it is promoted to a fully supported option, e.g. {ncmp-root}/ncmp/v1/ch/335ff/data/ds/ncmp-datastore:passthrough-operational?
              resourceIdentifier=/&options=(fields=ericsson-enm-comtop:ManagedElement/ericsson-enm-
              gnbcucp:GNBCUCPFunction/EndpointResource/LocalSctpEndpoint/attributes(sctpEndpointRef),
         

KMC : Do we support restconf like queries or xpath only?

(L) .

*Spec out before Sept'23. No implementation. 

2324

Enhanced query support (scope)

Currently the passthrough has an 'fields' parameter to do a scoped query.  
              scope=ericsson-enm-comtop:ManagedElement/ericsson-enm-gnbcucp:GNBCUCPFunction/
               EndpointResource/LocalSctpEndpoint/attributes(interfaceUsed==X2))

KMC : Do we support restconf like queries or xpath only?

(L)

*Spec out before Sept'23. No implementation. 

2425

TBC

Support ncmp-datastores:running for reading data (single CM handle, synchronous only)

See CPS-391 page for details about supported operations and combinations. Note: There can be some overlap between work items for #5, #6, #11 and #12.

Read from operations.

(S) - Forward only.  No validation or data enhancements (add prefixis)

2526

TBC

Support ncmp-datastores:running for writing data (single CM handle, synchronous only)

(S)  As per #18

2627

TBC

Support relationships for 'Instance Identifier'

Should be possible to identify a cmhandle using multiple instance identifiers.
(M) - Not sure. Scope not known yet. 

Fine-grained cache configuration

2829

Support for HTTPS and authentication

• Use Service-Mesh? Both ONAP and Ericsson (applied ISTIO using Envoy Proxy side-car) are working towards this. Check with Gareth Roper 

Validation required whether this is still needed. 

TBC

Send notifications on write operation in ncmp-datastores:running for (single CM handle, synchronous only)

3031

schema-set update for CM handle with cached data present

Need to address case with incompatible model changes.
Scope: Upgrade of model that is cached?  Lee Anjella to confirm. 

3132

Invoke YANG modelled RPC

Specification required. Rebbot/Reset type of actions on node. 

3233

DMI Audit for DMI restarts

After restart, trustlevel loses all data. TrustLevel is not currently in use now, however this becomes an issues after TrustLevel restart. The states goes to 'NONE' after TrustLevel restart

TBC

Fetch a list of cmhandles along with their private properties as response from NCMP.