CPS-3241: Support Multiple Data Producer IDs — Requirements Gathering Prep

CPS-3241: Support Multiple Data Producer IDs — Requirements Gathering Prep

References

Summary

Today a CM Handle has a single dataProducerIdentifier; CPS-3241 requires to add a list of alternative producer identifiers and fails over to the next one when the primary returns a server-side (5xx) error, promoting the working one and emitting an LCM event.

Requirements

Scope

Current

Requirement

Open issues

Decisions

Scope

Current

Requirement

Open issues

Decisions

1

Registration model

CM Handle holds a single dataProducerIdentifier (String), leaf data-producer-identifier.

No concept of alternative producers. (YangModelCmHandle, NcmpServiceCmHandle)

Add otherDataProducerIdentifiers

  1. Can an entry in otherDataProducerIdentifiers equal the primary or entries should be unique?

    1. If entries need to be unique, fail over duplication?

  2. Order of otherDataProducerIdentifiersimportant? i.e. when used to swap for the primary, first takes priority

 

2

Registration - Create

Bulk operation; each CM Handle succeeds/fails independently with per-handle error responses.

Persist otherDataProducerIdentifiers on create using the model rules.

  1. Validation failure fails only the affected CM Handle; others in the batch succeed. - same as current behaviour?

  1. same behaviour as current

3

Registration - Update

CmHandleRegistrationServicePropertyHandler updates dataProducerIdentifier and already emits an LCM event on change (skips if unchanged). Collection fields use "replace list content".

Full-replace semantics for the list (consistent with existing collection fields). Emit LCM event only when the list changes. New value rides along in the existing LCM V2 payload map, no event schema change.

 

Agreed, same behaviour as current

Csaba Kocsis

4

CM Handle read / query output

CM Handle GET (full and lightweight) and the lightweight query service return the single dataProducerIdentifier. (RestOutputCmHandleMapper, ParameterizedCmHandleQueryService)

Decide whether the otherDataProducerIdentifiers list is exposed in read/query responses. See contract before/after below.

Include the "others" list in GET / query output, or only the primary?

 

5

Legacy data; try-other

Synchronous single-handle read/write resolve one CM Handle and call DMI once; on failure a DmiClientRequestException (with NcmpResponseStatus) surfaces to the caller. (DmiDataOperations) DMI endpoint resolved by dmi-service-name; the dataProducerIdentifier is sent in the payload and used for DCM grouping (DataProducerId + DMI plugin). Multi-handle async/topic path = batch.

Minimal change: same read/write behaviour, only add failover. On primary failure, retry against the next otherDataProducerIdentifier in order.

If all fail → unchanged error (last attempt's error surfaces as today).

Batch/async out of scope.

  1. Trigger scope: which failures qualify as "5xx / server-side"? Any 5xx, or a specific code (e.g. 505 gateway)?

  2. On successful failover: persist a promotion or per-request-only?

  3. Is a successful retry transparent to the caller (clean 200), or is failover signalled in the response/log

  4. Is there a max "others" tried per request + cumulative timeout budget if the list grows beyond 1?

Minimal-change behaviour

  1. awaiting Csaba Kocsis

  2. persist a promotion, update cm handle’s primary to the working dataProducerId.

  3.  

  4.  

6

Promote primary / other & LCM event

Swap + LCM-emit logic exists only inside the registration-update flow (updateDataProducerIdentifier), using updateCmHandleField + CmHandleTransitionPair + lcmEventProducer.

Provide a reusable "promote/swap primary ↔ other, persist, emit LCM event" operation callable from both registration-update and the try-other data paths.

  1. Old primary on promotion: demote into others list (swap) or drop it?

  2. Transient-5xx: promote on first 5xx or after N failures? Reversible?

  3. Concurrency: if two requests hit the primary and both 5xx at once, both try-other and both promote: guard against duplicate promotions / duplicate LCM events / DB write races

  4. Recovery: when a demoted old primary recovers, nothing promotes it back (reactive-only); is "sticky until it also fails" acceptable long-term, or is rebalancing expected

Swap triggers LCM event Csaba Kocsis

  1.  

7

ProvMnS - intercept error, then try-other

ProvMnS is forward-only: NCMP does not inspect the DMI result, so it cannot react to a 5xx. Retrieval requires a non-blank dataProducerIdentifier (ProvMnSCmHandleRetriever → UNPROCESSABLE_CONTENT otherwise); the request is built from the CM Handle in ParametersBuilder.

First make NCMP intercept/detect the DMI error (stop being forward-only). Then apply the same try-other + promote mechanism as the legacy path.

(1) Intercepting the error is a prerequisite for try-other — confirm the two are sequenced (intercept first) and the reaction mirrors the legacy path.
(2) How is the error intercepted on the forward-only path — buffer the full DMI response to inspect status (latency/memory cost), or inspect status code only?

Sequence intercept before try-other; reuse legacy mechanism. To be confirmed.

8

DCM- promote primary / other (Java)

DCM groups sub-jobs via ProducerKey, where the data producer id is concatenated with the DMI plugin as dmiServiceName#dataProducerIdentifier . DCM does its own call/error handling.

NCMP provides a new Java method DCM calls to promote the next "other" (incl. LCM event). DCM implements its own try-other and invokes it. REST outside our scope.?

  1. Promoting/swapping the producer id changes the ProducerKey (dmiServiceName#dataProducerIdentifier), so sub-jobs re-group ,confirm this is the intended effect.

NCMP provides the Java method only (signed off: model interfaces not affected).

9

Other events

LCM V2 payload is a generic map; adding a tracked property needs no schema change (verified in PayloadFactory).

Confirm which existing events (beyond LCM) are affected and whether any schema impact exists.

Are any non-LCM events affected?

Csaba Kocsis expects none - To be verified.

10

Documentation

Docs describe the single dataProducerIdentifier and REST output.

Document the new field, registration contract, try-other / promotion behaviour, and REST output changes.

-

 

11

YANG registry schema

leaf data-producer-identifier { type string; }

backward compatible

 

 

12

REST output

  • RestOutputCmHandle (full GET)

  • RestOutputCmHandleLightweight (lightweight GET / southbound search)

 

 

expose otherDataProducerIdentifiers in GET and query responses, or keep it registration-only and return just the primary on read?

 

Routing note (important for how "try-other" works): within NCMP the DMI endpoint is resolved from the CM Handle's dmi-service-name (DmiServiceNameResolver.resolveDmiServiceName(...)), not from dataProducerIdentifier. The producer id is carried in the request payload and used as a grouping key (ProducerKey = dmiServiceName#dataProducerIdentifier). So swapping to an "other" producer id changes the payload/grouping NCMP sends, but the NCMP→DMI route is unchanged - whether that actually redirects to the other E// depends on how the DMI/adapter uses the producer id (adapter changes are out of scope).

Out of scope

  • Legacy Batch (fire & forget, async, Kafka results).

  • Model endpoint / model interfaces.

  • DCM implementation costs (NCMP provides the Java method only).

  • Adapter changes to support ProvMnS use cases.