Versions Compared

Version Old Version 112 New Version 113
Changes made by

Toine Siebelink

Toine Siebelink

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Overview

Jira Legacy
serverSystem ONAP Jira
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
keyCPS-1628

  • This page gives a description of CPS events including their fields
  • Events will be split into events and Kafka headers

...

#DescriptionNotesDecision
1No Event properties defined for DMI AVC Event

Priyank Maheshwari will need to specify  and agreed event structure for DMI AVC Event with stakeholders  ie. provide Jira ticket


Event Body should be compatible with RFC8641

kieran mccarthy has confirmed.

Priyank Maheshwari created JIRA to create the event body schema. 

2Bulk Operation events details have not yet be defined (just headers)

Sourabh Sourabh to provide Jira tickets

Jira Legacy
serverSystem ONAP Jira
serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
keyCPS-1658

3Should all the events have same kafka headers

kieran mccarthy Possibly Common (base) set of headers but mandatory aspect might differ. In practice we might need a separate headers (schema?) for each event

4Clarify the format of the version eventSchemaVersion

v1 or 1.0

EX: 1.0.0 (without 'v')

kieran mccarthy to check ORAN preference

kieran mccarthy confirms through email on to use semantic versioning which ORAN follows https://semver.org.

5What to do with additional event headers (from DMI Plugins)

kieran mccarthy   if DMI produce additional headers NCMP will discard those i.e. not included in forwarded events

6Event(Content) field in DMI Async Request Response Event has inconsistent name (compared with other schemas)
  1. Add V2 file
  2. Deprecate V1
  3. Support both versions for a while
  4. Delete the V1 version (after some time )

CPS Team Create a V2 of the schema and rename eventContent as event data. Do it as part of the schema addition.

7NCMP Async Request Response Event (#5) contains both an Event and ForwardedEvent

ForwardedEvent is not wrapped inside Event but question now is if we need 2 events at all?!

Sourabh Sourabh  and RAVITEJA KARUMURI (EST)  can check how it is actually working and then we decide ( create a JIRA ticket )

Wiki for the Study on NCMPAsyncRequestResponse event schema

Conclusion: events not designed as proposed, very inconsistent. Never go a bug because these async events aren't used at all (confirmed by Csaba Koscis) Instead bulk request wil be used for topology use cases.

kieran mccarthy and team agreed to:

  1. Temporary disable the legacy async request feature (task created: 
    Jira Legacy
    serverSystem ONAP Jira
    serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
    keyCPS-1694
    )
  2. As part of a lower priority work items (but during Montreal) fix related events with learnings from the new batch-usecase. (task created
    Jira Legacy
    serverSystem ONAP Jira
    serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
    keyCPS-1693
    )
8Dmi Data AVC Event, use of eventSource field

Priyank Maheshwari wanted to store 'datastore' in this field but kieran mccarthy explained it to use for different purposes

kieran mccarthy Clients can use this field as per their requirements. 

9Can Kafka Headers be described with 'schema's owned and managed by NCMP 

POC to follow. 

Jira Legacy
serverSystem ONAP Jira
serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
keyCPS-1657

Defining header schema.

Integration of header with kafka.

Naming and versioning convention for the header schemas. 'id' 

Does the headers schema have a version too?!


 Priyank Maheshwari confirmed headers can be described with a separate schema.

Both header schema's and event schemas will be published on https://docs.onap.org/projects/onap-cps/en/latest/cps-events.html

Header schema name and version will be maintained in the 'id' metadata field of header's schema . 

10Depending #10 can schema inherit/extend a common schema for common headers

Commonly define them and then define what are mandatory(required) or optional as per the schema extending it.

If a field is not used in the extended schema then it should be able to handle it.

Extend the POC ( on #9 ) to cover this.

 Priyank Maheshwari did the POC and the conclusion of that was that :-

  • One schema can extend the other schema.
  • We cannot override the mandatory/optional parameter from the Parent schema. 

Toine Siebelink agrees to go ahead with separate schema/headers per event.  There will be some duplication but it will have its advantage when versioning. 

11Is anyone using Async Request feature?

See

Jira Legacy
serverSystem ONAP Jira
serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
keyCPS-1660
we need to re-design event  #4 (Covers Point 6 as well) and #5 in a backward incompatible way. If no-one is using this feature right now (suspected) we can do this more easily/cheaper!

 Csaba Kocsis confirmed this is not used by Ericsson currently. No plans to use soon for single-cmhandle requests either (TBC). Need to decide priority (Csaba Kocsis to find out of fixing the legacy schema(s)
See decision on issue #7

12Do we need additionalProperties for DMI ASync Data Request respondes (events #4, #5)The original code populates a framework defined 'additionalProperties' field with a singel key-value pair: "response-data",{<json data>}. No other (private) properties are added either in DMI PLugin or NCMP code. The name is just coincidence and misleading. In fact this 'additionalProperties' field should NOT have been used at all!No, the new schema should NOT add  'additionalProperties' field at all use 'additionalProperties:no' in the schema
13AVC Subscription Event (DMI → NCMP) (events #3)
  • Want to understand what 'data' is datatype referring to under Subscription Event?
    • What value comes under 'schemaName' & 'SchemaVersion' of Datatype definition under AVC Subscription Event?
    • Reconfirmation needed on 'schemaName' & 'SchemaVersion'  should be in the payload?

In meeting kieran mccarthy updated #3 is ON HOLD to analyse further. 

Agreed with Toine Siebelink  on that Priyank Maheshwari  will look into this from now as they are working on something related to this.

14Align headers with CNCF Cloud EventsUsing standard headers as defined by Cloud Events and possibly common header extension See Table below, CNFC Cloud Event alignment
CPS will use Cloud Events  3PP for all current and legacy events to ensure common format

kieran mccarthyand Toine Siebelink  agreed on general idea but exact list of common headers need to be agreed 
Jira (first impl.) to be added! 

Jira Legacy
serverSystem ONAP Jira
serverId4733707d425b2b0a-2057557c-3a0f3c0c-ae5eb515-4fd8aff50176579789cceedb
keyCPS-1717

15

During the meeting we saw that the Kafka header fields were prefixed with "ce_ " so need to check if we are ok with that.



16Do will still need/ can we stil use schemas for header details with CNCF library ?! How to publish info about non-standard headers like correlationid ?
17inconsistent casing convention for header v. json dataJust observing. All header ion CNCF are lowercase whereas json field are camelCase. Don't want to change but want to make sure agree...
18Confirm 'source'. is to be added to ALL events (declared mandatory for CNFC events)

...

Expand
titleNCMP Async Request Response Event

NcmpAsyncRequestResponse Event Object

FieldTypeDescriptionKafka HeaderEvent Payload FieldNotesCloudEvents Mapping
eventIDstringThe unique id identifying the event generated by DMIx

standard ( id ) 
eventCorrelationIdstring

The request id passed by NCMP

x

extensions ( correlationid )
eventTimestringThe timestamp when original event occurredx

standard ( time) 
eventTargetstringThe target of the eventx

extensions ( target)
eventTypestringThe type of the eventx

standard ( type ) 
eventSchemastring

The event schema for async request response events

x

standard ( dataschema ) 
eventSchemaVersionstring

The event schema version for async request response events

x

N/A included in 'dataschema'
eventEventThe payload of an event

Defined by event object belowstandard ( source ) 
forwardedEventForwardedEventThe payload of a forwarded event

Relation to Event field unclear, do we need 2 events at all see issue #7

standard ( data ) 

Type: Event (cps:org.onap.cps.ncmp.events:ncmp-async-request-response-event-schema:v1)

FieldTypeDescriptionKafka HeaderEvent Payload FieldNotes
eventIDstringThe unique id identifying the event generated by DMI
xOnly in payload in v1 of the payload schema.  To be removed in v2 version (will be in the header only)
eventCorrelationIdstring

The request id passed by NCMP


xas above
eventTimestringThe timestamp when original event occurred
xas above
eventTargetstringThe target of the event
xas above
eventTypestringThe type of the event
xas above
eventSchemastring

The event schema for async request response events


xas above
eventSourcestring

The source of the event


x
response-data-schemastring

The received schema of response data


x
response-statusstring

The received status of the response


x


response-codestring

The received code of the response


x
response-dataobject

The data payload


xcontains payload of type object

Type: forwardedEvent (cps:org.onap.cps.ncmp.events:ncmp-async-request-response-event-schema:v1)

FieldTypeDescriptionKafka HeaderEvent Payload FieldNotes
eventIDstringThe unique id identifying the event generated by DMI
?Can NCMP put these in the header and remove them from the 'forwardedEvent'
eventCorrelationIdstring

The request id passed by NCMP


?as above
eventTimestringThe timestamp when original event occurred
?as above
eventTargetstringThe target of the event
?as above
eventTypestringThe type of the event
?as above
eventSchemastring

The event schema for async request response events


?as above
eventSchemaVersionstring

The event schema version for async request response events


?as above
eventSourcestring

The source of the event


?as above
response-data-schemastring

The received schema of response data


x
response-statusstring

The received status of the response


x


response-codestring

The received code of the response


x
response-dataobject

The data payload


xcontains payload of type object


Proposed New Schema:
Jira Legacy
server

System

ONAP Jira
serverId

4733707d

425b2b0a-

2057

557c-

3a0f

3c0c-

ae5e

b515-

4fd8aff50176

579789cceedb
keyCPS-1660


FieldTypeDescriptionKafka HeaderEvent Payload FieldNotesCloudEvents Mapping
eventIDstringThe unique id identifying the event generated by DMI


standard (id)
eventCorrelationIdstring

The request id passed by NCMP




extensions(correlationid)
eventTimestringThe timestamp when original event occurred


standard (time)
eventSourcestringThe source of the event

What value should contain in this? Needs clarification, see issue #8standard (source)
eventTypestringThe type of the event


standard (type)
eventSchemastring

The event schema for async request response events




standard (dataschema)
eventSchemaVersionstring

The event schema version for async request response events




N/A included in 'dataschema'
eventEventThe payload of an event

Defined by event object belowstandard (data)

Type: Event (cps:org.onap.cps.ncmp.events:ncmp-async-request-response-event-schema:v1)

FieldTypeDescriptionKafka HeaderEvent Payload FieldNotes
eventCorrelationIdstring

The request id passed by NCMP



as above @Toine asked to add this property.
responseStatusstring

The received status of the response




responseCodestring

The received code of the response




responseDataobject

The data payload



contains payload of type object



...

excerpt
Expand
titleDMI Async Batch Response Event

BatchResponseEvent Object

Excerpt


FieldTypeDescriptionKafka HeaderEvent Payload FieldNotesCloudEvents Mapping
eventIDstringThe unique id identifying the event generated by DMIx

Mandatory

Generated by DMI-Plugin
eventCorrelationIdstring

The request id passed by NCMP

x
Mandatory. This is requestId that NCMP sent to client as an ACK.
Example : “request-1234" (UUID)		extensions (correlationid)
eventTargetstringThe destination topic of the clientx
Mandatory. TBC example: my-topicextensions ( target  TBC)
sourcestringThe source of the event

Optional. Example : dmi-plugin:enm-1 (dmi service name)
eventTimestringThe timestamp when original event occurredx

Optional. The timestamp should follow that on https://www.rfc-editor.org/rfc/rfc3339#section-5. This follows ISO 8601 and is what is used/referenced in 3GPP standards
Example: 1985-04-12T23:20:50.52Z this represents 20 minutes and 50.52 seconds after the 23rd hour of April 12th, 1985 in UTC.

standard ( time) 
eventTypestringThe type of the eventx
Mandatory. NCMP owns eventType that would be used by dmi-plugins
The Convention of eventType and Schema names is under discussion, see open issue #17 on  CPS-1515 Spike		standard ( type ) 
eventSchemastring

The event schema for async request response events

x
Mandatory.
The Convention of eventType and Schema names is under discussion, see open issue #17 on  CPS-1515 Spike
standard ( dataschema ) 
eventSchemaVersionstring

The event schema version for async request response events

x

N/A included in 'dataschema'
eventEventThe payload of an event
xjava object not yet defined by schema, see issue #2standard ( data ) 
specversionStringVersion of CNCF specx
hardcoded value of 1.0standard (hardcode value)



Type: Event (cps:org.onap.cps.ncmp.event.async:dmi-async-batch-response-event-schema:v1)

FieldTypeDescriptionKafka HeaderEvent Payload FieldNotes
No Properties defined (Entire event treated as single object)


...

CNCF Cloud Event alignment

Introduction

CNCF CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems.

  • These cloud events will be applied to all the internal and external events we have in CPS , NCMP and DMI Plugin.
  • Cloud events will be taking care of the fields which are part of kafka headers or part of the actual payload ( fields other than "data" are sent as kafka headers )
  • These CNCF cloud events will be applied to all the events listed in above sections ( LCM , DMI Data AVC etc. )

...

Before

After

CloudeEvent builder methodExample ValueNotes

eventId

id

.withId()
Mandatory

eventSource

source

.withSource()
Mandatory

N/A

specversion  (default 1.0)

.v1()1.0Mandatory  - This is the version of the cloud events

eventType

type

.withType()
Mandatory

eventTime

time

.withTime()
Optional

eventSchema

dataschema

.withDataSchema()

Optional  includes the version of the schema


datacontenttype
withDataContentType()
application/jsonOptional
eventCorrelationIdcorrelationid.withExtension()

Optional This will be part of the extensions field in the cloud events and all the restrictions of the attribute field naming applies to it.

i.e these fields will be in the small case.

This is marked as optional as it only applies to some events.

eventdata.withData(json TBC)
Mandatory actual event/payload now would be under "data" field.

...