CPS-2141 Define DMI REST interface for Datajobs

Issues & Decisions

Issue

Notes 

Decision

Issue

Notes 

Decision

1

Is the requestId in the southbound request equal to the dataJobId coming in from the northound interfaces? (see writeJob for example)

Can the dataJobId used as the requestId? @Csaba Kocsis 

Aug 1, 2024 

@Csaba Kocsis and @Toine Siebelink  agreed that requestId should be renamed to jobId on all interfaces

NCMP → DMI

As part of the overall datajob solution there must be an interface for data flow between the NCMP-lib in DCM and the DMI Plugin.

This story involves the design and implementation of that interface for both the DMI request and response.

  • Define new REST interface :

    • Method: POST

    • Path: /dmi/v1/cmwriteJob/{requestId}

Only writeJob required for MVP. Read, Status, Result will be completed in a later iteration.

Acceptance Criteria (pending stakeholder agreement):

  1. New Interface DMI-I-02 with document and diagram updates. Both Wiki and RTD

  2. Define swagger for request/response (additional info for forwarding data job request : moduleSetTag, privateProperties, dataProducerId, dataJobId)

  3. Agree and implement yaml for this interface

  4. Stakeholder acceptance of yaml

Proposed DMI REST Interface & Forwarded DMI Java Request Data Structure from NCMP
openapi: 3.0.3 info: title: NCMP Data Subjob API description: Support datajobs through one or more subjob for each DMI and Data Producer Identifier combination version: 1.0.0 servers: - url: /dmi tags: - description: DMI plugin rest apis name: dmi-datajob paths: /v1/readJob/{requestId}: post: description: Create a read request operationId: readDataJob parameters: - $ref: '#/components/parameters/requestIdInPath' requestBody: description: Operation body content: application/3gpp-json-patch+json: schema: $ref: '#/components/schemas/SubjobReadRequest' tags: - dmi-datajob responses: "200": description: Response for subjob content: application/json: schema: type: object properties: dataProducerJobId: type: string description: The data job ID. /v1/writeJob/{requestId}: post: description: Create a write request operationId: writeDataJob parameters: - $ref: '#/components/parameters/requestIdInPath' requestBody: description: Operation body content: application/3gpp-json-patch+json: schema: $ref: '#/components/schemas/SubjobWriteRequest' tags: - dmi-datajob responses: "200": description: Response for subjob content: application/json: schema: type: object properties: dataProducerJobId: type: string description: The data job ID. components: parameters: requestIdInPath: description: Identifier for the overall Datajob in: path name: requestId required: true schema: example: some-identifier type: string schemas: SubjobReadRequest: type: object required: - dataProducerId - data properties: dataAcceptType: description: Defines the data response accept type example: application/vnd.3gpp.object-tree-hierarchical+json type: string dataContentType: description: Defines the data request content type example: application/3gpp-json-patch+json type: string dataProducerId: description: ID of the producer registered by DMI for the paths in the operations in this request example: my-data-producer-identifier type: string data: example: op: read operationId: 1 path: SubNetwork=Europe/SubNetwork=Ireland/MeContext=NR03gNodeBRadio00003/ManagedElement=NR03gNodeBRadio00003/GNBCUCPFunction=2 attributes: userLabel scope: scopeTyp: BASE_ONLY type: array items: type: object required: - path - op properties: path: description: Defines the resource on which operation is executed example: SubNetwork=Europe/SubNetwork=Ireland/MeContext=NR03gNodeBRadio00003/ManagedElement=NR03gNodeBRadio00003 type: string op: description: Describes the operation to execute example: read type: string operationId: description: Unique identifier for the operation within the request example: 1 type: string attributes: description: This parameter specifies the attributes of the scoped resources that are returned type: array items: example: cellId type: string fields: description: This parameter specifies the attribute fields of the scoped resources that are returned type: array items: type: string filter: description: This parameter is used to filter the scoped Managed Objects. Only Managed Objects passing the filter criteria will be fetched example: NRCellDU/attributes/administrativeState==LOCKED type: string scopeType: description: ScopeType selects MOs depending on relationships with Base Managed Object example: BASE_ONLY type: string scopeLevel: description: Only used when the scope type is BASE_NTH_LEVEL to specify amount of levels to search example: 0 type: integer moduleSetTag: description: Module set identifier example: my-module-set-tag type: string cmHandleProperties: description: Private properties of the cm handle for the given path $ref: '#/components/schemas/CmHandleProperties' SubjobWriteRequest: type: object required: - dataProducerId - data properties: dataAcceptType: description: Defines the data response accept type example: application/vnd.3gpp.object-tree-hierarchical+json type: string dataContentType: description: Defines the data request content type example: application/3gpp-json-patch+json type: string dataProducerId: description: ID of the producer registered by DMI for the paths in the operations in this request example: my-data-producer-identifier type: string data: example: op: add path: SubNetwork=Europe/SubNetwork=Ireland/MeContext=NR03gNodeBRadio00003/ManagedElement=NR03gNodeBRadio00003/GNBCUCPFunction=1/EUtraNetwork=1/EUtranFrequency=12 value: id: 12 attributes: userLabel: label12 type: array items: type: object required: - path - op properties: path: description: Defines the resource on which operation is executed example: SubNetwork=Europe/SubNetwork=Ireland/MeContext=NR03gNodeBRadio00003/ManagedElement=NR03gNodeBRadio00003 type: string op: description: Describes the operation to execute example: add type: string operationId: description: Unique identifier for the operation within the request example: 1 type: string moduleSetTag: description: Module set identifier example: my-module-set-tag type: string cmHandleProperties: description: Private properties of the cm handle for the given path $ref: '#/components/schemas/CmHandleProperties' value: description: Value dependent on the op specified. Resource for an add. Object for a replace. ActionParameters for an action. type: object oneOf: - $ref: '#/components/schemas/Resource' - $ref: '#/components/schemas/ActionParameters' - $ref: '#/components/schemas/Object' CmHandleProperties: description: Private properties of the cm handle for the given path type: object Resource: type: object properties: id: description: Identifier of the resource object example: resource-identifier type: string attributes: description: Key value map representing the objects class attributes and values type: object additionalProperties: example: 'userLabel: label11' type: string ActionParameters: description: The input of the action in the form of key value pairs type: object additionalProperties: type: string Object: type: object



Data SubJob create (NCMP → DMI)

This is a mirror of the Datajob read/write request with the added details needed for the DMI plugins. Below you can see the changes for the READ but the same changes are applicable for the WRITE too.

Method: POST

Path: /dmi/v1/cmreadJob



DMI Data SubJob READ request

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

destination

The destination of the results. ( e.g. S3 Bucket)

Query

string

N

dataaccepttype







Define the data response accept type. Passible values:

·        application/vnd.3gpp.object-tree-hierarchical+json (default)

·        application/vnd.3gpp.object-tree-flat+json

request body

enum

N

datacontenttype

Define the data request content type. Passible values:

·        application/3gpp-json-patch+json (default)

request body

enum

N

dataProducerId

ID of the producer registered by DMI for the alernateIDs in the operations in this request.

request body

String

Y

dataJobId

Identifier for the overall Datajob

request body

String

Y

data

List of operations to be executed. 

request body

List of DMI3gppReadOperation

Y

DMI3gppReadOperation

Name

Description

Type

Mandatory

Name

Description

Type

Mandatory

path

It is a unique identifier of a managed object (MO) on a network element. 
Defines the resource on which operation is executed. Typically could be Fully Distinguished Name (FDN).

String

Y

op

Describes the operation to execute. The value can be: "read" 

String

Y

operationId

Unique identifier of the operation within the request

Integer

N

attributes

This parameter specifies the attributes of the scoped resources that are returned. 

List of String

N

fields

This parameter specifies the attribute fields of the scoped resources that are returned.

This should be used if an attribute is a struct and only a subset of its fields should be returned.

List of String

N

filter

The parameter is used to filter the scoped Managed Objects. Only Managed Objects passing the filter criteria will be fetched.

List of String

N

scopeType

ScopeType selects MOs depending on relationships with Base Managed Object.



enum[BASE_ONLY, BASE_ALL, BASE_NTH_LEVEL, BASE_SUBTREE]

N

scopeLevel

Only used when the scope type is BASE_NTH_LEVEL

Integer

N

moduleSetTag

Module set identifier

String

N

privateProperties

CMHandle private properties for the alternateID in the path.

Map

N

Response

Status code: 200 - OK

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

dataProducerJobId

The ID of the created job by the data producer.

response body

String

Y

dmiPluginName

The target DMI that the job was performed on

response body

String

Y

dataProducerJobID

ID of the job created by the data producer

response body

String

Y



DMI Data SubJob WRITE request

Method: POST

Path: /dmi/v1/cmwriteJob

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

destination

The destination of the results. ( e.g. S3 Bucket)

Query

string

N

dataaccepttype

Define the data response accept type. Passible values:

·        application/vnd.3gpp.object-tree-hierarchical+json (default)

·        application/vnd.3gpp.object-tree-flat+json

request body

enum

N

datacontenttype

Define the data request content type. Passible values:

·        application/3gpp-json-patch+json (default)

request body

enum

N

dataProducerId

ID of the producer registered by DMI for the alernateIDs in the operations in this request.

request body

String

Y

dataJobId

Identifier for the overall Datajob

request body

String

Y

data

List of operations to be executed. 

request body

List of DMI3gppWriteOperation

Y

DMI3gppWriteOperation

Name

Description

Type

Mandatory

Name

Description

Type

Mandatory

path

It is a unique identifier of a managed object (MO) on a network element. 
Defines the resource on which operation is executed. Typically could be Fully Distinguished Name (FDN).

String

Y

op

Describes the operation to execute. The value can be:

"add": creates a new MO with the id and attributes given in the value
"replace":  replaces the value of an attribute specified in the path (attribute is separated with a # from the MO path) with the object in the value
"remove": removes the MO given in the path, value is not applicable
"action": executes an MO action in the path (action is separated with a # from the MO path) with the input parameters in the value

String

Y

operationId

Unique identifier of the operation within the request

Integer

N

value

NA if op == remove

Resource if op == add

Object if op == replace

ActionParameters if op == action





Object

N

moduleSetTag

Module set identifier

String

N

privateProperties

CMHandle private properties for the alternateID in the path.

Map

N



Response

Status code: 200 - OK

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

dataProducerJobId

The ID of the created job by the data producer.

response body

String

Y

dmiPluginName

The target DMI that the job was performed on

response body

String

Y

dataProducerJobId

ID of the job created by the data producer

response body

String

Y



Data subjob status check (NCMP → DMI)

Request:

Method: GET

Path: /dmi/v1/cmwriteJob/dataProducer/{dataProducerd}/dataProducerJob/{dataProducerdJobId}/status

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

dataProducerId

ID of the producer registered by DMI for the alernateIDs in the operations in this request.

Path

String

Y

dataProducerJobId

Identifier of the job created by the data producer.

Path

String

Y

Response:

Status code: 200 - OK

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

status

enum that represents the status of the subjob.
Possible values:

response body

enum:
[ NOT_STARTED, RUNNING, FINSHED, FAILED, PARTIALLY_FAILED, CANCELLING, CANCELLED ]

Y



Data subjob retrieve result (NCMP → DMI)

Request:

Method: GET

Path: /dmi/v1/cmwriteJob/dataProducer/{dataProducerd}/dataProducerJob/{dataProducerdJobId}/result?destination={destination}

Parameters:

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

dataProducerId

ID of the producer registered by DMI for the alternateIDs in the operations in this request.

Path

String

Y

dataProducerJobId

Identifier of the job created by the data producer. (not globally unique)

Path

String

Y

destination

The destination of the results: Kafka topic name or s3 bucket name. This shall be put into the Kafka message headers returned to NCMP

Query

String

Y

Response:

WRITE

Status Code: 200 - Ok

Data part of the message

READ

Status Code: 200 - Ok

Name

Description

Location

Type

Mandatory

Name

Description

Location

Type

Mandatory

resulturi

internal ncmp topic name

response body

String

Y