Types of Users and Usage Instructions:

1.    Deployment of DCAE MOD components via Helm charts

The DCAE MOD components are deployed using the standard ONAP OOM deployment process.   When deploying ONAP using the helm deploy command, DCAE MOD components are deployed when the dcaemod.enabled flag is set to true, either via a --set option on the command line or by an entry in an overrides file.  In this respect, DCAE MOD is no different from any other ONAP subsystem.

The default DCAE MOD deployment relies on an nginx ingress controller being available in the Kubernetes cluster where DCAE MOD is being deployed.   The Rancher RKE installation process sets up a suitable ingress controller.   In order to enable the use of the ingress controller, it is necessary to override the OOM default global settings for ingress configuration.   Specifically, the installation needs to set the following configuration in an override file:

global:
  ingress:
    enabled: true
    virtualhost:
        enabled: false

When DCAE MOD is deployed with an ingress controller, several endpoints are exposed outside the cluster at the ingress controller's external IP address and port.   (In the case of a Rancher RKE installation, there is an ingress controller on every worker node, listening at the the standard HTTP port (80).)  These exposed endpoints are needed by users using machines outside the Kubernetes cluster.

To access the design Web UI, for example, a user would use the URL : http://ingress_controller_address:ingress_controller_port/nifi.
ingress_controller_address is the the IP address or DNS FQDN of the ingress controller and
ingress_controller_port is the port on which the ingress controller is listening for HTTP requests.  (If the port is 80, the HTTP default, then there is no need to specify a port.)

There are two additional internal endpoints that users need to know, in order to configure a registry client and a distribution target in the design tool's controller settings.

Using DCAE MOD without an Ingress Controller

<to be supplied>

2.    Configuring DCAE mod

The Nifi Registry UI (http://<hostname>/nifi-registry ) should now load if you don’t see errors with the Nifi Registry docker container logs (do docker logs modstandup_nifi-registry_1) and it is now able to find the postgresql database (which is now running as a separate process).

At this point, you can create a new bucket using the wrench tool in the UI and let’s say we name this new-bucket

Creating a bucket is no longer needed.  The Helm installation automatically creates a bucket.

Now let’s access the Nifi (DCAE designer) UI - http://<hostname>/nifi

a. Get the artifacts to test and onboard.

A Sample Request body should be of the type

{ "owner": "<some value>", "spec": <some json object> }

 where the json object inside the spec field can be a component spec json.

You would have to use a request body of this type in the onboarding requests you make using curl or the onboarding swagger interface.

The Sample Request body for a component model-b1 looks like so -

{
"spec": {
"parameters": [],
"artifacts": [
{
"type": "docker image",
"uri": "tlab-nexus.research.att.com:18444/model-b1:1"
}
],
"self": {
"version": "1.0.0",
"name": "model-b1",
"component_type": "docker",
"description": "Automatically generated from Acumos model"
},
"streams": {
"publishes": [
{
"type": "message_router",
"version": "1.0.0",
"config_key": "predict_publisher",
"format": "OutputFormat"
}
],
"subscribes": [
{
"type": "message_router",
"version": "1.0.0",
"config_key": "predict_subscriber",
"format": "PredictIn"
}
]
},
"auxilary": {
"healthcheck": {
"endpoint": "/healthcheck",
"type": "http"
}
},
"services": {
"provides": [],
"calls": []
}
},
"owner": "aoadapter"
}

A Sample request body for a sample data format  looks like so-

{
"spec": {
"self": {
"version": "1.0.0",
"name": "OutputFormat"
},
"jsonschema": {
"definitions": {},
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"metric_values": {
"items": {
"type": "number"
},
"type": "array"
},
"time_stamps": {
"items": {
"type": "string"
},
"type": "array"
},
"anomaly_decisions": {
"items": {
"minimum": -9007199254740991,
"type": "integer",
"maximum": 9007199254740991
},
"type": "array"
},
"metric_name": {
"items": {
"type": "string"
},
"type": "array"
}
},
"title": "OutputFormat"
},
"dataformatversion": "1.0.1"
},
"owner": "aoadapter"
}

b. To onboard a data format and a component

Each component has a description that tells what it does.

These requests would be of the type

curl -X POST -u <user>:<password> http://<onboardingapi host>/onboarding/dataformats     -H "Content-Type: application/json" -d @<filepath to request>

curl -X POST -u <user>:<password> http://<onboardingapi host>/onboarding/components     -H "Content-Type: application/json" -d @<filepath to request>

In our case,

curl -X POST -u acumos:integration2019 http://localhost/onboarding/dataformats     -H "Content-Type: application/json" -d @df-request-model-b1-reporting-subscribe.json

curl -X POST -u acumos:integration2019 http://localhost/onboarding/components     -H "Content-Type: application/json" -d @components-request-model-b1-reporting.json

In a Helm deployment the hostname would not be localhost.  It would be the IP address (or DNS FQDN if there is one) for one of the Kubernetes nodes.

More posting requests for data formats and components

curl -X POST -u acumos:integration2019 http://localhost/onboarding/dataformats     -H "Content-Type: application/json" -d @df-request-model-b1-etl-subscribe.json

curl -X POST -u acumos:integration2019 http://localhost/onboarding/dataformats     -H "Content-Type: application/json" -d @df-request-model-b1-etl-publish.json

curl -X POST -u acumos:integration2019 http://localhost/onboarding/components     -H "Content-Type: application/json" -d @components-request-model-b1-etl.json

curl -X POST -u acumos:integration2019 http://localhost/onboarding/dataformats     -H "Content-Type: application/json" -d @df-request-model-b1-publish.json

curl -X POST -u acumos:integration2019 http://localhost/onboarding/dataformats     -H "Content-Type: application/json" -d @df-request-model-b1-subscribe.json

curl -X POST -u acumos:integration2019 http://localhost/onboarding/components     -H "Content-Type: application/json" -d @components-request-model-b1.json

c. Verify the resources were created using

curl -X GET -u acumos:integration2019 http://localhost/onboarding/dataformats

curl -X GET -u acumos:integration2019 http://localhost/onboarding/components

d. Verify the genprocessor (which polls onboarding periodically to convert component specs to nifi processor), converted the component

Open  http://<genprocessor_host>/nifi-jars in a browser.

These jars should now be available for you to use in the nifi UI as processors

In a Helm deployment <genprocessor_host> would be the IP address (and port, if it's not the default port 80) for a Kubernetes node.

3.    Design & Distribution Flow

To start creating flows, we need to create a process group first. The name of the process group will be the name of the flow.

a. Configure Nifi Registry url

Next check Nifi settings by selecting the Hamburger button in the Nifi UI. It should lead you to the Nifi Settings screen

Add a registry client. The Url will be as shown for the current docker compose files. This url is on the internal docker network. This might change when not using docker compose.

See above for the correct URL to use in a Helm environment.

Now enter the process group and drag and drop components that we need into the Process group.

If the Nifi registry linking worked, you should see the “Import” button when you try to add a Processor or Process group to the Nifi canvas, like so-

By clicking on the import button, we can import already created saved and version controlled flows from the Nifi registry, if they are present.

We can save created flows by version controlling them like so starting with a 'right click' anywhere on the canvas-

Ideally you would name the flow and process group the same, because functionally they are similar.

When the flow is checked in, the bar at the bottom shows a green checkmark

Note: Even if you move a component around on the canvas, and its position on the canvas changes, it is recognized as a change, and it will have to recommitted.

b. Adding components and building the flow

You can add additional components in your flow and connect them.

ModelB1Etl connects to ModelB1.

Along the way you need to also provide topic names in the settings section. These can be arbitrary names.

ModelB1 connects to ModelB1Reporting

At this point, in this design, we will drag and drop an Input port and call it ‘ves’ which will be a representation for incoming VES collector data. VES data source connects to ModelB1ETL

To recap, see how ves input port connects to ModelB1Etl connects to ModelB1 which connects to ModelB1Reporting. Look at the connection relationships. Currently there is no way to validate these relationships. Notice how it is required to name the topics by going to Settings.

The complete flow after joining our components looks like so

c) Add distribution target which will be the runtime api url

Once we have the desired flow checked in, we can go ahead and set the distribution target in the controller settings

See above for the correct URL.

d. Submit/ Distribute the flow:

Once your flow is complete and saved in the Nifi registry, you can choose to submit it for distribution.

If the flow was submitted successfully to the runtime api, you should get a pop up a success message like so -

At this step, the design was packaged and sent to Runtime api.

The runtime is supposed to generate the blueprint out of the packaged design/flow and push it to the DCAE inventory.

e. Checking the components in the DCAE inventory

In a Helm deployment, inventory is not exposed externally.  These steps should probably refer to the DCAE dashboard, which allows viewing blueprints in inventory.

curl -X GET -k https://135.207.216.182:32346/dcae-service-types | jq . | less

If you want to view all versions of the components

curl -X GET -k "https://135.207.216.182:32346/dcae-service-types?onlyLatest=false" | jq . | less

You should see the generated artifact/ blueprint in the DCAE inventory. The name for each component will be appended by the flow name followed by underscore followed by the component’s name.

These blueprints can also be seen in the DCAE Inventory dashboard.