Versions Compared

Old Version 2

changes.mady.by.user Carsten Lund

Saved on

compared with

New Version Current

changes.mady.by.user Deven

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Note these instruction refers to current master branch. The release-1.0.0 branch have development issues that have only been fixed in master branch.

And this does NOT refer to DCAEGEN2 which have a completely new setup/process and is getting released in the R1/Amsterdam Release.

Refer to DCAE MOD User Guide for latest

DCAE Controller Overview

Make sure your settings.xml includes the onap public repo for some snapshot jar downloads during the build.

DCAE Controller LF GIT repositories

  1. dcae/controller
  2. dcae/operation/utils
  3. dcae/controller/analytics
  4. dcae/demo

SOMF - Sirius Operational Management Framework

...

  1. ncomp/maven - contains a few maven related artifacts
  2. ncomp/utils - contains Java utilities and libraries that are used across DCAE Controller projects
  3. ncomp/core - contains core models
  4. ncomp/sirius/manager - contains the core SOMF implementation
  5. ncomp/docker - contains SOMF adaptor used to communicate with Docker Engine API
  6. ncomp/openstack- contains SOMF adaptor used to communicate with OpenStack API
  7. ncomp/cdap- contains SOMF adaptor used to communicate with CDAP API+

Development Setup

GIT repositories and build order

  1. ncomp/maven
  2. ncomp/utils
  3. dcae/operation/utils
  4. ncomp/core
  5. ncomp/sirius/manager
  6. ncomp/cdap
  7. ncomp/docker
  8. ncomp/openstack
  9. dcae/controller
  10. dcae/controller/analytics
  11. dcae/demo

Eclipse Setup

Since DCAE Controller/SOMF is build on top of EMF developement development is tightly tied to using Eclipse IDE with specific plugin installed. This section describe the process to setup a developement development Eclipse.

  1. Install Neon.3 Java Developer Version http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/neon3
  2. Install software from Neon update site
    1. Eclipse Plug-in Development Environment           3.12.2.v20161124-1400
    2. EMF - Eclipse Modeling Framework Xcore SDK   1.4.0.v20160526-0606
  3. Install software from Eclipse Market Place
    yEdit
    1. YEdit Feature     1.0.20.201509041456-RELEASE
  4. Install Groovy from http://dist.springsource.org/snapshot/GRECLIPSE/e4.6/

      ...

        1. Groovy-Eclipse Feature             2.9.2.xx-201703131833-e46

      ...

        1. Groovy-Eclipse M2E integration            2.9.2.xx-201703131833-e46

      ...

        1. Groovy Compiler 2.4 Feature  2.9.2.xx-201703131833-e46

      Import Eclipse Projects

      The GIT repositories include the Eclipse Project Metadata (e.g., .project, .classpath, and etc files) and thus can be importing into a workspace using the "Import Projects" → "Import existing Eclipse projects". Only the projects that need to be worked on need to be in the workspace.

      Working on DCAE Controller projects

      There are 2 projects for most features.

      1. A model project which contains the data model  using XCORE file. These projects always have a name (and artifactId) that ends in -model.This file is located on the src/main/xcore directory. And the project is setup so Eclipse will update automatically the generated Java code in the src/main/xcore-gen directory. This project is a straight EMF type project and contain very little SOMF specifics.
      2. A SOMF project which contains SOMF generated code and implementation of the APIs that the model defines.
        1. SOMF generator class Generator.java under the src/main/java folder. It contains the defined customization for creating the SOMF implementation. Running this class will generate new contents in the two generated folders.
        2. src/main/sirius-gen folder contains generated Java code for implementing Client and Server side functions.
        3. src/main/server-gen folder contains generated Bash and Groovy scripts for implementing Client and Server side functions.
        4. SOMF provider *Provider.java Java classes under the src/main/java folder. These implements the operation/methods that are defined in the model.

      DCAE Configuration Setup

      The DCAE configuration setup is trying to carefully handle Environmental and Non-Environmental configuration.

      The ONAP Demo setup is creating another level of complexity to the setup since the overall ONAP Heat templates will deliver environmental information that need to be include in the Environmental configuration. 

      This is the various configuration entities.

      1. DCAE Non Environmental ONAP Demo configuration. https://gerrit.onap.org/r/gitweb?p=dcae/demo.git;a=tree;f=OPENECOMP-DEMO;h=38b2f51a1be5c12f357e228b7a655c0ee97606b8;hb=HEAD with the main files
        1. location-types.yaml define the various DCAE entities (e.g., docker-host VMs, cdap cluster VMs, VES collector etc) that are part of the demo deployment
        2. vm-templates/vm-docker-host.yaml, vm-templates/vm-cdap-cluster.yaml, vm-templates/vm-postgresql.yaml . Define the deployment configuration for various VM deployments.
        3. docker-templates/docker-XX.yaml. Similar for docker deployments.
        4. cdap-templates/cdap-YY.yaml. Similar for docker deployments.
        5. steams.yaml. Define DCAE DMaaP setup. 

        6. HEAT provided information. When the ONAP demo is getting deploy the vm1-dcae-controller VM which will come up with the DCAE controller Docker container. This container will use configuration attributes setup in the file /opt/app/dcae-controller/config.yaml to apply to DCAE controller environment file, which will have expressions like @{XXX} replaced with the value of XXX in the config.yaml file.

          Variable NameSample ValuesNotes
          BASERACKSPACE,2-NIC,1-NIC-FLOATING-IPSRACKSPACE will only work in Rackspace, 2-NIC can work in both Rackspace and non Rackspace environment, and 1-NIC-FLOATING-IPS does not work in Rackspace.
          DCAE-VERSION1.1.0Note only 1.1.0 version is working outside of Rackspace
          OPENSTACK-AUTH-METHODpassword, api-keyNeed to use password for non-Rackspace environments
          DOCKER-VERSION1.1-STAGING-latest


      2. DCAE Environmental ONAP Demo configuration. Currently 3 deployment scenarios are supported (See
        Jira Legacy
        serverSystem Jira
        columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
        serverId4733707d-2057-3a0f-ae5e-4fd8aff50176
        keyDCAE-7
        for details about current status.)
        1. RACKSPACE. This scenario only work when the Cloud provider is Rackspace. In this each DCAE VM will be setup with a predefined fixed IP on a private network and it will get a random public IP on the public network. This scenario depends on the following values from the HEAT provided environment file: DCAE-VERSION, DOCKER-REGISTRY, DOCKER-VERSION, GIT-MR-REPO, HORIZON-URL, KEYSTONE-URL, NEXUS-PASSWORD, NEXUS-RAWURL, NEXUS-USER, OPENSTACK-KEYNAME, OPENSTACK-PASSWORD, OPENSTACK-PRIVATE-NETWORK, OPENSTACK-PUBKEY, OPENSTACK-REGION, OPENSTACK-TENANT-ID, OPENSTACK-TENANT-NAME, OPENSTACK-USER, POLICY-IP, STATE, ZONE.
        2. 2-NIC. This scenario works in most OpenStack environments and allows higher level of flexibility in assigning IPs etc. Similar to the RACKSPACE setup each DCAE VM will be setup with a predefined fixed IP on a private network and it will get a random public IP on the public network.This scenario depends on the following values from the HEAT provided environment file: BASE=2-NIC, DCAE-VERSION, DNS-IP-ADDR, DOCKER-REGISTRY, DOCKER-VERSION, FLAVOR-LARGE, GIT-MR-REPO, HORIZON-URL, KEYSTONE-URL, NEXUS-PASSWORD, NEXUS-RAWURL, NEXUS-USER, OPENSTACK-AUTH-METHOD, OPENSTACK-KEYNAME, OPENSTACK-PASSWORD, OPENSTACK-PRIVATE-NETWORK, OPENSTACK-AUTH-METHOD, OPENSTACK-PUBKEY, OPENSTACK-REGION, OPENSTACK-TENANT-ID, OPENSTACK-TENANT-NAME, OPENSTACK-USER, POLICY-IP, STATE, UBUNTU-1404-IMAGE, UBUNTU-1604-IMAGE, ZONE, dcae_cdap00_ip_addr, dcae_cdap01_ip_addr, dcae_cdap02_ip_addr, dcae_coll00_ip_addr, dcae_ip_addr, dcae_pstg00_ip_addr, public_net_id. The HEAT template demo/heat/OpenECOMP/onap_openstack_nofloat.yaml provides these variables.
        3. 1-NIC-FLOATING-IPS. This scenario works in most OpenStack environments and allows higher level of flexibility in assigning IPs etc. In this setup each DCAE VM will only have one NIC with an IP on the private network. In addition each VM will have a floating IP from the public network. In this case it is the floating VMs that can get predefined values and the IPs from the private network that are assigned to the NIC are randomly setup. This makes a few issues with the rest of the demo setup but should be working in the 1.1 release. This scenario depends on the following values from the HEAT provided environment file:BSAE=1-NIC-FLOATING-IPS, DCAE-VERSION, DNS-IP-ADDR, DOCKER-REGISTRY, DOCKER-VERSION, FLAVOR-LARGE, GIT-MR-REPO, HORIZON-URL, KEYSTONE-URL, NEXUS-PASSWORD, NEXUS-RAWURL, NEXUS-USER, OPENSTACK-AUTH-METHOD, OPENSTACK-KEYNAME, OPENSTACK-PASSWORD, OPENSTACK-PRIVATE-NETWORK, OPENSTACK-PUBKEY, OPENSTACK-REGION, OPENSTACK-TENANT-ID, OPENSTACK-TENANT-NAME, OPENSTACK-USER, POLICY-IP, STATE, UBUNTU-1404-IMAGE, UBUNTU-1604-IMAGE, ZONE, dcae_cdap00_float_ip_addr, dcae_cdap01_float_ip_addr, dcae_cdap02_float_ip_addr, dcae_coll00_float_ip_addr, dcae_float_ip_addr, dcae_pstg00_float_ip_addr. The HEAT template demo/heat/OpenECOMP/onap_openstack_float.yaml provides these variables. One requirement for this setup is that the floating IPs that is getting used are already associated with the Openstack Tenant used.

      The overall deployment flow is the following.

      1. HEAT template
        1. Create the vm1-dcae-controller VM.
        2. Setup  /opt/app/dcae-controller/config.yaml
        3. Start the Docker Container for the DCAE Controller
      2. The DCAE controller will (running /opt/app/dcae-controller-platform-server/bin/controller-startup.sh)
        1. Determine the BASE based on the BASE attribute in config.yaml
        2. Determine the ZONE based on the ZONE attribute in config.yaml
        3. Substitute the HEAT delivered attributes from config.yaml into /opt/app/dcae-controller-platform-server/OPENECOMP-DEMO-$BASE and produce a /opt/app/dcae-controller-platform-server//opt/app/dcae-controller-platform-server/OPENECOMP-DEMO-$ZONE.
          1. bin/dcae-controller.sh rackspace-substitute --from OPENECOMP-DEMO-$BASE --to OPENECOMP-DEMO-$ZONE --file /opt/app/dcae-controller/config.yaml
        4. Merge the Non-environmental configuration OPENECOMP-DEMO with the environmental configuration OPENECOMP-DEMO-$ZONE to create the complete configuration setup for this specific environment which will be placed in GITLINK/OPENECOMP-DEMO-$ZONE
          1. java -cp 'lib/*' org.openecomp.dcae.controller.operation.utils.GenControllerConfiguration $ZONE . GITLINK OPENECOMP-DEMO
        5. The DCAE Controller will be started up and synced with the configuration
          1. bin/dcae-controller.sh start

          2. bin/dcae-controller.sh sync-configuration --environment OPENECOMP-DEMO-$ZONE

        6. The DCAE Controller can then Deploy the various DCAE components define in the demo environment

          1. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s vm-docker-host-1
          2. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s vm-postgresql
          3. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s vm-cdap-cluster
          4. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s docker-databus-controller
          5. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s cdap-helloworld
          6. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s cdap-tca-hi-lo
          7. bin/dcae-controller.sh deploy-service-instance -i $ZONE -s docker-common-event

      Common Deployment Issues Seen

      {"auth":{"RAX-KSKEY:apiKeyCredentials":{"username":"<some-user-here>","apiKey":"<some-key-here>"}}}

      Using Rackspace authentication in non Rackspace. Make sure your are using 1.1 DCAE controller container and that OPENSTACK-AUTH-METHOD = 'password', DOCKER-VERSION =1.1-STAGING-latest and DCAE-VERSION = 1.1.0

      java.lang.IllegalArgumentException: !Absolute URI: null/servers

      Connectivity to OpenStack APIs is not working look in logs/*.err for the error returns by the keystone API call.

      Reporting Issues

      Due to the large number of possible issues in a deployment please run the following script and include the output in the Ticket.


      #!/bin/bash
      PW=$(grep OPENSTACK-PASSWORD /opt/app/dcae-controller/config.yaml | sed s/OPENSTACK-PASSWORD:.//)
      set -e
      echo ======= config.yaml
      cat /opt/app/dcae-controller/config.yaml | sed "s/$PW/XXXXXX/"
      echo ======= docker
      docker images
      docker ps -a
      ID=$(docker ps | grep dcae-controller: | cut -c1-12)
      echo ======= docker logs
      docker logs $ID 2>&1
      echo ======= dcae-controller.sh.log
      docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/dcae-controller.sh.log
      echo ======= reports
      docker exec -e GROOVY_HOME=/opt/app/groovy-2.4.6 $ID /opt/app/dcae-controller-platform-server/bin/dcae-controller.sh report -n /reports/dcae/vms
      docker exec -e GROOVY_HOME=/opt/app/groovy-2.4.6 $ID /opt/app/dcae-controller-platform-server/bin/dcae-controller.sh report -n /reports/dcae/service-instances
      echo ======= logs err
      docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/controller-platform-server-controller.err | head -10000 | sed "s/$PW/XXXXXX/"
      echo ======= logs out
      docker exec $ID cat /opt/app/dcae-controller-platform-server/logs/controller-platform-server-controller.out | head -10000