Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Jira Legacy
serverSystem Jira
serverId4733707d-2057-3a0f-ae5e-4fd8aff50176
keyCPS-802

...

CSIT/CCSDK Automation Issues

...

Ticket logged: 

Jira Legacy
serverSystem Jira
serverId4733707d-2057-3a0f-ae5e-4fd8aff50176
keySDNC-1667

Where were we?

Our integration (and manual) testing using SDN-C v.1.8.1 worked fine. At a high level the setup followed these steps

...

  1. Installing pre-existing certs. This caused issues with SDN-C v. 2.2.3 installation, so we removed this step[ (we assume SDN-C now includes its own and/or ODL certs)
  2. Install SDN-C (output includes details on ODL certification installation)

    Code Block
    languageyml
    titleSDNC Certificate Success
    collapsetrue
    100% [========================================================================]
    Karaf started in 44s. Bundle stats: 433 active, 434 total
    Certificate installation in progress. Elapsed time - 60 secs. Waiting for 10 secs before checking the status.. 
    Certificate installation in progress. Elapsed time - 70 secs. Waiting for 10 secs before checking the status.. 
    Certificate installation in progress. Elapsed time - 80 secs. Waiting for 10 secs before checking the status.. 
    Certificate installation in progress. Elapsed time - 90 secs. Waiting for 10 secs before checking the status.. 
    Start cert provisioning. Log file: /opt/opendaylight/current/data/log/installCerts.log
    Certificate installation script completed execution 
    Everything OK in Certificate Installation 


  3. Mount Node
  4. RestConf queries work fine:
    We can also query SDNC to return all nodes using http://localhost:8282/restconf/config/network-topology:network-topology/topology/topology-netconf

    the nodes can also be retrieved using /restconf


  5. /rest based request fail
    http://localhost:8282/rests/data/network-topology:network-topology/topology=topology-netconf/node=DemoNode/yang-ext:mount/turing-machine:turing-machine we receive the following error:

    Code Block
    languageyml
    titlePostman Response
    linenumberstrue
    {
        "errors": {
            "error": [
                {
                    "error-tag": "resource-denied-transport",
                    "error-type": "protocol",
                    "error-message": "Mount point does not exist."
                }
            ]
        }
    }


  6. CPS CSIT test fail with same root cause

    Info
    iconfalse
    titleCSIT Error

    09:49:08.028 [http-nio-8080-exec-8] ERROR o.o.c.n.d.e.DmiExceptionHandler - Exception occurred
    org.springframework.web.client.HttpServerErrorException$ServiceUnavailable: 503 Service Unavailable: [{"errors":{"error":[{"error-tag":"resource-denied-transport","error-type":"protocol","error-message":"Mount point does not exist."}]}}]


...

Perhaps there is a change in the way /rests behaves that we are unfamiliar with or perhaps our configuration is incorrect. To sum up: we can successfully start SDNC, mount a node, query nodes using /restconf but all /rests calls seem to fail. This could be an issue with certs or TLS.

Open Questions

#Question/IssueNotes/Decision
1Are we to generate certs for SDNC ourselves or can we rely on the certs used as part of SDNC itself?
2

Do we have some incorrect config in our docker-compose file? 

Code Blocklanguageymltitle4
 As mentioned on https://docs.onap.org/projects/onap-sdnc-oam/en/istanbul/cert_installation.html certs folder is required as part of installing SDNC through docker-compose
2

Do we have some incorrect config in our docker-compose file? 

Code Block
languageyml
titleCPS SDNC docker-compose.yml
linenumberstrue
collapsetrue
# ============LICENSE_START=======================================================
# Copyright (C) 2022 Nordix Foundation
# ================================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============LICENSE_END=========================================================

version: '3'

services:
  mariadb:
    image: mariadb:10.5
    container_name: sdnc_db_container
    ports:
      - "3306:3306"
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD:-password}
      - MYSQL_ROOT_HOST=%
      - MYSQL_USER=${MYSQL_USER:-sdnc}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD:-password}
      - MYSQL_DATABASE=${MYSQL_DATABASE:-sdncdb}
    logging:
      driver: "json-file"
      options:
        max-size: "30m"
        max-file: "5"

  ansible:
    image: onap/sdnc-ansible-server-image:2.2.2
    depends_on :
      - mariadb
    container_name: sdnc_ansible_container
    entrypoint: ["/opt/ansible-server/startAnsibleServer.sh"]
    ports:
      - "8000"
    links:
      - mariadb:dbhost
      - mariadb:sdnctldb01
      - mariadb:sdnctldb02
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD:-password}
      - MYSQL_USER=${MYSQL_USER:-sdnc}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD:-password}
      - MYSQL_DATABASE=${MYSQL_DATABASE:-sdncdb}
      - ANSIBLE_TRUSTSTORE_PASSWORD=${ANSIBLE_TRUSTSTORE_PASSWORD:-changeit}
    logging:
      driver:   "json-file"
      options:
        max-size: "30m"
        max-file: "5"

  sdnc:
    image: onap/sdnc-image:${VERSION:-2.2.3}
    depends_on :
      - mariadb
      - ansible
    container_name: sdnc_controller
    entrypoint: ["/opt/onap/sdnc/bin/startODL.sh"]
    ports:
      - "8282:8181"
    links:
      - mariadb:dbhost
      - mariadb:sdnctldb01
      - mariadb:sdnctldb02
      - ansible:ansiblehost
    environment:
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD:-password}
      - MYSQL_USER=${MYSQL_USER}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD-password}
      - MYSQL_DATABASE=${MYSQL_DATABASE:-sdncdb}
      - SDNC_CONFIG_DIR=/opt/onap/sdnc/data/properties
      - SDNC_BIN=/opt/onap/sdnc/bin
      - ODL_CERT_DIR=/tmp
      - ODL_ADMIN_USERNAME=${ODL_USER:-admin}
      - ODL_ADMIN_PASSWORD=${ODL_PASSWORD:-Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U}
      - ODL_USER=${ODL_USER:-admin}
      - ODL_PASSWORD=${ODL_PASSWORD:-Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U}
      - SDNC_DB_INIT=true
      - HONEYCOMB_USER=${HONEYCOMB_USER:-admin}
      - HONEYCOMB_PASSWORD=${HONEYCOMB_PASSWORD:-admin}
      - TRUSTSTORE_PASSWORD=${TRUSTSTORE_PASSWORD:-changeit}
      - KEYSTORE_PASSWORD=${KEYSTORE_PASSWORD:-adminadmin}
      - SO_USER=${SO_USER:-sdncaBpmn}
      - SO_PASSWORD=${SO_PASSWORD:-password1$$}
      - NENG_USER=${NENG_USER:-ccsdkapps}
      - NENG_PASSWORD=${NENG_PASSWORD:-ccsdkapps}
      - CDS_USER=${CDS_USER:-ccsdkapps}
      - CDS_PASSWORD=${CDS_PASSWORD:-ccsdkapps}
      - ANSIBLE_USER=${ANSIBLE_USER:-sdnc}
      - ANSIBLE_PASSWORD=${ANSIBLE_PASSWORD:-sdnc}
      - SQL_CRYPTKEY=${SQL_CRYPTKEY:-fakECryptKey}
      - A1_TRUSTSTORE_PASSWORD=a1adapter
    dns:
      - ${DNS_IP_ADDR-10.0.100.1}
    logging:
      driver:   "json-file"
      options:
        max-size: "30m"
        max-file: "5"
    extra_hosts:
      aaf.osaaf.org: 10.12.6.214
3

Using the docker-compose file wherein the certs folder is specified through volume mount, we are able to mount and access nodes directly through SDNC but fails through NCMP


Need mount to specific files, see https://gerrit.onap.org/r/c/cps/+/126945/14..15/csit/plans/cps/sdnc/docker-compose.yml


Original guide used for sdnc docker-compose can be found here: Istanbul - Run.

3

CPS has certs within our repo which were generated for previous versions of SDNC. If we mount the volume as such:

volumes:
- $SDNC_CERT_PATH:/opt/opendaylight/current/certs

where SDNC_CERT_PATH is the absolute path of the certs within the cps repo, we get the following error in SDNC cert logs:

18:23:42 2022-02-07 18:09:57,310 - root - ERROR - Error while extracting zip file(s). Exiting Certificate Installation.
18:23:42 2022-02-07 18:09:57,310 - root - INFO - Error details : [Errno 13] Permission denied: '/opt/opendaylight/current/certs/keys0'
18:23:42 Stoppping SDNR container due to failure in installing Certificates 

This is how we installed and used certs for SDNC 1.8.1 so has the process of accessing the certs changed?

This was resolved by adding separate volume mounts for the files contained with the certs folder.


 Old:

Code Block
languageyml
volumes:
     - $SDNC_CERT_PATH:/opt/opendaylight/current/certs


New:

Code Block
languageyml
volumes:
      - ./certs/certs.properties:/opt/opendaylight/certs/certs.properties
      - ./certs/keys0.zip:/opt/opendaylight/certs/keys0.zip