Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 160 Next »

Follow the steps below to set up the development environment on your client machine, and to establish credentials with which you may access the repositories.

Create a Linux Foundation Account

Follow the instructions on the identity portal to create a Linux Foundation account and get access to the ONAP Gerrit instance.  Then verify that you can log in at https://gerrit.onap.org/ and that you can see the ONAP list of repositories.

Mac/OSX

get your MBP 2016 ready for development

generate keys - either ssh-add the key or rename it as id_rsa and put it into your ~/.ssh dir

Install the JDK (as dmg) - http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Install homebrew package manager - 

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

python should already be installed. - do a --version check

install pip - sudo easy_install pip

install openstack tools

install docker (dmg) - https://docs.docker.com/docker-for-mac/install/#download-docker-for-mac

Unix Setup

Redhat/CentOS

20190130 - re-validating for RHEL 7.6 subscription



install java 

set JAVA_HOME in ~/.bashrc

yum install java-1.8.0-openjdk-devel

get it via alternatives --config javac

currently

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-3.b12.el7_3.x86_64

install git and development tools
yum install git
yum groupinstall 'Development Tools'

this will give you make and python

install npm
(will also be installed automatically by the ecomp build) - TODO: UPDATE as ecomp is locked

download and install Maven 3.6.3

create an ~/.m2 folder and copy our settings.xml there

add maven to your path - check it with mvn --version

http://maven.apache.org/download.cgi

tar -xvf apache-maven-3.6.3-bin.tar.gz

Add the following option to your MAVEN_OPTS in order not to periodically hang on downloading artifacts in linux

-Djava.net.preferIPv4Stack=true

enable IP4 forwarding to enable the RHEL VM to act as an edge router - without this docker containers cannot communicate

add to /etc/sysctl.conf
net.ipv4.ip_forward = 1

Install docker (required for some repos like dcae-inventory off dcae
sudo yum install -y yum-utils
sudo yum-config-manager     --add-repo     https://download.docker.com/linux/centos/docker-ce.repo
sudo yum-config-manager --enable docker-ce-edge
sudo yum makecache fast
sudo yum install docker-ce
sudo systemctl start docker
20190130


sudo curl https://releases.rancher.com/install-docker/17.03.sh | sh

sudo wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 -O jq
sudo chmod 777 jq
# not +x or jq will not be runnable from your non-root user
sudo mv jq /usr/local/bin
jq --version

sudo usermod -aG docker ubuntu

Ubuntu 16.04



enable ubuntu desktop

apt-get update

apt-get install ubuntu-desktop

enable static IP

vi /etc/network/interfaces

add

iface enp0s25 inet static

address 192.168.15.101

netmask 255.255.255.0

network 192.168.15.0

broadcast 192.168.15.255

gateway 192.168.15.1

dns-nameservers 8.8.4.4

disable strict host checking (client and server)
in ~/.ssh/config
Host *
   StrictHostKeyChecking no
   UserKnownHostsFile=/dev/null

Enable non CD repositories so an apt-get update will work

comment out the CD first line in /etc/apt/sources.list
enable ssh if skipped during installation so we can remote sshsudo apt-get install openssh-server
enable root login and gui (no sudo su -)
sudo passwd root

sudo usermod -U root

sudo reboot now

sudo vi /etc/lightdm/lightdm.conf

[SeatDefaults]

greeter-session=unity-greeter

user-session=ubuntu

greeter-show-manual-login=true


: enable root login

vi /etc/ssh/sshd_config
FROM:
PermitRootLogin prohibit-password
TO:
PermitRootLogin yes

systemctl restart sshd

The 131 version of Java 8 is installed by default - currently running 151

set JAVA_HOME in ~/.bashrc or /etc/environment

ubuntu@ubuntu:~$ echo $JAVA_HOME
/usr/lib/jvm/java-1.8.0-openjdk-amd64

if not

sudo apt install openjdk-8-jdk

install git and development tools
sudo apt-get install git
install npm
(will also be installed automatically by the ecomp build)
sudo apt install npm

download and install Maven 3.5.0

create an ~/.m2 folder and copy our settings.xml there

add maven to your path - check it with mvn --version

http://maven.apache.org/download.cgi

tar -xvf apache-maven-3.5.0-bin.tar.gz

or

sudo apt install maven

Add the following option to your MAVEN_OPTS in order not to periodically hang on downloading artifacts in linux

in /etc/environment

-Djava.net.preferIPv4Stack=true

Install docker (required when building docker images)


use the current as of 20180621 17.03 version in the script
which is


curl https://releases.rancher.com/install-docker/17.03.sh | sh


Build ONAP

root@obriendev:/home/ubuntu/onap# mvn clean install -U -DskipTests=true -Dmaven.test.skip=true -Dmaven.javadoc.skip=true

-Dadditionalparam=-Xdoclint:none


Maven Projects

If you have imported an existing maven project into eclipse - you may run into eclipse-only build issues (both runtime when deploying the war to a local tomcat and at design time resolving spring jars) because the maven dependencies libarary is missing from project | Java Build Path | Maven Dependencies

Specific to WAR projects - insert the following in your .classpath file off the project

	<classpathentry exported="true" kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER">
		<attributes>
			<attribute name="maven.pomderived" value="true"/>
			<attribute name="org.eclipse.jst.component.nondependency" value=""/>
			<attribute name="org.eclipse.jst.component.dependency" value="/WEB-INF/lib"/>
		</attributes>
	</classpathentry>



Install git-review

<<TODO: describe a Linux client set-up for Git and git-review>>

To install git-review on Linux (Ubuntu), type the following command.

sudo apt-get install git-review

Windows Setup

Note: The DCAE and SDC  repo will not fully clone in a windows environment (even git-bash) until some file paths are reduced below 255 chars -  DCAE-24 - Getting issue details... STATUS  and  SDC-21 - Getting issue details... STATUS  - you may use Linux or OSX native or VMs as an alternative for now.

Enabling long paths in windows - thanks Michael Lando

git config --system core.longpaths true

Windows Subsystem for Linux

Just install the Windows Subsystem for Linux https://docs.microsoft.com/en-us/windows/wsl/install-win10  and the Ubuntu 16 tools https://www.microsoft.com/en-ca/p/ubuntu/9nblggh4msv6?rtc=1&activetab=pivot:overviewtab and skip git-bash, putty and cygwin.


Installing Git

To install Git on your client, visit: https://www.atlassian.com/git/tutorials/install-git, and download the appropriate installation image for your host. Once downloaded, execute the installation script ("open" the .msi file).

During installation, you will see the following dialog box.  Include the "Git Bash" and "Git GUI" components (this is the default).To do this, leave the following boxes checked:

If you intend to type Git commands from the Windows Command Prompt, leave the default setting for adjusting the PATH environment:

The repositories use Unix-style line-feed for line termination, so on your windows Git client, select the conversion option:

Select the following option based on your own preference (non-default shown, so 'winpty' is not required):

Choose the defaults here:

The installer lets you choose the experimental 'difftool'; either choice will work. Continue with the installation dialogs until the installation is complete.

Install Python on Windows (optional) - required to get pip for git-review

Not all components require Python for development. See the documentation for your component in the Development Guides section. (Note: the "node-gyp" tool requires Python.)

Download Python version 2.7.10:

Execute the install file that you just downloaded. (You must have administrative privileges.) During installation, select "Add python.exe to Path", overriding the default setting. Also, be sure to install "pip" (this is the default).

Continue with the installation until it is completed.

Set up Git review and Configure Git

Run Git Bash as Administrator (for example, in Windows 10, use the Windows menu, scroll to the Git directory, right-click on Git-bash, and select "Run as administrator":

Within Git Bash, type the following command: assumes you already installed python above

$ pip install git-review

To use git review, you have to be in a Git clone directory that already contains a (possibly hidden) .gitreview configuration file (see Gerrit/Advanced usage#Setting up a repository for git-remote), otherwise you will get the error message UnboundLocalError: local variable 'no_git_dir' referenced before assignment.

The Git and git-review installation steps above are derived from the description at: https://www.mediawiki.org/wiki/Gerrit/git-review#Windows 

Configure Git to remember your Linux Foundation user name and email address (the user name and email address associated with your Linux Foundation login):

git config --global user.email <your_LF_account_email>

git config --global --add gitreview.username <your_LF_user_name>

Configure git review

git config --global gitreview.remote origin


If you are using VPN, you might encounter a proxy problem while connecting to the Linux Foundation website. To avoid the problem, you should add the proxy setting in git config, using the following command:

git config --global https.proxy https://<proxy username>:<proxy password>@<proxy url> 

git config --global http.proxy http://<proxy username>:<proxy password>@<proxy url>

NOTE: When entering the proxy username, you might be required to add the domain name in front of the username.

Install Putty and Generate Keys

An alternate method for creating public and private keys, without using PuTTY, is given here: https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/

If you already have keys registered with PuTTY, skip this section.

PuTTY is a terminal emulator that uses Secure SHell (SSH) to communicate with a host. You need to install PuTTY in order to create a public/private encryption key pair. This pair of keys enables Git to communicate securely with the repositories.

To install PuTTY, visit, www.putty.org and follow the links to the appropriate Windows installation for your machine (32-bit or 64 bit). The downloaded file will be named putty-0.67-installer.msi or the like.

Execute the .msi file. Use all the default settings, thereby ensuring that you will install pageant.exe, plink.exe, and puttygen.exe.

Generate Public and Private Keys

OSX

ssh-keygen -t rsa


Windows

The puttygen utility, installed with putty, generates public and private keys used for secure communication with Gerrit.

You should execute puttygen.exe as an Administrator, otherwise, the key files will be stored in %localappdata%\VirtualStore\PuTTY, typically C:\Users\<user>\AppData\Local\VirtualStore\Program Files (x86)\PuTTY.

To do this, right-click on C:\Program Files (x86)\PuTTY\puttygen.exe and select Run as Administrator:

Answer "Yes" to the User Account Control challenge. Then click the "Generate" button in the puttygen window. puttygen will ask you to move your mouse over its dialog box to generate random seed bits. After it has computed a public key, it will display a dialog box resembling the following:

If you want to edit this key pair in the future, create a text file containing a short phrase, called a "passphrase". Copy and paste this passphrase into the "Key passphrase" and "Confirm passphrase" lines, then select "Save public key" and choose a filename (e.g. public.txt); also "Save private key" and choose another filename (e.g. private.ppk). Since you ran puttygen as Administrator, the files are visible to Windows Explorer.

Close puttygen.

Load Your Private Key Into Pageant

Pageant is an SSH authentication agent. It holds your private keys in memory, already decoded, so that you can use them often without needing to type a passphrase.

To load your private key into pageant, right-click on the private key file (e.g. private.ppk in this example), and select Load into pageant:

This action not only loads private.ppk into pageant, but also starts pageant if it is not already running. You will be prompted for the passphrase that you entered into puttygen when you created the key. If successful, pageant will load the key without further interaction.

Pageant: a background job with a dialog box

pageant runs without opening any visible window. When it is running, it puts an icon of a computer wearing a hat () into the System tray.

You can right-click on this icon to perform actions with pageant, including "View Keys" (which brings up a dialog box that allows you to add or remove keys.)

The "Add Key" button offers another way to enter your private key into pegeant: after clicking on "Add Key", select the file containing your private key (for example, private.ppk).

Set the GIT_SSH Environment Variable

From the Desktop, right-click the Windows logo at the very bottom left corner of the screen,  , and select "System". Within the System dialog box, select "Advanced system settings" to get the Power User Task Menu.

In the next dialog box, select "Environment Variables":

If you don't already have a GIT_SSH entry in your System Variables list, click "New..." and create GIT_SSH, defined to be the location of plink.exe :

Copy/paste the known_hosts entries listed, into a file named known_hosts under your .ssh profile (something like C:\Users\<my-id>\.ssh) (if known_hosts does not exist, create it).

Install NodeJS

The dcae build will install npm in most cases - however if you run into issues like the following - install it yourself

Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:npm (npm install) on project ecompportal-FE-os: Failed to run task: 'npm install' failed. (error code -4048)


Install NodeJs from https://nodejs.org/en/

verify your install by doing 

$ npm -version
3.10.10

From git bash

  • npm install bower -g
  • npm install grunt-cli -g

IDEs

IDE: Eclipse: Does Not Support Gerrit Plugin

Eclipse plugins (including the Mylin Gerrit connector plugin and Egit) do not support a 2.12 Gerrit server, so if you use Eclipse as your development IDE, you must do reviews using the Gerrit web interface.

IDE: Eclipse: Setting up Eclipse

Install

Download and run the installer from: Install Eclipse. Select "Eclipse IDE for Java Developers" to install.

ONAP Eclipse Java Formatter

As per https://gerrit.onap.org/r/#/c/79312/1/onap-java-formatter.xml, an ONAP Eclipse Java Formatter XML file has been merged and can be imported into Eclipse. Updates to the settings should be committed and merged back into the repository (e.g. AAI-2198 - Getting issue details... STATUS ) . TBC: This should probably be in oparent for all to share.

Maven Plugins to Reformat Java code

Using the Eclipse Formatter file above and maven plugin configuration in pom.xml as per https://gerrit.onap.org/r/#/c/79312/1/pom.xml (https://code.revelc.net/formatter-maven-plugin/ for Eclipse formatter and https://github.com/diffplug/spotless/tree/master/plugin-maven for import order)

Example pom.xml configuration
            <plugins>
                <!--
                Using https://code.revelc.net/formatter-maven-plugin/ for Eclipse formatter
                Using https://github.com/diffplug/spotless/tree/master/plugin-maven for import order
                Use in combination to rewrite code and imports, then checkstyle
                
                mvn formatter:format spotless:apply process-sources
                -->
                <plugin>
                     <groupId>net.revelc.code.formatter</groupId>
                     <artifactId>formatter-maven-plugin</artifactId>
                     <version>2.8.1</version>
                     <configuration>
                        <configFile>${project.parent.basedir}/onap-java-formatter.xml</configFile>
                     </configuration>
                     <!-- https://code.revelc.net/formatter-maven-plugin/
                          use mvn formatter:format to rewrite source files
                          use mvn formatter:validate to validate source files -->
                 </plugin>
                <plugin>
                  <groupId>com.diffplug.spotless</groupId>
                  <artifactId>spotless-maven-plugin</artifactId>
                  <version>1.18.0</version>
                  <configuration>
                    <java>
                     <importOrder>
                       <order>com,java,javax,org</order>
                     </importOrder>
                    </java>
                  </configuration>
                <!-- https://github.com/diffplug/spotless/tree/master/plugin-maven
                     use mvn spotless:apply to rewrite source files
                     use mvn spotless:check to validate source files -->
                </plugin>
            </plugins>


the combination can be used in a maven command to rewrite code and imports, then checkstyle audit like so

Example maven command
mvn formatter:format spotless:apply process-sources

Import Google style preferences

If the above ONAP Eclipse formatter is not used, follow instructions on: Installing the coding style settings in eclipse.

See also Java code style

Modify settings to adapt to ONAP

  1. Select "Window -> Preferences".
  2. In the filter text box, type "tab".
  3. Under "Editors -> Text Editors", set "Displayed tab width:" to 4.
  4. Under "Editors -> Text Editors", activate "Insert spaces for tabs".
  5. If you want a visual aid for the print margin, under "Editors -> Text Editors", activate "Show print margin" and set "Print margin column:" to 120.
  6. Under "XML -> XML Files -> Editor", change to "Indent using spaces".
  7. Clear the filter text box and type"format".
  8. Under "Java -> Code Style Formatter", for "Active profile:" select GoogleStyle.
  9. Press the "Edit..." button.
  10. Expand the "Indentation" section.
  11. Select "Spaces only" for "Tab policy", and "4" for "Indentation size".
  12. Expand the "Line Wrapping" section.
  13. Set "Maximum line width:" to "120", and press "Apply".
  14. Expand the "Comments" section.
  15. Set "Maximum width for comments:" to "120"
  16. Unset "Count width from comment's starting position"
  17. Unset "Enable header comment formatting", and press "Apply" and then "OK"
  18. Clear the filter text box and type "save actions".
  19. Under "Java -> Editor -> Save Actions", check the "Perform the selected actions on save", and "Additional actions" check boxes.
  20. Press the "Configure..." button.
  21. On the "Code Organizing" tab in the "Formatter" box, check the "Remove trailing whitespace" check box.
  22. Press "OK".
  23. Select "Java -> Editor -> Content Assist -> Favorites".
  24. For each of the types below, press "New Type..." and type in the type and press "OK".
    org.junit.Assert
    org.junit.Assume
    org.junit.jupiter.api.Assert
    org.junit.jupiter.api.Assumptions
    org.junit.jupiter.api.DynamicContainer
    org.junit.jupiter.api.DynamicTest
    org.mockito.ArgumentMatchers
    org.mockito.Mock
    org.mockito.Mockito
  25. Select "Java -Code Style -> Organize Imports".
  26. Remove all items from the list.
  27. Press "Apply and Close".

To format a file open in an editor after these style preferences, press "Shift + Ctrl + F".

Install useful plugins

Install EclEmma for code coverage, and SonarLint for static code analysis.

Set up Sonar towards ONAP

To bind your projects to the ONAP Sonar server, follow the instructions below. Your projects should be imported in to Eclipse before this.

  1. Right click on the project and select "SonarLint -> Bind to SonarQube or SonarCloud...".
  2. Select "sonarcloud" and press "Next".
  3. Click "Generate Token"
  4. A browser opens and you are taken to a Sonarcloud login page
  5. Login with an appropriate account from the list presented, most likely your GitHub account
  6. You are now directed to a sonarcloud token generation page
  7. Enter a name for your token and click "Generate"
  8. Copy the token hex string that is generated from the browser and paste it into the "Token" field in Eclipse and click "Next"
  9. In the "Organization" field, enter the string "onap" and press "Next"
  10. The Connection name "SonarCloud/onap" should be found by the system, click "Next"
  11. The connection should be successfully created, click "Finish"
  12. Press "Add...".
  13. Select the projects you want to add and press "OK".
  14. Press "Next".
  15. Start typing the name of your project, and it should appear in a list box where it should be selected.
  16. Press "Finish".

To see messages from Sonar introduced by edits made in the projects, select "Window -> Show Wiew -> Other...". Expand "SonarLint" and select "SonarLint -> On-The-Fly".

Setting up the ONAP Checkstyle in Eclipse

(There may well be an easier way but this works)

The first issue is that the current ONAP master tagged version of the checkstyle does not work with Eclipse Oxygen/Photon (and maybe other versions) because of the "curly bracket" issue. There is a fix on the way but it's not here yet. The current tagged version of oparent we are using in Policy is 1.2.1.

Hack oparent to fix "curly bracket" issue

To get around this issue, check out oparent, checked out the 1.2.1 tag, fixed the "curly bracket" bug and built it on my local machine.

  1. Check out oparent
  2. git tag -l
  3. git co tags/1.2.3 -b 1.2.3
  4. If you want CheckStyle to ignore generated files, do the following:
    1. Add the following tag to the above file:
      <module name="SuppressionFilter">
        <property name="file" value="<absolute path to the directory of the file>/suppressions.xml"/>
      </module>
    2. Create a file called "suppressions.xml" in the folder given above, and put the following content in it:
      <?xml version="1.0"?>
      <!DOCTYPE suppressions PUBLIC
        "-//Checkstyle//DTD SuppressionFilter Configuration 1.2//EN"
        "https://checkstyle.org/dtds/suppressions_1_2.dtd">
      <suppressions>
        <suppress files="[\\/]generated-sources[\\/]" checks="[a-zA-Z0-9]*"/>
      </suppressions>
  5. The versions in the POMs in oparent are snapshot, so we need to change those to 1.2.1
     mvn versions:set -DnewVersion=1.2.1
  6. Now build locally:
    mvn clean install

Set "ONAP" configuration in Eclipse

To set the newly built checkstyle files in Eclipse:

  1. Preferences->Checkstyle
  2. Click "New"
  3. Select "External Configuration File"
  4. Give it a name eg ONAP
  5. Point at the file <your_git_folder>/oparent/checkstyle/src/main/resources/onap-checkstyle/onap-java-style.xml  
    (assuming you have downloaded the oparent repo)
  6. Click OK
  7. Select "ONAP" configuration and click "Set as Default"
  8. Select "Apply and Close"

Apply "ONAP" configuration to a project in Eclipse

Now we need to activate the checkstyle on one project and set it as the blueprint for all of them:
Select a project in eclipse and right click->Properties->Checkstyle
check "Checkstyle active for this project"
Select the "ONAP" checkstyle profile
Click "Apply and Close"

Spread blueprint to other projects in Eclipse

Now spread the profile to all other projects:

  1. Select all the projects you want to apply the profile to in the Eclipse project explorer (not the one that you set up above)
  2. Right click->Checkstyle->Configure projects from blueprint
  3. Select the project you set up above
  4. Now all the projects have the correct checkstyle setup.

Sorry this was so long, how can something so simple be so complex!

IDE: IntelliJ

Installing Intellij Community Edition on Linux 

Download

https://www.jetbrains.com/idea/download/#section=linux

Unpack the ideaIC.gz or ideaIU.gz file.

A new instance must not be extracted over an existing one. The target folder must be empty.

tar xfz ideaIC.tar.gz or ideaIU.tar.gz. <new_archive_folder>

The recommended install location according to the filesystem hierarchy standard (FHS) is /opt. For example, it's possible to enter the following command:

sudo tar xf -*.tar.gz -C /opt/

Go to the bin directory, for example

cd opt/-*/binH

Start Intellij

Run idea.sh from the bin subdirectory.

Refer to https://www.jetbrains.com/help/idea/installing-and-launching.html

Installing IntelliJ Community Edition and the Java SDK on Windows

If you are using IntelliJ to do Java compilations, download the IntelliJ installation image from https://www.jetbrains.com/idea/. The installation dialog gives you the opportunity to associated IntelliJ with file type .java, among others. Installation takes several minutes.

When you start your first IntelliJ project, you will need to specify the Java Software Development Kit (SDK) as the project's. If you do not have the Java SDK installed on your machine, you can download it from within IntelliJ by selecting Maven as the New Project type and clicking on "Download SDK": (Or, visit http://www.oracle.com/technetwork/java/javase/downloads/index.html).

Install useful plugins

Install SonarLint to check for SonarQube violations and CheckStyle-IDEA to apply checkstyle code formatting

Configure ONAP code formatting for IntelliJ

ONAP is using google java style as mentioned in ONAP java style.  Import intellij-java-google-style.xml in intelliJ settings→code style→java and modify the following items,

Tabs and Indents→Indent ( from 2 to 4)

Wrapping and Braces → hard wrap at ( 120)


SO Project: Method above won't work, install plugin Eclipse Code Formatter and configure it with the corresponding xml file (so/project-configs/code-tools/onap-eclipse-format.xml)

Configure ONAP code CheckStyle for IntelliJ

  1. Select, File, Settings, Tools, Checkstyle  
  2. Click on + beside the 'Configuration File' box to add a configuration
  3. Set description to something like 'ONAP Rules'
  4. Click on Browse to select the file <your_git_folder>/oparent/checkstyle/src/main/resources/onap-checkstyle/onap-java-style.xml  
    (assuming you have downloaded the oparent repo)
  5. Complete the Wizard (you can set exclusion properties if needed)
  6. Select the Configuration File you just added by selecting the relevant checkbox
  7. Click [OK] to close the settings popup

Configure ONAP copyright for IntelliJ

You can create copyright profile to auto-generate the copyright text. e.g, add the following text to new profile,

Replace <organization name>
============LICENSE_START=======================================================
 Copyright (C) $today.year <organization name>
 ================================================================================
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
       http://www.apache.org/licenses/LICENSE-2.0
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 
 SPDX-License-Identifier: Apache-2.0
 ============LICENSE_END=========================================================

Set up SonarLint towards ONAP for IntelliJ

PreRequisites

To use SonarLint with Onap projects you need to setup a connection with SonarCloud

  1. Select, File, Settings, Tools, SonarLint.
  2. Click on + beside the 'SonarCube / SonarClod connections' box to add a new connection
  3. Set Configuration Name to something like 'ONAP'
  4. Ensure sonarcloud is selected  and click [Next]
  5. Click [Create Token]
  6. Select GitHub, logon and follow the GitHub wizard to create a Token
  7. Copy Token into IntelliJ wizard
  8. Click [Next] (might have to wait a little while your data is being downloaded)
  9. Select 'Open Network Automation Platform (ONAP)' from 'Your Organizations' and click [Next]
  10. Click [Finish] to close the wizard
  11. Click [OK] to leave settings

Then you need to bind each project to the corresponding project in GitHub/SonarCloud:

  1. Select, File, Settings, Tools, SonarLint, Project Settings
  2. Enable (check) 'Binfd project to SonarQube / SonarCloud'
  3. Select the connection created above using the [Configure the connection] button
  4. Enter the project name (e.g. 'onap_ccsdk-sli-plugins') or select it using [Search in list..] 
  5. Start typing the name of your project, and it should appear in a list box where it should be selected.
  6. Click [OK] to finish

Other Tools

The component you are working on may require additional tool installations, see the relevant section under Development Guides.

Maven

For console (mvn) Install maven 3.3.3+ (currently 3.5.0) - as 3.2.x will have issues with the ODL certificate not matching

Will also fix

hostname in certificate didn't match: <nexus.opendaylight.org> != <logs.opendaylight.org> OR <logs.opendaylight.org>


Selecting a connection method

Once Git is installed, you will need to select a connection method to the Gerrit host, https://gerrit.onap.org/.

We recommend using SSH, in which case you need to register your private key in your account settings on the Gerrit host. However, if you are using a Virtual Private Network (VPN), you will require the HTTPS method, since most VPNs won't allow an ssh connection.

SSH Connection (Recommended)

Log in to the Gerrit host https://gerrit.onap.org/, pull down the menu under your user name (at the extreme top right of the browser window), click on "Settings":

Click your account name on the top right corner of the website and click on Settings.

In the Settings window, select SSH Public Keys, and then click Add Key... .

 Enter the public PuTTY key you generated above (by copying and pasting the entire contents of the file public.txt, in our example above.) and click Add.


HTTPS Connection (Alternate)

If you choose to use HTTP/HTTPS, you'll need to generate an access password. Log in to the Gerrit host https://gerrit.onap.org/, pull down the menu under your user name (at the extreme top right of the browser window), click on "Settings", and select "HTTP Password":


The icon at the right of the generated password allows you to copy the generated password to your clipboard, it will be needed when you interact with Git/Gerrit.

This password may have a limited time to live, so you might get errors like this one:

Problem running 'git remote update gerrit'
Fetching gerrit
fatal: unable to access 'https://USERNAME@gerrit.onap.org/r/a/mso/ ': Unknown SSL protocol error in connection to gerrit.onap.org:443 error: Could not fetch gerrit

Regenerating a password will most likely solve the connectivity issue. Sometimes, the Gerrit interface on HTTPS might be temporarily faulty, so retries might be needed.

Working with a Proxy

For developers working with a proxy, you might have proxy issues connecting to Linux Foundation website. To avoid the connection issue, you must define the proxy setting for both IntelliJ and Git.

To configure the proxy settings in IntelliJ, click File -> Settings -> Appearance & Behavior -> System Settings -> HTTP Proxy, enter the correct proxy settings and click Apply. In case of Manual proxy configuration, while entering the Host name, there is no need to write http:// in front of the URL. For example, if the proxy value is http://one.proxy.com, you would put one.proxy.com as the Host name.



To configure the proxy settings for Git, play the following command:

git config --global https.proxy https://<proxy username>:<proxy password>@<proxy url> 
git config --global http.proxy http://<proxy username>:<proxy password>@<proxy url>

Note: while entering the proxy username, sometimes it may require to add the domain name in front.


Building all ONAP projects

see Building Entire ONAP

Maven Example settings.xml

The most current settings.xml that you should use to build ONAP can be found in the oparent repo at https://git.onap.org/oparent/plain/settings.xml

Save this Maven settings.xml as your ~/.m2/settings.xml 

WSL users can point to one common repo across Windows and Ubuntu as :

ln -s /mnt/c/Users/<username>/.m2 ~

Some notes on the settings.xml:


If you are behind a proxy you can add a proxy section to your settings.xml 

	<proxies>
        <proxy>
            <id>evil-corp-http</id>
            <active>true</active>
            <protocol>http</protocol>
            <host>proxy.evil-corp.com</host>
            <port>599</port>
            <nonProxyHosts>localhost|127.0.0.1|*.evil-corp.com|*.happy.evil-corp.com|fun.evil-corp.com</nonProxyHosts>
        </proxy>
        <proxy>
            <id>evil-corp-https</id>
            <active>true</active>
            <protocol>https</protocol>
            <host>proxy.evil-corp.com</host>
            <port>599</port>
            <nonProxyHosts>localhost|127.0.0.1|*.evil-corp.com|*.happy.evil-corp.com|fun.evil-corp.com</nonProxyHosts>
        </proxy>
	</proxies>

Local SonarQube Setup

It can be useful to have SonarQube running locally rather than pushing through CI for feedback.

This can be done using docker:

Images of the Community, Developer, and Enterprise Editions are available on Docker Hub.

  1. Start the server by running:

    $ docker run -d --name sonarqube -p 9000:9000 sonarqube:latest
  2. Log in to http://localhost:9000 with System Administrator credentials (login=admin, password=admin).
  3. Click the Create new project button to analyze your first project.




Update in Progress below

This page is due some major revision, below section is a placeholder / WIP for new/updated layout 

ONAP Developer Set-Up (revised)

Create a Linux Foundation Account

Follow the instructions on the identity portal to create a Linux Foundation account and get access to the ONAP Gerrit instance.

Verify that you can log in at https://gerrit.onap.org/ and that you can see the ONAP list of repositories.

Basic Setup

Git

Install git for your OS in accordance with https://www.atlassian.com/git/tutorials/install-git

Since this is common for all OS, we will also use it to generate our SSH keys as well :

ssh-keygen

This should generate a private and public ssh key.

The public ssh key can then be uploaded to Gerrit or elsewhere as needed for authorization.

Java

ONAP is moving to Java 11 but many projects still use Java 8.

Download and install the appropriate openjdk version in accordance with the component you are using and set JAVA_HOME.

Maven

Download the latest Maven using installer from https://maven.apache.org/download.cgi

Add maven to your path variable.

To test the new installation run:

mvn -version

Use the settings.xml found in the oparent repo.

Save this Maven settings.xml as your ~/.m2/settings.xml 

WSL users can point to one common repo across Windows and Ubuntu as :

ln -s /mnt/c/Users/<username>/.m2 ~


If you are behind a proxy you can add a proxy section to your settings.xml 

 Proxy settings
	<proxies>
        <proxy>
            <id>evil-corp-http</id>
            <active>true</active>
            <protocol>http</protocol>
            <host>proxy.evil-corp.com</host>
            <port>599</port>
            <nonProxyHosts>localhost|127.0.0.1|*.evil-corp.com|*.happy.evil-corp.com|fun.evil-corp.com</nonProxyHosts>
        </proxy>
        <proxy>
            <id>evil-corp-https</id>
            <active>true</active>
            <protocol>https</protocol>
            <host>proxy.evil-corp.com</host>
            <port>599</port>
            <nonProxyHosts>localhost|127.0.0.1|*.evil-corp.com|*.happy.evil-corp.com|fun.evil-corp.com</nonProxyHosts>
        </proxy>
	</proxies>

OS Specific Instructions

MAC/OSX  ( to be reviewed by a MAC user )

get your MBP 2016 ready for development

Install homebrew package manager - 

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

python should already be installed. - do a --version check

install pip - sudo easy_install pip

install openstack tools

install docker (dmg) - https://docs.docker.com/docker-for-mac/install/#download-docker-for-mac

Redhat/CentOS ( to be reviewed by a Redhat/CentOS user )

20190130 - re-validating for RHEL 7.6 subscription



Add the following option to your MAVEN_OPTS in order not to periodically hang on downloading artifacts in linux

-Djava.net.preferIPv4Stack=true

enable IP4 forwarding to enable the RHEL VM to act as an edge router - without this docker containers cannot communicate

add to /etc/sysctl.conf
net.ipv4.ip_forward = 1

Install docker (required for some repos like dcae-inventory off dcae

sudo yum install -y yum-utils
sudo yum-config-manager     --add-repo     https://download.docker.com/linux/centos/docker-ce.repo
sudo yum-config-manager --enable docker-ce-edge
sudo yum makecache fast
sudo yum install docker-ce
sudo systemctl start docker
20190130


sudo curl https://releases.rancher.com/install-docker/17.03.sh | sh

sudo wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 -O jq
sudo chmod 777 jq
# not +x or jq will not be runnable from your non-root user
sudo mv jq /usr/local/bin
jq --version

sudo usermod -aG docker ubuntu



to install kubernetes follow Cloud Native Deployment#Scriptedundercloud(Helm/Kubernetes/Docker)andONAPinstall-SingleVM

Ubuntu ( to be reviewed by a Ubuntu user )



enable ubuntu desktop

apt-get update

apt-get install ubuntu-desktop

enable static IP

vi /etc/network/interfaces

add

iface enp0s25 inet static

address 192.168.15.101

netmask 255.255.255.0

network 192.168.15.0

broadcast 192.168.15.255

gateway 192.168.15.1

dns-nameservers 8.8.4.4

disable strict host checking (client and server)
in ~/.ssh/config
Host *
   StrictHostKeyChecking no
   UserKnownHostsFile=/dev/null

Enable non CD repositories so an apt-get update will work

comment out the CD first line in /etc/apt/sources.list
enable ssh if skipped during installation so we can remote sshsudo apt-get install openssh-server
enable root login and gui (no sudo su -)
sudo passwd root

sudo usermod -U root

sudo reboot now

sudo vi /etc/lightdm/lightdm.conf

[SeatDefaults]

greeter-session=unity-greeter

user-session=ubuntu

greeter-show-manual-login=true


: enable root login

vi /etc/ssh/sshd_config
FROM:
PermitRootLogin prohibit-password
TO:
PermitRootLogin yes

systemctl restart sshd

Add the following option to your MAVEN_OPTS in order not to periodically hang on downloading artifacts in linux

in /etc/environment

-Djava.net.preferIPv4Stack=true

Install docker (required when building docker images)

use the current as of 20180621 17.03 version in the script
https://git.onap.org/logging-analytics/tree/deploy/rancher/oom_rancher_setup.sh
which is


curl https://releases.rancher.com/install-docker/17.03.sh | sh


Windows

Note: The DCAE and SDC  repo will not fully clone in a windows environment (even git-bash) until some file paths are reduced below 255 chars -  DCAE-24 - Getting issue details... STATUS  and  SDC-21 - Getting issue details... STATUS  - you may use Linux or OSX native or VMs as an alternative for now.

Enabling long paths in windows

git config --system core.longpaths true

Connecting to ONAP Gerrit

SSH Connection (Recommended)

Log in to the Gerrit host https://gerrit.onap.org/, pull down the menu under your user name (at the extreme top right of the browser window), click on "Settings":

Click your account name on the top right corner of the website and click on Settings.

Add the public ssh key that you created in the previous step.

HTTPS Connection

 Click here to expand...

If you choose to use HTTP/HTTPS, you'll need to generate an access password. Log in to the Gerrit host https://gerrit.onap.org/, pull down the menu under your user name (at the extreme top right of the browser window), click on "Settings", and select "HTTP Password"

This password may have a limited time to live, so you might get errors like this one:

Problem running 'git remote update gerrit'
Fetching gerrit
fatal: unable to access 'https://USERNAME@gerrit.onap.org/r/a/mso/ ': Unknown SSL protocol error in connection to gerrit.onap.org:443 error: Could not fetch gerrit

Regenerating a password will most likely solve the connectivity issue. Sometimes, the Gerrit interface on HTTPS might be temporarily faulty, so retries might be needed.


Working with a Proxy

 Click here to expand...

For developers working with a proxy, you might have proxy issues connecting to Linux Foundation website. To avoid the connection issue, you must define the proxy setting for both IntelliJ and Git.

To configure the proxy settings in IntelliJ;

click File -> Settings -> Appearance & Behavior -> System Settings -> HTTP Proxy

 enter the correct proxy settings and click Apply.

In case of Manual proxy configuration, while entering the Host name, there is no need to write http:// in front of the URL. For example, if the proxy value is http://one.proxy.com, you would put one.proxy.com as the Host name


To configure the proxy settings for Git, play the following command:

git config --global https.proxy https://<proxy username>:<proxy password>@<proxy url> 
git config --global http.proxy http://<proxy username>:<proxy password>@<proxy url>

Note: while entering the proxy username, sometimes it may require to add the domain name in front.


IDEs

Eclipse

Install Eclipse

Download and run the installer from: Install Eclipse. Select "Eclipse IDE for Java Developers" to install.

ONAP Eclipse Java Formatter 

Download onap-java-formatter.xml and import into Eclipse. 

( Updates to the settings should be committed and merged back into the repository. )

ONAP uses Google Java Style with some modifications. ( See Java code style )

 Maven Plugins to Reformat Java code

Using the Eclipse Formatter file above and maven plugin configuration in pom.xml as per pom.xml 

(https://code.revelc.net/formatter-maven-plugin/ for Eclipse formatter and https://github.com/diffplug/spotless/tree/master/plugin-maven for import order)

Example pom.xml configuration
            <plugins>
                <!--
                Using https://code.revelc.net/formatter-maven-plugin/ for Eclipse formatter
                Using https://github.com/diffplug/spotless/tree/master/plugin-maven for import order
                Use in combination to rewrite code and imports, then checkstyle
                
                mvn formatter:format spotless:apply process-sources
                -->
                <plugin>
                     <groupId>net.revelc.code.formatter</groupId>
                     <artifactId>formatter-maven-plugin</artifactId>
                     <version>2.8.1</version>
                     <configuration>
                        <configFile>${project.parent.basedir}/onap-java-formatter.xml</configFile>
                     </configuration>
                     <!-- https://code.revelc.net/formatter-maven-plugin/
                          use mvn formatter:format to rewrite source files
                          use mvn formatter:validate to validate source files -->
                 </plugin>
                <plugin>
                  <groupId>com.diffplug.spotless</groupId>
                  <artifactId>spotless-maven-plugin</artifactId>
                  <version>1.18.0</version>
                  <configuration>
                    <java>
                     <importOrder>
                       <order>com,java,javax,org</order>
                     </importOrder>
                    </java>
                  </configuration>
                <!-- https://github.com/diffplug/spotless/tree/master/plugin-maven
                     use mvn spotless:apply to rewrite source files
                     use mvn spotless:check to validate source files -->
                </plugin>
            </plugins>


the combination can be used in a maven command to rewrite code and imports, then checkstyle audit like so

Example maven command
mvn formatter:format spotless:apply process-sources

Install useful plugins

Install EclEmma for code coverage and SonarLint for static code analysis.

Set up Sonar towards ONAP

To bind your projects to the ONAP Sonar server, follow the instructions below.

Your projects should be imported in to Eclipse before this.

 Click here to expand...
  • Right click on the project and select "SonarLint -> Bind to SonarQube or SonarCloud...".
  • Select "sonarcloud" and press "Next".
  • Click "Generate Token"
  • A browser opens and you are taken to a Sonarcloud login page
  • Login with an appropriate account from the list presented, most likely your GitHub account
  • You are now directed to a sonarcloud token generation page
  • Enter a name for your token and click "Generate"
  • Copy the token hex string that is generated from the browser and paste it into the "Token" field in Eclipse and click "Next"
  • In the "Organization" field, enter the string "onap" and press "Next"
  • The Connection name "SonarCloud/onap" should be found by the system, click "Next"
  • The connection should be successfully created, click "Finish"
  • Press "Add...".
  • Select the projects you want to add and press "OK".
  • Press "Next".
  • Start typing the name of your project, and it should appear in a list box where it should be selected.
  • Press "Finish".

To see messages from Sonar introduced by edits made in the projects, select "Window -> Show View -> Other...". Expand "SonarLint" and select "SonarLint -> On-The-Fly".

Setting up the ONAP Checkstyle in Eclipse

Set "ONAP" configuration in Eclipse

To set the newly built checkstyle files in Eclipse:

  1. Preferences->Checkstyle
  2. Click "New"
  3. Select "External Configuration File"
  4. Give it a name eg ONAP
  5. Point at the file <your_git_folder>/oparent/checkstyle/src/main/resources/onap-checkstyle/onap-java-style.xml  
    (assuming you have downloaded the oparent repo)
  6. Click OK
  7. Select "ONAP" configuration and click "Set as Default"
  8. Select "Apply and Close"

Apply "ONAP" configuration to a project in Eclipse

Now we need to activate the checkstyle on one project and set it as the blueprint for all of them:
Select a project in eclipse and right click->Properties->Checkstyle
check "Checkstyle active for this project"
Select the "ONAP" checkstyle profile
Click "Apply and Close"

Spread blueprint to other projects in Eclipse

Now spread the profile to all other projects:

  1. Select all the projects you want to apply the profile to in the Eclipse project explorer (not the one that you set up above)
  2. Right click->Checkstyle->Configure projects from blueprint
  3. Select the project you set up above
  4. Now all the projects have the correct checkstyle setup.


Hack oparent to fix "curly bracket" issue

 Click here to expand...

The first issue is that the current ONAP master tagged version of the checkstyle does not work with Eclipse Oxygen/Photon (and maybe other versions) because of the "curly bracket" issue. There is a fix on the way but it's not here yet. The current tagged version of oparent we are using in Policy is 1.2.1.

To get around this issue, check out oparent, checked out the 1.2.1 tag, fixed the "curly bracket" bug and built it on my local machine.

  1. Check out oparent
  2. git tag -l
  3. git co tags/1.2.3 -b 1.2.3
  4. If you want CheckStyle to ignore generated files, do the following:
    1. Add the following tag to the above file:
      <module name="SuppressionFilter">
        <property name="file" value="<absolute path to the directory of the file>/suppressions.xml"/>
      </module>
    2. Create a file called "suppressions.xml" in the folder given above, and put the following content in it:
      <?xml version="1.0"?>
      <!DOCTYPE suppressions PUBLIC
        "-//Checkstyle//DTD SuppressionFilter Configuration 1.2//EN"
        "https://checkstyle.org/dtds/suppressions_1_2.dtd">
      <suppressions>
        <suppress files="[\\/]generated-sources[\\/]" checks="[a-zA-Z0-9]*"/>
      </suppressions>
  5. The versions in the POMs in oparent are snapshot, so we need to change those to 1.2.1
     mvn versions:set -DnewVersion=1.2.1
  6. Now build locally:
    mvn clean install

IntelliJ

Optional 

Python

Check which version of python your project is using before installing. 

Download Python from https://www.python.org/downloads/

Add python binary to Path.

Install "pip3", if you plan to use git-review.

git-review

In a shell, type the following command (assumes you installed python above) :  

pip3 install git-review

To use git review, you have to be in a Git clone directory that already contains a (possibly hidden) .gitreview configuration file (see Gerrit/Advanced usage#Setting up a repository for git-remote)

The Git and git-review installation steps above are derived from the description at: https://www.mediawiki.org/wiki/Gerrit/git-review#Windows 

Configure Git to remember your Linux Foundation user name and email address (the user name and email address associated with your Linux Foundation login):


git config --global user.email <your_LF_account_email>

git config --global --add gitreview.username <your_LF_user_name>


Configure git review

git config --global gitreview.remote origin


If you are using VPN, you might encounter a proxy problem while connecting to the Linux Foundation website. To avoid the problem, you should add the proxy setting in git config, using the following command:


git config --global https.proxy https://<proxy username>:<proxy password>@<proxy url> 

git config --global http.proxy http://<proxy username>:<proxy password>@<proxy url>


NOTE: When entering the proxy username, you might be required to add the domain name in front of the username.

Node-JS

The dcae build will install npm in most cases.

However you can install it yourself from https://nodejs.org/en/download/

Verify your installation by 

npm -v

Local SonarQube Setup

It can be useful to have SonarQube running locally rather than pushing through CI for feedback.

This can be done using docker:

Images of the Community, Developer, and Enterprise Editions are available on Docker Hub.

  1. Start the server by running:

    $ docker run -d --name sonarqube -p 9000:9000 sonarqube:latest
  2. Log in to http://localhost:9000 with System Administrator credentials (login=admin, password=admin).
  3. Click the Create new project button to analyze your first project.

Troubleshooting

Appendix

  • No labels