ONAP on Amazon EC2

Warning: Draft Content

This wiki is under construction

VID-17 - Getting issue details... STATUS

Amazon AWS is currently hosting our RI for ONAP Continuous Deployment on my private account for now - I have requested a grant specific to the jenkins, kibana and cd instances.

Auto Continuous Deployment via Jenkins and Kibana

Option 0: Deploy OOM Kubernetes to a spot VM

Recommendations


HD	120G EBS
CPU	32 vCPU
RAM	64 g

Peak Performance Metrics

We hit a peak of 44 cores during startup, with an external network peak of 1.2Gbps (throttled nexus servers at ONAP), a peak SSD write rate of 4Gbps and 55G ram on a 64 vCore/256G VM on AWS Spot.

AWS CLI EC2 Creation and Deployment

Install the AWS CLI on the bastion VM

Allocate an EIP static public IP (one-time)

Create a Route53 Record Set - Type A (one-time)

Request a spot EC2 Instance

Clean Ubuntu 16 = ami-cd0f5cb6

# request the usually cheapest $0.13 spot 64G EBS instance at AWS
aws ec2 request-spot-instances --spot-price "0.25" --instance-count 1 --type "one-time" --launch-specification file://aws_ec2_spot_cli.json

# don't pass in the the following - it will be generated for the EBS volume
            "SnapshotId": "snap-0cfc17b071e696816"
launch specification json
{
      "ImageId": "ami-c0c964ba",
      "InstanceType": "r4.2xlarge",
      "KeyName": "obrien_systems_aws_20141115",
      "BlockDeviceMappings": [
        {
          "DeviceName": "/dev/sda1",
          "Ebs": {
            "DeleteOnTermination": true,
            "VolumeType": "gp2",
            "VolumeSize": 120
          }
        }
      ],
      "SecurityGroupIds": [ "sg-322c4842" ]
}
# results
{
    "SpotInstanceRequests": [
        {
            "Status": {
                "Message": "Your Spot request has been submitted for review, and is pending evaluation.", 
                "Code": "pending-evaluation", 
                "UpdateTime": "2017-10-29T14:58:58.000Z"
            }, 
            "ProductDescription": "Linux/UNIX", 
            "InstanceInterruptionBehavior": "terminate", 
            "SpotInstanceRequestId": "sir-1tyr5etg", 
            "State": "open", 
            "LaunchSpecification": {
                "Placement": {
                    "AvailabilityZone": "us-east-1a"
                }, 
                "ImageId": "ami-cd0f5cb6", 
                "BlockDeviceMappings": [
                    {
                        "DeviceName": "/dev/sda1", 
                        "Ebs": {
                            "SnapshotId": "snap-0cfc17b071e696816", 
                            "DeleteOnTermination": true, 
                            "VolumeType": "gp2", 
                            "VolumeSize": 120
                        }
                    }
                ], 
                "KeyName": "obrien_systems_aws_20141115", 
                "SecurityGroups": [
                    {
                        "GroupName": "all_open", 
                        "GroupId": "sg-322c4842"
                    }
                ], 
                "SubnetId": "subnet-ece37889", 
                "Monitoring": {
                    "Enabled": false
                }, 
                "InstanceType": "r4.2xlarge"
            }, 
            "Type": "one-time", 
            "CreateTime": "2017-10-29T14:58:58.000Z", 
            "SpotPrice": "0.250000"
        }
    ]
}

Get EC2 instanceId after creation

aws ec2 describe-spot-instance-requests  --spot-instance-request-id sir-1tyr5etg
            "InstanceId": "i-02a653592cb748e27",

Associate EIP with EC2 Instance

Can be done separately as long as it is in the first 30 sec during initialization and before rancher starts on the instance.

aws ec2 associate-address --instance-id i-02a653592cb748e27 --allocation-id eipalloc-375c1d02
{
    "AssociationId": "eipassoc-a4b5a293"
}

Reboot EC2 Instance to apply DNS change to Rancher in AMI

aws ec2 reboot-instances --instance-ids i-02a653592cb748e27

SSH and upload OOM

Run the following (assumes oom repo folder, requires onap-parameters.yaml on the FS, assumes master branch)
https://github.com/obrienlabs/onap-root/blob/master/cd.sh

http://jenkins.onap.info/job/oom-cd/

Run OOM

Run Healthcheck

Run Automated Robot parts of vFirewall VNF

Report Results

Stop Spot Instance

obrienbiometrics:aws michaelobrien$ aws ec2 terminate-instances --instance-ids i-0040425ac8c0d8f63
{
    "TerminatingInstances": [
        {
            "InstanceId": "i-0040425ac8c0d8f63", 
            "CurrentState": {
                "Code": 32, 
                "Name": "shutting-down"
            }, 
            "PreviousState": {
                "Code": 16, 
                "Name": "running"
            }
        }
    ]
}

Verify Instance stopped

Video on Installing and Running the ONAP Demos#ONAPDeploymentVideos

WE can run ONAP on an AWS EC2 instance for $0.17/hour as opposed to Rackspace at $1.12/hour for a 64G Ubuntu host VM.

I have created an AMI on Amazon AWS under the following ID that has a reference 20170825 tag of ONAP 1.0 running on top of Rancher

ami-b8f3f3c3 : onap-oom-k8s-10

https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Images:visibility=public-images;search=ami-b8f3f3c3;sort=name

EIP 34.233.240.214 maps to http://dev.onap.info:8880/env/1a7/infra/hosts

A D2.2xlarge with 61G ram on the spot market https://console.aws.amazon.com/ec2sp/v1/spot/launch-wizard?region=us-east-1 at $0.16/hour for all of ONAP

It may take up to 3-8 min for kubernetes pods to initialize as long as you preload the docker images OOM-328 - Getting issue details... STATUS

Workaround for the disk space error - even though we are running with a 1.9 TB NVMe SSD

https://github.com/kubernetes/kubernetes/issues/48703

Use a flavor that uses EBS like M4.4xLarge which is OK

Use a flavor that uses EBS like M4.4xLarge which is OK - except for AAI right now

Expected Monthly Billing

r4.2xlarge is the smallest and most cost effective 64g min instance to use for full ONAP deployment - it requires EBS stores. This is assuming 1 instance up at all times and a couple ad-hoc instances up a couple hours for testing/experimentation.

Option 1: Migrating Heat to CloudFormation

Resource Correspondence

ID	Type	Parent	AWS	Openstack

Using the CloudFormationDesigner

https://console.aws.amazon.com/cloudformation/designer/home?region=us-east-1#

Decoupling and Abstracting Southbound Orchestration via Plugins

Part of getting another infrastructure provider like AWS to work with ONAP will be in identifying and decoupling southbound logic from any particular cloud provider using an extensible plugin architecture on the SBI interface.

see Multi VIM/Cloud (5/11/17), VID project (5/17/17), Service Orchestrator (5/14/17), ONAP Operations Manager (5/10/17), ONAP Operations Manager / ONAP on Containers

Design Issues

DI 1: Refactor nested orchestration in DCAE

Replace the DCAE Controller

DI 2: Elastic IP allocation

DI 3: Investigate Cloudify plugin for AWS

Cloudify is Tosca based - https://github.com/cloudify-cosmo/cloudify-aws-plugin

Links

Waiting for the EC2 C5 instance types under the C620 chipset to arrive at AWS so we can experiment under EC2 Spot - http://technewshunter.com/cpus/intel-launches-xeon-w-cpus-for-workstations-skylake-sp-ecc-for-lga2066-41771/ https://aws.amazon.com/about-aws/whats-new/2016/11/coming-soon-amazon-ec2-c5-instances-the-next-generation-of-compute-optimized-instances/

http://docs.aws.amazon.com/cli/latest/userguide/cli-install-macos.html

use

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
aws --version
aws-cli/1.11.170 Python/2.7.13 Darwin/16.7.0 botocore/1.7.28