ONAP SR-IOV NIC design [DRAFT]

Overall design

1. Scenario

Let us say, a site has three kinds of compute nodes with respect to SRIOV-NIC

1st set contains, say SRIOV-NIC cards of type XYZ (PCIe vendor ID: 1234 Device ID: 5678) and YUI(PCIe vendor ID:2345 Device ID:6789.

2nd set contains say SRIOV-NIC cards of type ABC (PCIe vendor ID: 4321 Device ID: 8765).

3rd set does not contain any SRIOV NIC cards.

1.1 Openstack Config SRIOV

Openstack configuration:

NIC configuration refer to https://docs.openstack.org/neutron/pike/admin/config-sriov.html
Create flavors.
Flavor1:
$ openstack flavor create flavor1 --id auto --ram 512 --disk 40 --vcpus 4
$ openstack flavor set flavor1 --property pci_passthrough:alias=NIC-sriov-1234-5678-physnet1:1
$ openstack flavor set flavor1 --property pci_passthrough:alias=NIC-sriov-2345-6789-physnet2:1

Flavor2:
$ openstack flavor create flavor2 --id auto --ram 512 --disk 40 --vcpus 4
$ openstack flavor set flavor2 --property pci_passthrough:alias=NIC-sriov-4321-8765:2

Flavor3
$ openstack flavor create flavor3 --id auto --ram 512 --disk 40 --vcpus 4

1.2 Mutli-cloud discovery

When it reads the flavors information from openstack site, if the pci_passthrough alias start with NIC-sriov, then it assumes that it is SRIOV NIC type.

Next two integers are meant for vendor id and device id.

If it is present after device id, it is assumed to be provider network.

As part of discovery, it populates the A&AI with two PCIe features for Flavor1.

hpa-feature=”pciePassthrough”,

architecture=”{hw_arch}",

version=”v1”,

Hpa-attribute-key	Hpa-attribute-value
pciVendorId	1234
pciDeviceId	5678
pciCount	1
interfaceType	SRIOV-NIC
providerNetwork	Physnet1

hpa-feature=”pciePassthrough”,

architecture=”{hw_arch}",

version=”v1”,

Hpa-attribute-key	Hpa-attribute-value
pciVendorId	2345
pciDeviceId	6789
pciCount	1
interfaceType	SRIOV-NIC
providerNetwork	Physnet2

1.3 SO call OOF

SO will get pciVendorId, pciDeviceId and interfaceType from CSAR file, then call to OOF. OOF will response homing information to SO, SO don't interpret it and pass through it to Multi-cloud.

HOT template that uses parameter to be filled up based on OOF output

parameters:
  oof_returned_vnic_type_for_firewall:
  type:string
  description:This parameter value is determined by OOF. If OOF selects the region and flavor that support SRIOV-NICs, then OOF returns 'direct'. If not, it returns 'normal'
resources:
  network_protected:
    type: OS::Neutron::Net
    properties:
      name: mynetwork_protected
      network_type: vlan
      physical_network: physnet1
      segmentation_id: 100
  network_unprotected:
    properties:
      name: mynetwork_unprotected
      network_type: vlan
      physical_network: physnet2
      segmentation_id: 200
/** Subnet details are not provided for brevity**/
  protected_interface:
     type: OS::Neutron::Port
     properties:
       name: protected_port
       network_id : { get_resource: network_protected }
       binding:vnic_type: {get_param: oof_returned_vnic_type_for_firewall}
  unprotected_interface:
     type: OS::Neutron::Port
     properties:
       name: unprotected_port
       network_id : { get_resource: network_unprotected }
       binding:vnic_type: {get_param: oof_returned_vnic_type_for_firewall}

  virtual_firewall_appliance:
     type: OS::Nova::Server
     image: ....
     networks:
        - port: {get_resource: protected_interface}
        - port: {get_resource: unprotected_interface}

1.4 VF-C call OOF

VF-C will get pciVendorId, pciDeviceId and interfaceType from CSAR file, then call to OOF. OOF will response homing information to VF-C.

VF-C get OOF response, It will create network of SRIOV-NIC using physical network.
Then, It will create port based on network using interface type.

1.5 OOF Response

OOF just check /cloud-infrastructure/cloud-regions/cloud-region/{cloud-owner}/{cloud-region-id}/flavors/flavor/{flavor-id}/hpa-capabilities

OOF will match the SRIOV information along with the constraint provided by Policy and add extra attributes inside the assignmentInfo data block when returning the response to SO and VF-C.

Sample looks like below.

"assignmentInfo": [ 
    { "key":"locationId",
      "value":"DLLSTX1A" },

    { "key":"locationType",
      "value":"openstack-cloud" },

    { "key":"vimId",
      "value":"rackspace_DLLSTX1A" },

    { "key":"oofDirectives",
      "value":{

        "directives":[

          { "vnfc_directives":[
               {
                  "vnfc_id":"<ID of VNFC>",
                  "directives":[
                     {
                        "directive_name":"<Name of directive,example flavor_directive>",
                        "attributes":[
                           {
                              "attribute_name":"<name of attribute, such as flavor label>",
                              "attribute_value":"<value such as cloud specific flavor>"
                           }
                        ]
                     },
                     {
                        "directive_name":"<Name of directive,example vnic-info>",
                        "attributes":[
                           {
                              "attribute_name":"<name of attribute, such as vnic-type>",
                              "attribute_value":"<value such as direct/normal>"
                           },
                           {
                              "attribute_name":"<name of attribute, such as provider netweork>",
                              "attribute_value":"<value such as physnet>"
                           }
                        ]
                     }
                  ]
               }
            ]
         },
         {
            "vnf_directives":{
               "directives":[
                  {
                     "directive_name":"<Name of directive>",
                     "attributes":[
                        {
                           "attribute_name":"<name of attribute>",
                           "attribute_value":"<value>"
                        }
                     ]
                  },
                  {
                     "directive_name":"<Name of directive>",
                     "attributes":[
                        {
                           "attribute_name":"<name of attribute>",
                           "attribute_value":"<value >"
                        },
                        {
                           "attribute_name":"<name of attribute>",
                           "attribute_value":"<value >"
                        }
                     ]
                  }
               ]
            }
         }
      ]
   }

For the newly added oofDirectives, we only return the vnfc part. For example:

"vnfc_directives": [

   {   "vnfc_id":"",

       "directives":[

           {  "directive_name": "flavor_directive",

              "attributes": [
                    {"attribute_name": "flavor_label_1", "attribute_value":"HPA.flavor"}

           },
          {   "directive_name": "vnic-info1",
                "attributes": [
                    {"attribute_name": "vnic-type", "attribute_value":"direct"},
                    {"attribute_name": "provider_network", "attribute_value":"physnet1"}
                ]
          },

          {   "directive_name": "vnic-info2",
                "attributes": [
                    {"attribute_name": "vnic-type", "attribute_value":"direct"},
                    {"attribute_name": "provider_network", "attribute_value":"physnet2"}
                ]
          }

       ]
    }       
]

It is worth noting that the vnic-type is converted from interfaceType in OOF.

If interfaceType is SRIOV-NIC, then OOF returns 'vnic-type' as 'direct', If interfaceType is not SRIOV-NIC, OOF return 'vnic-type' as 'normal'.

4. ONAP Module Modify

Module Name	Modification	status	owner	comments
SDC	Add SR-IOV NIC attributes.	Completed	Alex Lianhao
Policy	Add SR-IOV NIC attributes.	In Progress
VF-C	Add create port process.	In Progress	Haibin
SO	Add create port process.	In Progress
OOF	Add the process for cloud region HPA capabilities	In Progess	Ruoyu
AAI	Nothing, we just add one hpa-attribute-key and hpa-attribute-value	Completed	-	now [A] API can support.
ESR	Add SR-IOV NIC info to cloud extra info.	In Progress	Haibin
Multi-cloud	Register SR-IOV info to AAI.	In Progress	Haibin
VIM	Config SR-IOV NIC and create network with SR-IOV NIC.	In Progress	Haibin

[A] PUT /cloud-infrastructure/cloud-regions/cloud-region/{cloud-owner}/{cloud-region-id}/hpa-capabilities/hpa-capability/{hpa-capability-id}

4.1 VF-C Modification

When it create network, VF-C will get provider network including interface type, physical_network and vnic type from OOF pass it Multi-cloud.
When we create port , we need pass vnic type to multi-cloud. so we don't modify it.

4.2 Multi-cloud Modification

https://developer.openstack.org/api-ref/network/v2/

Multi-cloud need to get provider network information and modify all provider network information for the request of submitting to VIM.
Multi-cloud don't modify VF-C request, just pass to VIMs, so we don't modify it.

4.3 SO Modification

Please Marcus Williams helps to update it.

4.4 OOF Modification

As we known OOF just check /cloud-infrastructure/cloud-regions/cloud-region/{cloud-owner}/{cloud-region-id}/flavors/flavor/{flavor-id}/hpa-capabilities

Now, we need OOF also check and match /cloud-infrastructure/cloud-regions/cloud-region/{cloud-owner}/{cloud-region-id}/hpa-capabilities/hpa-capability/{hpa-capability-id}

Seond, we need to add some attributes in the response returning to SO and VF-C like below.

"oof_directives" : {
         "vnfc_directives":[
              { "vnfc_id" : "<ID of VNFC>" //Need not filled in today, but for future purposes
                "directives" : [
                  {"directive_name": "<Name of directive,example flavor_directive>",
                   "attributes": [
                       {"attribute_name": "<name of attribute, such as flavor label>", "attribute_value":"<value such as cloud specific flavor>"},

                    ]
                  },
                  {"directive_name": "<Name of directive,example vnic-info>",
                   "attributes": [
                       {"attribute_name": "<name of attribute, such as vnic-type>", "attribute_value":"<value such as direct/normal>"},
                       {"attribute_name":"<name of attribute, such as provider netweork>">, "attribute_value":"<value such as physnet>"
                    ]
                  }
                }
              ]
                      }.
         "vnf_directives": {
              "directives" : [
                  {"directive_name": "<Name of directive>",
                   "attributes": [
                       {"attribute_name": "<name of attribute>", "attribute_value":"<value>"},

                    ]
                  },
                  {"directive_name": "<Name of directive>",
                   "attributes": [
                       {"attribute_name": "<name of attribute>", "attribute_value":"<value >"},
                       {"attribute_name":"<name of attribute>">, "attribute_value":"<value >"
                    ]
                  }
          }

}

SR-IOV NIC related Capability in Data model

This is refer to Supported HPA Capability Requirements(DRAFT)#LogicalNodei/ORequirements

Logical Node i/O Requirements

Capability Name	Capability Value	Descriptiopn
pciVendorId		PCI-SIG vendor ID for the device
pciDeviceId		PCI-SIG device ID for the device
pciNumDevices		Number of PCI devices required.
pciAddress		Geographic location of the PCI device via the standard PCI-SIG addressing model of Domain:Bus:device:function
pciDeviceLocalToNumaNode	required notRequired	Determines if I/O device affinity is required.

Network Interface Requirements

Capability Name	Capability Value	Description
nicFeature	LSO, LRO, RSS, RDMA	Long list of NIC related items such as LSO, LRO, RSS, RDMA, etc.
dataProcessingAccelerationLibray	Dpdk_Version	Name and version of the data processing acceleration library required. Orchestration can match any NIC that is known to be compatible with the specified library.
interfaceType	Virtio, PCI-Passthrough, SR-IOV, E1000, RTL8139, PCNET	Network interface type
vendorSpecificNicFeature	TBA	List of vendor specific NIC related items.

Reference

https://docs.openstack.org/neutron/pike/admin/config-sriov.html

https://builders.intel.com/docs/networkbuilders/EPA_Enablement_Guide_V2.pdf of "2.3 Support for I/O Passthrough via SR-IOV"

HPA Policies and Mappings

Supported HPA Capability Requirements(DRAFT)#LogicalNodei/ORequirements