ONAP SR-IOV NIC design [DRAFT]

Background

Main purpose of this feature to place VNFCs in sites with SRIOV-NIC enabled compute nodes.

If there are no sites with available SRIOV-NIC compute nodes, OOF can choose next best compute node flavors.  Next best compute nodes may not SRIOV-NICs.  Of course VNFCs, in this case, assumed to be tested with and without SRIOV-NIC by vendors.

The parameters to instantiate VNFs differ whether SRIOV-NIC based switching or normal vSwitch based switching at the NFVI.

In case of Openstack based cloud-regions, port is expected to be created explicitly with binding:vnic_type as direct. In case of vSwitch based switching, there is no need for explicit creation of port, but port can be created with binding:vnic_type as normal. So, based on flavor, appropriate value is expected to be passed when talking to Openstack.

Some more additional requirements are also to be considered.

• A given VNFC might require multiple SRIOV VFs  assigned to it from different PCIe NIC cards.
• A given VNFC might require VFs from different types of PCIe NIC Cards (example: Intel, Melanox etc..)
• A given VNFC might require these VFs coming from different provider networks.

Those requirements can be satisfied, only if there are compute nodes with that kind of NIC hardware and only if ONAP can discover that from various cloud sites.

Overall design

1. Scenario

Let us say, a site has three kinds of compute nodes with respect to SRIOV-NIC

1st set contains, say SRIOV-NIC cards of type XYZ (PCIe vendor ID: 1234  Device ID: 5678) and YUI(PCIe vendor ID:2345 Device ID:6789.

2nd set contains say SRIOV-NIC cards of type ABC (PCIe vendor ID: 4321  Device ID: 8765).

3rd set does not contain any SRIOV NIC cards.

1.1 Openstack Config SRIOV

Openstack configuration:

1. NIC configuration  refer to https://docs.openstack.org/neutron/pike/admin/config-sriov.html

2. An example of a site having three types of compute nodes.   1st set of compute nodes have two SRIOV NIC cards with vendor/device id as 1234, 5678 and vendor/device id as 2345 &6789. 2nd set of compute nodes have two SRIOV-NIC of same type 4321 & 8765. And third set of compute nodes don't have any SRIOV-NIC cards.  And hence Openstack administrator at the site creates three flavors to reflect the hardware the site has.  As you see in this example, it is expected that alias format is followed. Alias value supposed to be of the form "NIC-sriov-<vendor ID>-<device ID>-<Provider network>

    $ openstack flavor create flavor1 --id auto --ram 512 --disk 40 --vcpus 4

    $ openstack flavor set flavor1 --property pci_passthrough:alias=SRIOV-NIC_1234_5678_physnet1:1

    $ openstack flavor set flavor1 --property pci_passthrough:alias=SRIOV-NIC_2345_6789_physnet2:1

    
   Flavor2:
    $ openstack flavor create flavor2 --id auto --ram 512 --disk 40 --vcpus 4

    $ openstack flavor set flavor2 --property pci_passthrough:alias=SRIOV-NIC_4321_8765:2

   
   Flavor3
    $ openstack flavor create flavor3 --id auto --ram 512 --disk 40 --vcpus 4

    

   
   

1.2 Mutli-cloud discovery

When it reads the flavors information from openstack site, if the pci_passthrough alias start with NIC-sriov, then it assumes that it is SRIOV NIC type.

Next two integers are meant for vendor id and device id.

If it is present after device id, it is assumed to be provider network.

As part of discovery, it populates the A&AI with two PCIe features for Flavor1.

hpa-feature=”pciePassthrough”,

architecture=”{hw_arch}",

version=”v1”,

hpa-feature=”pciePassthrough”,

architecture=”{hw_arch}",

version=”v1”,

1.3 SO call OOF

SO will get pciVendorId, pciDeviceId and interfaceType from CSAR file, then call to OOF. OOF will response homing information to SO, SO don't interpret it and pass through it to Multi-cloud.

HOT template that uses parameter to be filled up based on OOF output (Example)

1.4 VF-C call OOF

VF-C will get pciVendorId, pciDeviceId and interfaceType from CSAR file, then call to OOF. OOF will response homing information to VF-C.

VF-C get OOF response, It will create network of SRIOV-NIC using physical network.
Then, It will create port based on network using interface type.

1.5 OOF Response

OOF just check /cloud-infrastructure/cloud-regions/cloud-region/{cloud-owner}/{cloud-region-id}/flavors/flavor/{flavor-id}/hpa-capabilities

OOF will match the SRIOV information along with the constraint provided by Policy and add extra attributes inside the assignmentInfo data block when returning the response to SO and VF-C.

Sample looks like below.

"assignmentInfo": [ 
    { "key":"locationId",
      "value":"DLLSTX1A" },

    { "key":"locationType",
      "value":"openstack-cloud" },

    { "key":"vimId",
      "value":"rackspace_DLLSTX1A" },

    { "key":"oofDirectives",
      "value":{

        "directives":[

          { "vnfc_directives":[
               {
                  "vnfc_id":"<ID of VNFC>",
                  "directives":[
                     {
                        "directive_name":"<Name of directive,example flavor_directive>",
                        "attributes":[
                           {
                              "attribute_name":"<name of attribute, such as flavor label>",
                              "attribute_value":"<value such as cloud specific flavor>"
                           }
                        ]
                     },
                     {
                        "directive_name":"<Name of directive,example vnic-info>",
                        "attributes":[
                           {
                              "attribute_name":"<name of attribute, such as vnic-type>",
                              "attribute_value":"<value such as direct/normal>"
                           },
                           {
                              "attribute_name":"<name of attribute, such as provider netweork>",
                              "attribute_value":"<value such as physnet>"
                           }
                        ]
                     }
                  ]
               }
            ]
         },
         {
            "vnf_directives":{
               "directives":[
                  {
                     "directive_name":"<Name of directive>",
                     "attributes":[
                        {
                           "attribute_name":"<name of attribute>",
                           "attribute_value":"<value>"
                        }
                     ]
                  },
                  {
                     "directive_name":"<Name of directive>",
                     "attributes":[
                        {
                           "attribute_name":"<name of attribute>",
                           "attribute_value":"<value >"
                        },
                        {
                           "attribute_name":"<name of attribute>",
                           "attribute_value":"<value >"
                        }
                     ]
                  }
               ]
            }
         }
      ]
   }

}

]    

For the newly added oofDirectives, we only return the vnfc part. For example: 

"vnfc_directives": [

   {   "vnfc_id":"",

       "directives":[

           {  "directive_name": "flavor_directive",

              "attributes": [
                    {"attribute_name": "flavor_label_1", "attribute_value":"HPA.flavor"}

                           ]

           },
          {   "directive_name": "vnic-info1",
                "attributes": [
                    {"attribute_name": "vnic-type", "attribute_value":"direct"},
                    {"attribute_name": "provider_network", "attribute_value":"physnet1"}
                ]
          },

          {   "directive_name": "vnic-info2",
                "attributes": [
                    {"attribute_name": "vnic-type", "attribute_value":"direct"},
                    {"attribute_name": "provider_network", "attribute_value":"physnet2"}
                ]
          }

       ]
    }       
]

It is worth noting that the vnic-type is converted from interfaceType in OOF.

If interfaceType is SRIOV-NIC, then OOF returns 'vnic-type' as 'direct', If interfaceType is not SRIOV-NIC, OOF return 'vnic-type' as 'normal'.

1.6 Policy Data

"flavorLabel": "flavor_label_1",

"sriovNICLabel": "oof_returned_vnic_type_1

"flavorProperties":[

    {

         "hpa-feature" : "pciePassthrough",

         "mandatory" : "True",

         "architecture": "generic",

         "hpa-feature-attributes": [

             {"hpa-attribute-key":"pciVendorId", "hpa-attribute-value": "1234","operator": "=", "unit": ""},

             {"hpa-attribute-key":"pciDeviceId", "hpa-attribute-value": "5678","operator": "=", "unit": ""},

             {"hpa-attribute-key":"pciCount", "hpa-attribute-value": "1","operator": "=", "unit": ""},

             {"hpa-attribute-key":"cardType "hpa-attribute-value": "sriov-nic","operator": "=", "unit": ""},

             {"hpa-attribute-key":"providerNetwork "hpa-attribute-value": "physnet1","operator": "=", "unit": ""},

         ]

    },

   {

         "hpa-feature" : "pciePassthrough",

         "mandatory" : "True",

         "architecture": "generic",

         "hpa-feature-attributes": [

             {"hpa-attribute-key":"pciVendorId", "hpa-attribute-value": "2345","operator": "=", "unit": ""},

             {"hpa-attribute-key":"pciDeviceId", "hpa-attribute-value": "6789","operator": "=", "unit": ""},

             {"hpa-attribute-key":"pciCount", "hpa-attribute-value": "1","operator": "=", "unit": ""},

             {"hpa-attribute-key":"cardType "hpa-attribute-value": "sriov-nic","operator": "=", "unit": ""},

             {"hpa-attribute-key":"providerNetwork "hpa-attribute-value": "physnet2","operator": "=", "unit": ""},

         ]

     },

]

2. ONAP Module Modify 

3. SR-IOV NIC related Capability in Data model

This is refer to Supported HPA Capability Requirements(DRAFT)#LogicalNodei/ORequirements

Logical Node i/O Requirements

Network Interface Requirements

4. Reference

https://docs.openstack.org/neutron/pike/admin/config-sriov.html

https://builders.intel.com/docs/networkbuilders/EPA_Enablement_Guide_V2.pdf  of "2.3 Support for I/O Passthrough via SR-IOV"

HPA Policies and Mappings

Supported HPA Capability Requirements(DRAFT)#LogicalNodei/ORequirements