...
Note on Openstack FPGA support
One good thing that is happening in Rocky release is that all resources whether it is CPU, disk, memory, PCIe SRIOV VF or even FPGA/GPU, are represented in uniform way. And hence the request for resources via flavor is also becomes uniform.There are no longer unique way of representing flavor attributes. Please see this design note here : https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/granular-resource-requests.html
In summary: Any resource request in flavor always start with property resource<N>:<resource name>:<count>. Any associated trait for the resource or traits by themselves can be represented as trait<N>:<trait name>=required.
In case of FPGA and GPU, there is additional generic property is getting added, whose syntax is function<N>:<Function type>=required. Note that this keyword is not interpreted by NOVA. It is only interpreted by Cyborg.
One example:
- Say that Openstack instance supports ARRIA10 FPGAs and functions such as IPSEC_ACCELERATION, PUBLIC_CRYPTO_ACCELERATION, TENSORFLOW_ACCELERATION, COMPRESSION_ACCELERATION.
- Say that compute nodes have 3 FPGA cards in each of them.
- Say that Openstack operator thinks that there would be
- Some workload types that require IPSEC_ACCLERATION and COMPRESSION_ACCELERATION together.
- Some workload types that require TENSORFLOW_ACCELERATION (2 of them) and COMPRESSION_ACCELERATION together
- Some workload types that require IPSEC_ACCELERATION, PUBLIC_CRYPTO_ACCLERATION and COMPRESSION_ACCELERATION.
Openstack operator creates following flavors as follows
Flavor 1:
- resource1:CUSTOM_ACCELERATOR_FPGA=1
- trait1:FPGA_INTEL_ARRIA10=required
- function1:IPSEC_ACCLERATION_AF=required
- resource2:CUSTOM_ACCELERATOR_FPGA=1
- trait2:FPGA_INTEL_ARRIA10=required
- function2:COMPRESSON_ACCLERATION_AF=required
Flavor 2:
- resource1:CUSTOM_ACCELERATOR_FPGA=2
- trait1:FPGA_INTEL_ARRIA10=required
- function1:TENSORFLOW_ACCLERATION_AF=required
- resource2:CUSTOM_ACCELERATOR_FPGA=1
- trait2:FPGA_INTEL_ARRIA10=required
- function2:COMPRESSON_ACCLERATION_AF=required
Flavor 3:
- resource1:CUSTOM_ACCELERATOR_FPGA=1
- trait1:FPGA_INTEL_ARRIA10=required
- function1:IPSEC_ACCLERATION_AF=required
- resource2:CUSTOM_ACCELERATOR_FPGA=1
- trait2:FPGA_INTEL_ARRIA10=required
- function2:PUBLIC_CRYPTO_ACCLERATION_AF=required
- resource3:CUSTOM_ACCELERATOR_FPGA=1
- trait3:FPGA_INTEL_ARRIA10=required
- function3:COMPRESSON_ACCLERATION_AF=required
Some points to note are:
- Grouping of various properities together is achieved by <N> in resource, trait and function keyword.
- Even though FPGA accelerators typically are PCIe cards, PCIe vendor ID/device ID and VFs are hidden. They don't need to be specified.