This post shows the configuration flow that is required for OpenStack based on Mirantis Fuel 6.1
The reader is expect to have good experience with Mirantis Fuel installation as well with Mellanox adapters and drivers.
Using Mellanox OpenStack plugin together with ConnectX-3 adapters, helps to accelerate the cloud performance using SR-IOV based network connectivity for the VMs and ISER based block storage.
Mellanox OpenStack Plugin for Mirantis Fuel 6.1 supports InfiniBand or Ethernet networks.
- HowTo Install Mirantis Fuel 6.1 OpenStack with Mellanox Adapters Support (Ethernet Network)
- HowTo Install Mirantis Fuel 6.1 OpenStack with Mellanox Adapters Support (InfiniBand Network)
- HowTo Install Mirantis Fuel 5.1/5.1.1 OpenStack with Mellanox Adapters Support (Ethernet Network)
- HowTo Install Mirantis Fuel 5.1/5.1.1 OpenStack with Mellanox Adapters Support (InfiniBand Network)
- HowTo Configure iSER Block Storage for OpenStack Cloud with Mellanox ConnectX-3 Adapters
- Designing CloudX Solution using Mirantis Fuel OpenStack Software
- Mellanox OFED Driver Installation and Configuration for SR-IOV
- HowTo Configure 56GbE Link on Mellanox Adapters and Switches
- Mirantis Fuel OpenStack Plugins
- Running Mirantis Fuel OpenStack v6.0 | Documentation
The setup is based on the following post:
1. In case having Ethernet network, all required VLANs should be configured in addition flow control should be enabled on the switch ports.
2. In case of InfinBand network refer to the following at Mellanox OpenStack wiki for configuring the Subnet Manager (SM).
3. Mellanox SR-IOV and iSER are supported only when selecting Neutron with VLAN on the Fuel dashboard.
4. OVS bonding and Mellanox SR-IOV based networking over the Mellanox ConnectX-3 adapter family are not supported.
5. In order to use SR-IOV, one should choose KVM hypervisor and “Neutron with Vlan segmentation” in the Network settings tab on the Fuel dashboard.
6. Mellanox plugin supports both Ethernet and Infinband configuration. The plugin will recognize the network type, and install the cloud according to it. However, in order to support the Inifinband auto-detection, the default bootstrap should be upgraded. The Mellanox plugin installation will automatically replace and upgrade the bootstrap as part of its installation.
7. The SR-IOV feature supports only KVM hypervisor + Neutron with VLAN segmentation.
8. The iSER feature requires “Cinder LVM over iSCSI for volumes” enabled in the “Storage” section.
9. You cannot install a plugin at the already existing environment. That means, the plugin will appear in the certain environment only if the plugin was installed before creating the environment.
10. Enabling the “Mellanox Openstack features” section enables Mellanox hardware support on your environment, regardless of the iSER & SR-IOV features.
11. In Ethernet cloud, when using SR-IOV & iSER, one of the virtual NICs for SR-IOV will be reserved to the storage network.
12 When using SR-IOV you can set the number of virtual NICs (virtual functions) to up to 64, if your hardware and system capabilities (memory, BIOS etc) support it. In any case of SRIOV hardware limitation, the installation will try to fallback the (Virtual Funtion) VF number to the default of 16 VFs.
13. As a prerequisite Fuel 6.1 should be installed in your network.
Mellanox OpenStack Plugin installation
2. Login to the fuel master and run:
# fuel plugins --install mellanox-plugin-1.0-1.0.0-0.noarch.rpm
Note: The Mellanox plugin replaces the current bootstrap image, the original image is backed up in /opt/old_bootstrap_image/
3. Verify that the plugin was successfully installed by having it listed in fuel plugins command.
[root@fuel ~]# fuel plugins
id | name | version | package_version
1 | mellanox-plugin | 1.0.0 | 2.0.0
4. Check your target nodes status by running fuel nodes command.
If you already have nodes in “discover” status (with the original bootstrap image) Use the reboot_bootstrap_nodes script to reboot your nodes with the new image.
For more info about using the script run reboot_bootstrap_nodes --help.
If fuel nodes command doesn't show any nodes then you can boot the nodes.
[root@fuel ~]# fuel nodes
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
59 | ready | Untitled (3c:9a) | 15 | 10.20.0.6 | c8:1f:66:ef:3c:9a | compute | | True | 15
54 | ready | Untitled (4f:aa) | 14 | 10.20.0.10 | c8:1f:66:ef:4f:aa | compute | | True | 14
57 | ready | Untitled (78:09) | 15 | 10.20.0.3 | c8:1f:66:ee:78:09 | controller | | True | 15
53 | ready | Untitled (31:f4) | 14 | 10.20.0.9 | c8:1f:66:ef:31:f4 | compute | | True | 14
58 | ready | Untitled (d9:5d) | 15 | 10.20.0.4 | c8:1f:66:f0:d9:5d | cinder | | True | 15
60 | ready | Untitled (8e:ac) | 15 | 10.20.0.5 | c8:1f:66:ee:8e:ac | compute | | True | 15
56 | ready | Untitled (4e:2d) | 14 | 10.20.0.8 | c8:1f:66:ed:4e:2d | cinder | | True | 14
52 | ready | Untitled (d8:7d) | 14 | 10.20.0.7 | c8:1f:66:f0:d8:7d | controller | | True | 14
5. Create an environment – for more information please see Mirantis Fuel OpenStack v6.0 | Documentation.
6. Open the Settings tab of the Fuel web UI and scroll down the page.
a. In Mellanox OpenStack features section, select the required features.
b. In SR-IOV feature supports only KVM hypervisor
c. Select Neutron with VLAN segmentation
d.The iSER feature requires Cinder LVM over iSCSI for volumes enabled in the “Storage” section.
7. In case of using Infiniband network, in order to verify Infiniband links for nodes:
- Access the Nodes tab in your environment.
- Click on the cog wheel on the right of a node:
c. Verify that the driver is eth_ipoib and the bus info is an IB interface.