HowTo Configure DevStack with Mellanox Drivers on Ubuntu 14.04

Version 9

    This post shows how to deploy Devstack environment on Ubuntu14.04 using two physical servers, OVS and ConnectX-4 / ConnectX-4 Lx in Ethernet environment.

     

    References

     

    Setup

    This post can be tested via simple setup that contains two servers and a switch with the following server functions:

    • Controller + Compute
    • Compute                 

     

    3.png

     

    Prerequisites

    1. Two servers with Ubuntu 14.04 installed

    2. MLNX_OFED 3.1 or later installed

    3. Ethernet Interfaces (ifconfig eth2 up) enabled

     

    Switch Configuration

    The switch should be configured as trunk/hybrid port (allow VLAN tag), in addition VLANs 2-10 should be open (as in the rest of this example).

    The following is an example for MLNX-OS switch configuration:

    1. Set switchport mode to trunk mode (can be also be hybrid):

    switch (config) # interface ethernet 1/1 switchport mode trunk

    switch (config) # interface ethernet 1/2 switchport mode trunk

     

    2. Create VLANs.

    switch (config) # vlan 2-10

    3. Allow vlans 2-10 on the ports above.

    switch (config) #interface ethernet 1/1 switchport trunk allowed-vlan add 2-10

    switch (config) #interface ethernet 1/2 switchport trunk allowed-vlan add 2-10

     

    Installation

    Run the following procedure on both servers:

     

    1. Create a Linux user for the DevStack installation.

    # groupadd stack

    # useradd -g stack -s /bin/bash -d /opt/stack -m stack

    # echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

     

    2. Make sure the git package is installed on the server.

    # apt-get install git

    ...

     

    3. Switch to the stack user.

     

    # su stack

     

     

    4. Add permissions to the /opt folder where the Devstack will be installed.

     

    # sudo chmod 777 /opt
    # cd /opt

     

     

    5. Clone the latest version of DevStack.

     

    # git clone https://git.openstack.org/openstack-dev/devstack

     

     

    6. Go to /opt/devstack folder (created by the git).

     

    # cd /opt/devstack

     

     

    7.  Switch to Kilo repository and confirm the applied.

    # git fetch origin stable/kilo:stable/kilo

    # git branch

    # git checkout stable/kilo

    # git branch

     

    8. Create a file called local.conf on each host and place it under /opt/devstack.

        Here is a configuration example for the Compute and Controller (Neutron) host:

    [[local|localrc]]

    DOWNLOAD_DEFAULT_IMAGES=False

    IMAGE_URLS="http://144.76.193.39/images/mellanox_eth.img,"

    ADMIN_PASSWORD=admin

    MYSQL_PASSWORD=password

    RABBIT_PASSWORD=password

    SERVICE_PASSWORD=password

    HOST_IP=$(host $(hostname) | cut -d' ' -f4)

    USE_SCREEN=True

     

    # Logging

    LOGDIR=${LOGDIR:-/opt/stack/logs}

    LOGFILE=$LOGDIR/stack.sh.log

    LOG_COLOR=False

     

    SWIFT_DATA_DIR=$DEST/data

     

    # Keystone

    SERVICE_TOKEN=servicetoken

    MULTI_HOST=1

     

    # Neutron

    Q_PLUGIN=ml2

    Q_AGENT=openvswitch

    Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch

    Q_USE_DEBUG_COMMAND=True

    Q_USE_SECGROUP=True

    ENABLE_TENANT_TUNNELS=False

    Q_ML2_TENANT_NETWORK_TYPE=vlan

    PHYSICAL_NETWORK=physnet1

    PHYSICAL_INTERFACE=eth0              # This is the interface name of the ConnectX-4 adapter.

    TENANT_VLAN_RANGE=physnet1:2:10      # This is the tenant VLAN range, in this case it is VLAN 2...10 (physnet1, is just a name for the physical network)

    ENABLE_TENANT_VLANS=True            

    ML2_VLAN_RANGES=physnet1:2:10        # This is the physical VLAN range, in this case it is VLAN 2...10 (physnet1, is just a name for the physical network)

    OVS_PHYSICAL_BRIDGE=br-eth0          # This is the name of the OVS bridge, the "-eth0" should be the physical interface written in PHYSICAL_INTERFACE

    VNCSERVER_LISTEN=10.20.2.75          # This is the management IP of the server.

     

    # Services

    disable_service h-eng h-api h-api-cfn h-api-cw n-net tempest

    enable_service neutron q-svc q-agt q-dhcp q-l3 q-meta n-novnc n-xvnc

     

     

        Here is a configuration example for the Compute only host:

    [[local|localrc]]

     

    ADMIN_PASSWORD=admin

    MYSQL_PASSWORD=password

    RABBIT_PASSWORD=password

    SERVICE_PASSWORD=password

    HOST_IP=$(host $(hostname) | cut -d' ' -f4)

    USE_SCREEN=True

     

    # Logging

    LOGDIR=${LOGDIR:-/opt/stack/logs}

    LOGFILE=$LOGDIR/stack.sh.log

    LOG_COLOR=False

     

    SWIFT_DATA_DIR=$DEST/data

     

    # Keystone

    SERVICE_TOKEN=servicetoken

    MULTI_HOST=1

     

     

    # Cinder

    VOLUME_BACKING_FILE_SIZE=1000M

     

    # Keystone

    SERVICE_TOKEN=servicetoken

     

    # Neutron

    Q_PLUGIN=ml2

    Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch

    Q_AGENT=openvswitch

    Q_USE_SECGROUP=False

    ENABLE_TENANT_VLANS=True

    Q_ML2_PLUGIN_TYPE_DRIVERS=vlan

    ENABLE_TENANT_TUNNELS=False

    Q_ML2_TENANT_NETWORK_TYPE=vlan

    PHYSICAL_NETWORK=physnet1

    PHYSICAL_INTERFACE=eth0         # This is the interface name of the ConnectX-4 adapter.

    NETWORK_API_EXTENSIONS=dhcp_agent_scheduler,external-net,ext-gw-mode,binding,quotas,agent,l3_agent_scheduler,provider,router,extraroute

    OVS_PHYSICAL_BRIDGE=br-eth0     # This is the name of the OVS bridge, the "-eth0" should be the physical interface written in PHYSICAL_INTERFACE

    TENANT_VLAN_RANGE=physnet1:2:10 # This is the tenant VLAN range, in this case it is VLAN 2...10 (physnet1, is just a name for the physical network)

    ML2_VLAN_RANGES=physnet1:2:10   # This is the physical VLAN range, in this case it is VLAN 2...10 (physnet1, is just a name for the physical network)

     

    ENABLED_SERVICES=neutron,rabbit,n-cpu,q-agt

     

    SERVICE_HOST=10.20.2.75          # This is the IP of the Controller

    MYSQL_HOST=10.20.2.75           # This is the IP of the Controller

    RABBIT_HOST=10.20.2.75          # This is the IP of the Controller

    Q_HOST=10.20.2.75                # This is the IP of the Controller

    GLANCE_HOSTPORT=10.20.2.75:9292  # This is the IP of the Controller

    NOVA_VNC_ENABLED=True

    NOVNCPROXY_URL="http://10.20.2.75:6080/vnc_auto.html"

    VNCSERVER_LISTEN=$HOST_IP

     

    SE_SCREEN=True

     

     

    10. Install the controller host.

    # ./stack

    ...

     

    11. Install the Compute node.

    # ./stack

    ...

     

    12. Add the physical interface (eth0) manually to the bridge on both nodes.

         This step is to prevent the VMs on the remote compute from coming up without an IP address. They come up without an IP address, because the DevStack does not attach the physical interface (eth0) we configured in local.conf to the OVS bridge (br-eth0).

     

    # ovs-vsctl add-port br-eth0 eth0

     

    At this point, the installation of the DevStack is completed. To login to the DevStack UI, use the management IP of the controller.

     

    For additional information refer DevStack Documentation.