HowTo Configure Switch Interface Using Cumulus Linux

Version 10

    In this post we will show how to change the port MTU and speed in Cumulus Linux OS. In addition, there are some useful commands.

    This post assumes that you already have Cumulus Linux installed on a Mellanox Spectrum switch.

     

    References

     

    Setup

    1. Two servers connected to a Cumulus Linux Switch.

     

    Make sure to have valid licence installed.

     

    Configuration

    1. Log into the switch using the following credentials:

    Username: cumulus

    Password: CumulusLinux!

     

    2. Switch to root user. Run sudo -i.

    Password: CumulusLinux!

    # sudo -i

    [sudo] password for cumulus:

     

    *****************************************************************

    Please send these support file(s) to support@cumulusnetworks.com:

    /var/support/cl_support__cumulus_20161018_211850.tar.xz

    /var/support/cl_support__cumulus_20161018_212948.tar.xz

    /var/support/cl_support__cumulus_20161018_213430.tar.xz

    /var/support/cl_support__cumulus_20161018_213607.tar.xz

    /var/support/cl_support__cumulus_20161018_213758.tar.xz

    /var/support/cl_support__cumulus_20161018_213924.tar.xz

    /var/support/cl_support__cumulus_20161018_220157.tar.xz

    *****************************************************************

     

    3. Make sure switchd service is running (If not - apply a valid license and restart the service):

    sudo systemctl status switchd

     

    4. Set Network Interfaces

    Starting Cumulus release 3.4 it is recommended to configure the interface using Network Command Line Utility (NCLU) as described below instead of manually editing configuration files.

     

    Enable the required network interfaces and apply configuration:

    net add interface swp5

    net add interface swp6

    net commit

    Set interface speed and mtu for swp6 interface:

    net add int swp6 mtu 9000

    net add int swp6 link speed 40000

    net commit

     

    For Cumulus releases prior to 3.4, Edit /etc/network/interfaces file:

    # vi /etc/network/interfaces

    Change link MTU and speed as follows:

    #swp5

    auto swp5

    iface swp5

    link-speed 100000

    mtu 1500

     

    # swp6

     

    auto swp6

    iface swp6

    link-speed 40000

    mtu 9000

    Apply the configuration:

    # ifreload -a

     

    Note: When changing the speed to 40GbE, you may need to force the speed on the server side as well. For more information please refer to HowTo Change the Ethernet Port Speed of Mellanox Adapters (Linux).

    Run the following on the server side on the link connected to port 6 (swp6), to change the link to 40Mb/s.

    # ethtool -s ens785f1 speed 40000 autoneg off

    If you want to split the port, please refer to HowTo Configure Breakout Cables 40G => 4x10G Using Cumulus Linux.

     

    5. Validation - to check the link status:

    # net show interface

     

    Name Speed MTU Mode Summary

    ----- ------ ------- ----- ------------- ------------------------

    UP lo N/A 65536 Loopback IP: 127.0.0.1/8, ::1/128

    UP eth0 1G 1500 Mgmt IP: 10.20.2.70/16(DHCP)

    ADMDN eth1 0M 1500 Mgmt

    UP swp5 100G 1500 NotConfigured

    UP swp6 40G 9000 NotConfigured

     

    You can see that both links, swp5 and swp6, are UP with the configured MTU and speed.

     

    For other useful commands, check Cumulus Linux: Useful Commands.

    Troubleshooting

    1. If you are using 100GbE adapters (e.g. ConnectX®-4), you may need to set the port speed on the server as well. Run:

    # ethtool -s ens785f1 speed 100000 autoneg off