How To Install/Upgrade Mellanox NATIVE ESXi Driver for VMware vSphere ESXi 6.5/6.7 host (CLI)

Version 23

    This is a basic post that shows how to install and test the Mellanox ConnectX-4/ConnectX-5 NATIVE ESXi Driver for VMware vSphere ESXi 6.5/6.7 on single host, and perform basic initial configuration steps to enable the driver by using ESXi cli interface.

     

    References

    Hardware and Software Requirements

    1. A server platform with an adapter card based on one of the following Mellanox Technologies’ HCA devices:

    2. Installer Privileges: The installation requires administrator privileges on the target machine.

    3. Device ID: For the latest list of device IDs, please visit Mellanox website.

    4. Supported NICs / Firmware: Recommended firmware versions you can find here.

        How To Update Firmware for ConnectX®-4/5 VPI PCI Express Adapter Cards (InfiniBand, Ethernet, VPI) on VMware ESXi 6.5/6.7.

     

     

    Driver Installation (CLI)

    1. Enable SSH Access to ESXi server.

    2. Log into ESXi vSphere Command-Line Interface with root permissions.

    3. Verify that the host is equipped with Mellanox adapter.

    ~ lspci | grep Mellanox

    0000:02:00.0 Network controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex] [vmnic2]

    0000:02:00.1 Network controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex] [vmnic3]

    Note: in this case, Mellanox card is using vmnic2 and vmnic3.

    4. Verify the driver's version was installed.

    ESXi 6.5

    ~ esxcli software vib list | grep nmlx
    nmlx5-core                     4.16.13.5-10EM-650.0.0-8277601        VMW       VMwareCertified   2018-04-24
    nmlx5-rdma                     4.16.13.5-10EM-650.0.0-8277601        VMW       VMwareCertified   2018-04-24

    ESXi 6.7

    ~ esxcli software vib list | grep nmlx

    nmlx4-core                     3.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-04-25

    nmlx4-en                       3.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-04-25

    nmlx4-rdma                     3.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-04-25

    nmlx5-core                     4.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-04-25

    nmlx5-rdma                     4.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-04-25

    5. Download a latest Mellanox native ESXi drivers from here.

    6. Unzip the binary image (.zip file).

    7.  Use SCP or any other file transfer method to copy the driver to the required ESXi host.

    8. Enter Maintenance Mode the ESXi host.

    9. Install the driver.

      ~ esxcli software vib install –d <path>/<bundle_file>

    ESXi 6.5

    ~ esxcli software vib install -d /tmp/MLNX-NATIVE-ESX-ConnectX-4-5_4.16.13.5-10EM-650.0.0.4598673.zip

    ESXi 6.7

    ~ esxcli software vib install -d /tmp/MLNX-NATIVE-ESX-ConnectX-4-5_4.17.13.8-10EM-670.0.0.8169922.zip

    10. Reboot the server.

    ~ reboot

    11. Verify the driver modules was installed and loaded successfully.

    esxcli software vib list | grep nmlx

    nmlx5-core                     4.17.13.8-1OEM.670.0.0.8169922        MEL       VMwareCertified   2018-06-07

    nmlx5-rdma                     4.17.13.8-1OEM.670.0.0.8169922        MEL       VMwareCertified   2018-06-07

    nmlx4-core                     3.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-06-06

    nmlx4-en                       3.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-06-06

    nmlx4-rdma                     3.17.9.12-1vmw.670.0.0.8169922        VMW       VMwareCertified   2018-06-06

    ~ esxcli system module list | grep nmlx5

    nmlx5_core                          true        true

    nmlx5_rdma                          true        true

    12. Check physical network interface status.

    ~ esxcli network nic list

    Name    PCI Device    Driver      Admin Status  Link Status  Speed   Duplex  MAC Address        MTU   Description

    ------  ------------  ----------  ------------  -----------  ------  ------  -----------------  ----  ----------------------------------------------------

    vmnic0  0000:81:00.0  igbn        Up            Up             1000  Full    0c:c4:7a:e3:5c:8c  1500  Intel Corporation I350 Gigabit Network Connection

    vmnic1  0000:81:00.1  igbn        Up            Down              0  Half    0c:c4:7a:e3:5c:8d  1500  Intel Corporation I350 Gigabit Network Connection

    vmnic2  0000:02:00.0  nmlx5_core  Up            Up           100000  Full    ec:0d:9a:8c:c7:96  1500  Mellanox Technologies MT28800 Family [ConnectX-5 Ex]

    vmnic3  0000:02:00.1  nmlx5_core  Up            Down              0  Half    ec:0d:9a:8c:c7:97  1500  Mellanox Technologies MT28800 Family [ConnectX-5 Ex]

    13. Exit Maintenance Mode the ESXi host.

     

    Done!