How To Install Mellanox Firmware Tools (MFT) on VMware ESXi 6.5/6.7

Version 12

    This post describes the procedure of how to install and run Mellanox Firmware Tools (MFT) v4.9.0 on VMware ESXi 6.5/6.7 versions.

    References

     

    Overview

    Mellanox ConnectX®-4/ConnectX-5 NATIVE ESXi is a software stack which operates across all Mellanox network adapter solutions supporting up to 100Gb/s Ethernet (ETH) and 2.5 or 5.0 GT/s PCI Express 2.0 and 3.0 uplinks to servers.

     

    Hardware and Software Requirements

    1. A server platform with an adapter card based on one of the following Mellanox Technologies’ HCA devices:

    2. Installer Privileges: The installation requires administrator privileges on the target machine.

    3. Device ID: For the latest list of device IDs, please visit Mellanox website.

     

    Mellanox Firmware Tools (MFT)

    The Mellanox Firmware Tools (MFT) package is a set of firmware management tools used to:

    • Generate a standard or customized Mellanox firmware image
    • Querying for firmware information
    • Burn a firmware image

     

    Setup

    The setup includes ESXi 6.5/6.7 server installed with ConnectX-4/5 adapter card.

     

    Installation

    1. Go to the MFT web page: http://www.mellanox.com/page/management_tools.

     

    1. Enable SSH Access to ESXi server.

    2. Log into ESXi vSphere Command-Line Interface with root permissions.

    3. Download the MFT for VMware vib package to the "/tmp" directory.

    4.  Verify that the files are placed in the /tmp directory.

    # cd /tmp
    # ls

    mft-4.9.0.38-10EM-650.0.0.4598673.x86_64.vib      nmst-4.9.0.38-1OEM.650.0.0.4598673.x86_64.vib

    5. Get the adapter list

    # esxcli network nic list

    Name   PCI Device    Driver      Admin Status   Link Status   Speed   Duplex   MAC Address        MTU   Description

    ------ ------------  ----------  ------------   -----------   ------  ------   -----------------  ----  ----------------------------------------------------

    vmnic0 0000:81:00.0  igbn        Up             Up            1000    Full     0c:c4:7a:e3:5c:20  1500  Intel Corporation I350 Gigabit Network Connection

    vmnic1 0000:81:00.1  igbn        Up             Down          0       Half     0c:c4:7a:e3:5c:21  1500  Intel Corporation I350 Gigabit Network Connection

    vmnic2 0000:02:00.0  nmlx5_core  Up             Up            100000  Full     ec:0d:9a:8a:27:8a  1500  Mellanox Technologies MT28800 Family [ConnectX-5 Ex]

    vmnic3 0000:02:00.1  nmlx5_core  Up             Down          0       Half     ec:0d:9a:8a:27:8b  1500  Mellanox Technologies MT28800 Family [ConnectX-5 Ex]

     

    # esxcli network nic get -n vmnic2

      Advertised Auto Negotiation: true

      Advertised Link Modes: Auto, 100000BaseCR4/Full

      Auto Negotiation: false

      Cable Type: DA

      Current Message Level: -1

      Driver Info:

    Bus Info: 0000:02:00:0

    Driver: nmlx5_core

    Firmware Version: 16.21.1000

    Version: 4.17.9.12

      Link Detected: true

      Link Status: Up

      Name: vmnic2

      PHYAddress: 0

      Pause Autonegotiate: false

      Pause RX: true

      Pause TX: true

      Supported Ports:

      Supports Auto Negotiation: true

      Supports Pause: true

      Supports Wakeon: false

      Transceiver:

      Virtual Address: 00:50:56:53:ee:ca

      Wakeon: None

    6. Install the vib package.

     

    # esxcli software vib install -v /tmp/nmst-4.9.0.38-1OEM.650.0.0.4598673.x86_64.vib

    Installation Result

       Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

       Reboot Required: true

       VIBs Installed: Mellanox_bootbank_net-mst_4.9.0.38-1OEM.650.0.0.4598673

       VIBs Removed:

       VIBs Skipped:

     

    # esxcli software vib install -v /tmp/mft-4.9.0.38-10EM-650.0.0.4598673.x86_64.vib

    Installation Result

       Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.

       Reboot Required: true

       VIBs Installed: Mellanox_bootbank_mft_4.9.0.38-0

       VIBs Removed:

       VIBs Skipped:

    The MFT tools are not located in the default path. In order to run any MFT tool either:

    • Enter the full path. For example: /opt/mellanox/bin/flint

    OR

    • Add MFT path to the default system path by running: export PATH=$PATH:/opt/mellanox/bin.

    Please note, the path is temporary and will hold only until reboot.

    8. Enter Maintenance Mode the ESXi host.

    9. Reboot the server.

    # reboot

    10. Exit Maintenance Mode the ESXi host.

    11. Start the mst driver. Run:

    # cd /opt/mellanox/bin

    #./mst start

    Module mst loaded successfully

    12.To print the current status of Mellanox devices.

    /opt/mellanox/bin/mst status

    MST devices:

    ------------

    mt4121_pciconf0

    13. Show the devices status with detailed information.

    /opt/mellanox/bin/mst status -vv

    PCI devices:

    ------------

    DEVICE_TYPE           MST                  PCI           RDMA          NET                NUMA 

    ConnectX5(rev:0)      mt4121_pciconf0      02:00.0                     net-vmnic2 

    ConnectX5(rev:0)      mt4121_pciconf0.1    02:00.1                     net-vmnic3

    14. Query the device.

    #./mlxfwmanager --query

    Querying Mellanox devices firmware ...

     

    Device #1:

    ----------

        Device Type: ConnectX5

      Part Number: MCX556A-EDA_Ax

      Description: ConnectX-5 Ex VPI adapter card; EDR IB (100Gb/s) and 100GbE; dual-port QSFP28; PCIe4.0 x16; tall bracket; ROHS R6

      PSID: MT_0000000009

      PCI Device Name: mt4121_pciconf0

      Base MAC: ec0d9a8a278a

      Versions: Current Available 

      FW 16.21.1000 N/A 

      PXE 3.5.0305 N/A

     

    Done !

    Appendix A

    mst Synopsis

    mst  [switches]

    Commands and Switches Description

    mst start                               Create special files that represent Mellanox devices in directory/dev. Load appropriate modules. After successfully completing this command, the mst driver will be ready to work.

     

    mst stop                                Stop Mellanox mst driver service and unload the kernel modules.

     

    mst restart                             "mst stop" followed by "mst start"

     

    mst server start [-p|--port port]       Start mst server to allow incoming connection.

                                            Default port is 23108.

     

    mst server stop                         Stop the mst server.

     

    mst status                              Print current status of Mellanox devices

                                            Options: -v run with a high verbosity level (print more info on each device)

     

    mst version                             Print the version info