What's New in MLNX_OFED v4.1 Release

Version 3

    This post highlights the features of MLNX_OFED v4.1.


    Download Driver

    Download the MLNX_OFED driver from Mellanox web page, click here.



    mlx_fs_dump tool is introduced in this release. This is a python tool that prints the steering rules in a readable manner. For basic usage and examples, seemlx_fs_dump.


    Probed VFs

    Probed VFs (mlx5): Sometimes it is not desirable to probe the virtual functions after SR-IOV is enabled. This can save host side resource usage by VF instances, which would eventually be probed to the VMs. It might be very inconvenient if there are a lot of VFs created and probed in the hypervisor. A new mlx5_core module parameter called probe_vf was added to provide this option.For more information, see HowTo Configure and Probe VFs on mlx5 Drivers.


    ECN Congestion Counters

    Four ECN/CNP Congestion counters were added to mlx5 driver in this release. For more information, see Understanding mlx5 Linux Counters and Status Parameters.

    The reason for Link DOWN was added to mlx5 drive. link_down_reason debug status was added. For more information, see Understanding mlx5 Linux Counters and Status Parameters.

    RDMA Diagnostic Counters

    Various RDMA Diagnostic counters were added to the following path:ls /sys/class/infiniband/mlx5_0/ports/1/hw_counters/For more information, see Understanding mlx5 Linux Counters and Status Parameters


    One of the features in ethtool is rx-fcs. Normally, the FCS of the packet will be truncated by the ASIC hardware before sending it to the application socket buffer (skb). Ethtool allows setting the rx-fcs not to be truncated, but to pass it to the application for analysis. For more information,see Understanding ethtool rx-fcs for mlx5 Drivers.

    RDMA and RoCE

    DSCP Trust

    This release supports DSCP in Trust mode. Users now have the ability to enable PFC based on the DSCP value. With this solution, VLAN header usage is no longer mandatory.To learn more about DSCP Trust mode and for a configuration example, see HowTo Configure Trust Mode on Mellanox Adapters.

    ECN Parameters

    ECN parameters were moved to a different location. All the parameters are now located in:/sys/kernel/debug/mlx5/<PCI BUS>/cc_params/

    RDMA CM Default RoCE Mode

    Starting with MLNX_OFED 4.1, the RDMA CM uses RoCEv2 for the supported adapters (ConnectX-3 Pro or later)  by default. When testing RDMA CM, it is recommended to update the client and server to MLNX_OFED 4.1.For more information, see RDMA CM and RoCE Version Defaults.


    NVMe-oF Target offload

    NVMe-oF target offload is supported in ConnectX-5 adapters. To enable it, see HowTo Configure NVMe-oF Target Offload.For testing your storage server and displaying the gained utilization, see Simple NVMe-oF Target Offload Benchmark.


    PeerDirect Programming

    New experimental verb "ibv_exp_open_device" was added to OFED v4.1 (on ConnectX-4 adapters). This verb adds the ability to open a device and create a context while giving the PCI peer attributes (in ibv_exp_open_device_attr) such as name and ID. For more information and coding examples, see PeerDirect Programming: Create Context with Peer Attributes.


    PTP over IPoIB

    Added support for PTP feature in IPoIB offloaded devices. This feature allows for accurate synchronization between the distributed entities over the network. The synchronization is based on symmetric Round TripTime (RTT) between the master and slave devices. The feature is enabled by default.For more information, see Running Linux PTP with ConnectX-4/ConnectX-5.

    IPoIB Improvements

    IPoIB interface was improved and optimized with the following hardware features for ConnectX-4 adapter cards:

    • Stateless offloads (LRO, aRFS, RSS, TSS)
    • Multi-queue support. All queues share the same transport
    • Interrupt moderation
    • Multi partitions optimizations
    • Share send/recv Work Queues
    • Vendor specific optimizations
    • Work queue processing
    • The IPoIB ULP was leveraged while the support of the “Legacy” mode remains.
    • Support UD mode only

    Time Synchronization