What's New in the MLNX_OFED 4.1 Release

Version 27

    This post highlights the features planned for the upcoming MLNX_OFED 4.1 release.

     

     

    References

     

    Download Driver

    Download the MLNX_OFED driver from Mellanox web page, click here.

     

    Tools

    mlx_fs_dump tool is introduced in this release. This tool dump the steering rules in a readable easy-to-understand manner. For basic usage and example see mlx_fs_dump page.

     

    Virtualization

    Probe VFs (mlx5): Sometimes it is not desirable to probe the virtual functions after SR-IOV is enabled. This can save host side resource usage by VF instances which would be eventually probed to the VMs. It might be very inconvenient if there are a lot of VFs created and probed in the hypervisor. A new mlx5_core module parameter called probe_vf was added to provide this option.

    For more info see HowTo Configure and Probe VFs on mlx5 Drivers

     

    Monitoring

    ECN Congestion Counters

    1. Four ECN/CNP Congestion counters were added to mlx5 driver in this release, for more info see Understanding mlx5 Linux Counters and Status Parameters .

     

    2. The reason for Link DOWN was added to mlx5 driver. link_down_reason debug status was added, for more info see Understanding mlx5 Linux Counters and Status Parameters .

     

    RDMA Diagnostic Counters

    Various RDMA Diagnostic counters where added to the following path:

    ls /sys/class/infiniband/mlx5_0/ports/1/hw_counters/

    For more info, see Understanding mlx5 Linux Counters and Status Parameters

     

    Ethtool

    One of the features in ethtool is rx-fcs.  Normally the FCS of the packet will be truncated by the ASIC hardware before sending it to the application socket buffer (skb). Ethtool allows to set the rx-fcs not to be truncated, but to pass it to the application for analysis. For more info and usage see Understanding ethtool rx-fcs for mlx5 Drivers .

     

    RDMA and RoCE

     

    Trust DSCP

    This release adds the support for "trust DSCP", the user have the ability to enabled PFC based on the DSCP value. With this solutions there is no need to use VLAN header (it is possible but not mandatory).

    To learn more about trust mode and configuration example, see HowTo Configure Trust Mode on Mellanox Adapters .

     

    ECN Parameters

    ECN parameters moved to different location. All the parameters are now located in

    /sys/kernel/debug/mlx5/<PCI BUS>/cc_params/

    In addition, few parameters were removed, and one parameter was added. For more info, se HowTo Configure DCQCN (RoCE CC) for ConnectX-4 (Linux)

     

    RDMA CM default RoCE mode

    Starting with MLNX_OFED 4.1 the RDMA CM will use by default RoCEv2 for the supported adapters (ConnectX-3 Pro or later). When testing RDMA CM, it is recommended to update the client and server to MLNX_OFED 4.1

    For more info, see RDMA CM and RoCE Version Defaults.

     

    Storage

    NVMe-oF Target offload

    NVMe-oF target offload is supported in ConnectX-5 adapters, to enable target offload, see HowTo Configure NVMe-oF Target Offload.

     

    Programming

    PeerDirect Programming

    New experimental verb ibv_exp_open_device added to OFED 4.1 (ConnectX-4 adapters). The following verb adds the ability to open a device and create a context while giving PCI peer attributes (in ibv_exp_open_device_attr)  such as name and ID. For more info and coding examples, see PeerDirect Programming : Open Device and Create Context with Peer Attributes

     

    InfiniBand

    PTP over IPoIB

    Add support for precision time protocol (PTP) over IPoIB. PTP helps to align the HCA time to an external clock. The master distributes the time over the network encapsulated in packets. The synchronization is based on symmetric Round Trip Time (RTT) between the master and the slave. The feature is supported by offloaded devices and is enabled by default, and used similar to PTP over Ethernet. For more information, see Running Linux PTP with ConnectX-4.

     

    IPoIB Improvements

    IPoIB Interfaced improved and optimized with the following hardware features for ConnectX-4.

    • Stateless offloads (LRO, aRFS, RSS, TSS)
    • Multi-queue support, all queues share the same transport.
    • Interrupt moderation
    • Multi partitions optimizations
    • Share send/recv Work Queues
    • Vendor specific optimizations
    • Work queue processing
    • The IPoIB ULP was leveraged while the support of the “Legacy” mode remains.
    • Support UD mode only

     

    A Simple way to verify the driver is using the new mode by shell command: “ip link show ibX” the driver mac will start with 00:xxxxxxx

     

    No new configuration added, all enabled by default and done internally within the IPoIB driver.

     

    Time Synchronization

    1PPS

    One Pulse Per Second (1PPS) is a time synchronization feature that allows the adapter to be able to send or receive 1 pulse per second on a dedicated pin on the adapter card.

    The supported ASICs are ConnectX-4 or later. Only one pin is supported and could be configured as 1PPS in or 1PPS out. For testing, special adapter should be used (with access to the 1PPS pin), contact Mellanox support for that.

    To learn how to test it see HowTo Test 1PPS on Mellanox Adapters.

     

    Additional features can be found in the MLNX_OFED Release Notes.