This post highlights the features of MLNX_OFED v4.1.
- Download Driver
- RDMA and RoCE
- Time Synchronization
Download the MLNX_OFED driver from Mellanox web page, click here.
Probed VFs (mlx5): Sometimes it is not desirable to probe the virtual functions after SR-IOV is enabled. This can save host side resource usage by VF instances, which would eventually be probed to the VMs. It might be very inconvenient if there are a lot of VFs created and probed in the hypervisor. A new mlx5_core module parameter called probe_vf was added to provide this option.
For more information, see HowTo Configure and Probe VFs on mlx5 Drivers.
ECN Congestion Counters
Four ECN/CNP Congestion counters were added to mlx5 driver in this release. For more information, see Understanding mlx5 Linux Counters and Status Parameters.
Understanding the reason for Link Down
The reason for Link DOWN was added to mlx5 drive. link_down_reason debug status was added. For more information, see Understanding mlx5 Linux Counters and Status Parameters.
RDMA Diagnostic Counters
Various RDMA Diagnostic counters were added to the following path:
For more information, see Understanding mlx5 Linux Counters and Status Parameters
One of the features in ethtool is rx-fcs. Normally, the FCS of the packet will be truncated by the ASIC hardware before sending it to the application socket buffer (skb). Ethtool allows setting the rx-fcs not to be truncated, but to pass it to the application for analysis. For more information,see Understanding ethtool rx-fcs for mlx5 Drivers.
RDMA and RoCE
This release supports DSCP in Trust mode. Users now have the ability to enable PFC based on the DSCP value. With this solution, VLAN header usage is no longer mandatory.
To learn more about DSCP Trust mode and for a configuration example, see HowTo Configure Trust Mode on Mellanox Adapters.
ECN parameters were moved to a different location. All the parameters are now located in:
In addition, few parameters were removed, and one parameter was added. For more information, see HowTo Configure DCQCN (RoCE CC) for ConnectX-4 (Linux).
RDMA CM Default RoCE Mode
Starting with MLNX_OFED 4.1, the RDMA CM uses RoCEv2 for the supported adapters (ConnectX-3 Pro or later) by default. When testing RDMA CM, it is recommended to update the client and server to MLNX_OFED 4.1.
For more information, see RDMA CM and RoCE Version Defaults.
NVMe-oF Target offload
NVMe-oF target offload is supported in ConnectX-5 adapters. To enable it, see HowTo Configure NVMe-oF Target Offload.
For testing your storage server and displaying the gained utilization, see Simple NVMe-oF Target Offload Benchmark.
New experimental verb "ibv_exp_open_device" was added to OFED v4.1 (on ConnectX-4 adapters). This verb adds the ability to open a device and create a context while giving the PCI peer attributes (in ibv_exp_open_device_attr) such as name and ID. For more information and coding examples, see PeerDirect Programming: Create Context with Peer Attributes.
PTP over IPoIB
Added support for PTP feature in IPoIB offloaded devices. This feature allows for accurate synchronization between the distributed entities over the network. The synchronization is based on symmetric Round Trip
Time (RTT) between the master and slave devices. The feature is enabled by default.
For more information, see Running Linux PTP with ConnectX-4/ConnectX-5.
IPoIB interface was improved and optimized with the following hardware features for ConnectX-4 adapter cards:
- Stateless offloads (LRO, aRFS, RSS, TSS)
- Multi-queue support. All queues share the same transport
- Interrupt moderation
- Multi partitions optimizations
- Share send/recv Work Queues
- Vendor specific optimizations
- Work queue processing
- The IPoIB ULP was leveraged while the support of the “Legacy” mode remains.
- Support UD mode only
A simple way to verify that the driver is using the new mode by shell command: “ip link show ibX” the driver mac will start with 00:xxxxxxx
No new configuration added, all enabled by default and done internally within the IPoIB driver.
One Pulse Per Second (1PPS) is a time synchronization feature that allows the adapter to be able to send or receive 1 pulse per second on a dedicated pin on the adapter card.
The supported ASICs are ConnectX-4 or later. Only one pin is supported and could be configured as 1PPS in or 1PPS out. For testing, a special adapter should be used (with access to the 1PPS pin) - contact Mellanox support.
To learn how to test it, see HowTo Test 1PPS on Mellanox Adapters.
Additional information on the content of MLNX_OFED release can be found in MLNX_OFED Release Notes document available on Mellanox official website.