This post highlights the features planned for the upcoming MLNX_OFED 4.1 release.
- Download Driver
- RDMA and RoCE
- Time Synchronization
Download the MLNX_OFED driver from Mellanox web page, click here.
Probe VFs (mlx5): Sometimes it is not desirable to probe the virtual functions after SR-IOV is enabled. This can save host side resource usage by VF instances which would be eventually probed to the VMs. It might be very inconvenient if there are a lot of VFs created and probed in the hypervisor. A new mlx5_core module parameter called probe_vf was added to provide this option.
For more info see HowTo Configure and Probe VFs on mlx5 Drivers
ECN Congestion Counters
1. Four ECN/CNP Congestion counters were added to mlx5 driver in this release, for more info see Understanding mlx5 Linux Counters and Status Parameters .
Understanding the reason for Link Down
2. The reason for Link DOWN was added to mlx5 driver. link_down_reason debug status was added, for more info see Understanding mlx5 Linux Counters and Status Parameters .
RDMA Diagnostic Counters
Various RDMA Diagnostic counters where added to the following path:
For more info, see Understanding mlx5 Linux Counters and Status Parameters
One of the features in ethtool is rx-fcs. Normally the FCS of the packet will be truncated by the ASIC hardware before sending it to the application socket buffer (skb). Ethtool allows to set the rx-fcs not to be truncated, but to pass it to the application for analysis. For more info and usage see Understanding ethtool rx-fcs for mlx5 Drivers .
RDMA and RoCE
This release adds the support for "trust DSCP", the user have the ability to enabled PFC based on the DSCP value. With this solutions there is no need to use VLAN header (it is possible but not mandatory).
To learn more about trust mode and configuration example, see HowTo Configure Trust Mode on Mellanox Adapters .
ECN parameters moved to different location. All the parameters are now located in
In addition, few parameters were removed, and one parameter was added. For more info, se HowTo Configure DCQCN (RoCE CC) for ConnectX-4 (Linux)
RDMA CM default RoCE mode
Starting with MLNX_OFED 4.1 the RDMA CM will use by default RoCEv2 for the supported adapters (ConnectX-3 Pro or later). When testing RDMA CM, it is recommended to update the client and server to MLNX_OFED 4.1
For more info, see RDMA CM and RoCE Version Defaults.
NVMe-oF Target offload
NVMe-oF target offload is supported in ConnectX-5 adapters, to enable target offload, see HowTo Configure NVMe-oF Target Offload.
New experimental verb ibv_exp_open_device added to OFED 4.1 (ConnectX-4 adapters). The following verb adds the ability to open a device and create a context while giving PCI peer attributes (in ibv_exp_open_device_attr) such as name and ID. For more info and coding examples, see PeerDirect Programming : Open Device and Create Context with Peer Attributes
PTP over IPoIB
Add support for precision time protocol (PTP) over IPoIB. PTP helps to align the HCA time to an external clock. The master distributes the time over the network encapsulated in packets. The synchronization is based on symmetric Round Trip Time (RTT) between the master and the slave. The feature is supported by offloaded devices and is enabled by default, and used similar to PTP over Ethernet. For more information, see Running Linux PTP with ConnectX-4.
IPoIB Interfaced improved and optimized with the following hardware features for ConnectX-4.
- Stateless offloads (LRO, aRFS, RSS, TSS)
- Multi-queue support, all queues share the same transport.
- Interrupt moderation
- Multi partitions optimizations
- Share send/recv Work Queues
- Vendor specific optimizations
- Work queue processing
- The IPoIB ULP was leveraged while the support of the “Legacy” mode remains.
- Support UD mode only
A Simple way to verify the driver is using the new mode by shell command: “ip link show ibX” the driver mac will start with 00:xxxxxxx
No new configuration added, all enabled by default and done internally within the IPoIB driver.
One Pulse Per Second (1PPS) is a time synchronization feature that allows the adapter to be able to send or receive 1 pulse per second on a dedicated pin on the adapter card.
The supported ASICs are ConnectX-4 or later. Only one pin is supported and could be configured as 1PPS in or 1PPS out. For testing, special adapter should be used (with access to the 1PPS pin), contact Mellanox support for that.
To learn how to test it see HowTo Test 1PPS on Mellanox Adapters.
Additional features can be found in the MLNX_OFED Release Notes.