This post describes the Dynamically-tuned Interrupt Moderation (DIM) feature and its affects on networking performance.
DIM is supported for ConnectX-4 (or newer) in Kernel 4.8 or MLNX_OFED Rel. 3.4 and later.
It is recommended that you review Understanding Interrupt Moderation before reading this post.
What is DIM?
Dynamic Interrupt Moderation (DIM) refers to any action made by hardware and/or software on run time to control interrupt rate on the system. The moderation action itself should not interfere with the system's operation and should not require any human interaction. In networking, dynamic interrupt moderation is used for controlling the rate of interrupts generated by the hardware for multiple traffic scenarios.
How to Verify that DIM is Working
Run the following command and check that Adaptive Rx is on. DIM is not supported for the Tx direction. RX DIM is on by default (for Kernel 4.8 or MLNX_OFED Rel. 3.4 or later).
# ethtool -c p5p1
Coalesce parameters for p5p1:
Adaptive RX: on TX: off
DIM is enabled by default. In case you wish to disable it (not recommended), run:
ethtool -C <interface> adaptive-rx off
To re-enabled it again run:
ethtool -C <interface> adaptive-rx on
The Benefits of DIM
DIM allows Mellanox's network adapters to self-configure their interrupt moderation parameters in order to achieve the best performance. DIM is able to identify and profile the networking traffic rapidly and set the network adapter's interrupt moderation parameters for optimized values. For high throughput scenarios this means maximum throughput with minimum interrupts. For latency scenarios this means minimum latency. Since the DIM algorithm operates on each channel independently, it can even identify mixed traffic scenarios (such as latency under load) and optimize each channel according to the traffic it handles. In this way, DIM assures maximum performance with minimum interrupts, which can translate into reduced operation costs or, alternatively, more processing power.
The graphs below show the affect of DIM on throughput, interrupt rate, and CPU utilization. As you can observe, there is no degradation in throughput even when loading the system with many TCP streams. Note that while overall interrupt rate and CPU utilization do increase, they do not do so at a dramatic rate. Even when fully utilizing the NIC, there is still a great deal of processing power available in the system.