Based on our Eth spec and our Connectx-4 HW, we do not support disabling FEC.
However, if the cable in use is below 3M, FEC will automatically be disabled.
If the Connectx-4 HCA card plugs into a switch that does not support FEC (Because our cards are set to auto-negotiate by default), if the other side supports FEC, FEC is then enabled if not, FEC is then disabled.
You can use the "ethtool" utility to enforce the 25Gbe speed.
Thank you very much for the answer. So I have no option to forcibly disable the FEC, which is bad. My application actually requires two adapters talking to each other directly and FPGA just listen the traffic using fiber taps.
So for copper cabling <3m FEC will be automatically disabled even when two adapters are talking to each other ? Can I achieve the same result on fiber using branded transceivers ?
When I set speed to 25G using ethtool seems like FEC is always enabled regardless of the equipment on the other side (I assume due to auto negotiation is disabled). Can you please confirm if my assumption is correct.
You are correct, FEC on our Connectx-4 cards can not be disabled.
Our next generation of Connectx-5 cards might have this ability though this is still work in progress.
Some Mellanox Active Fiber Cables have FEC deactivated on them (please see RN based on the FW on the card(s).
In regard to the auto-negotiation, that is the opposite. It is because the auto-negotiation is on that you have the following scenarios:
"If the Connectx-4 HCA card plugs into a switch that does not support FEC (Because our cards are set to auto-negotiate by default), if the other side supports FEC, FEC is then enabled if not, FEC is then disabled."
Have you disabled auto-negotiation in order to force the speed to 25Gbe?
With our Switches, you have the ability to disable FEC. Could can check if this feature is available with 3rd party Vendor Switches.
fec-override <fec-configuration> [force]
no fec-override <fec-configuration> [force]
Changes FEC configuration on a specific port or range of ports.
The no form of the command resets this parameter to its default value.
Syntax Description fec-configuration
• auto – auto-FEC selection
• no-fec – disables FEC
• fec-on – enables FEC
force Forces configuration (does not require toggling
interface to take effect)
• This command is supported only on Spectrum™ based switch systems
• Use this command with caution. There is no limitation in configuring nonstandard
FEC. It may cause the link to malfunction