I have 2 Mellanox CX516A in same x86 host. I successfully use a Finisar QSFP28 100G LR module plugged into each NIC and back-to-back cabled. Means NIC 1 port 0 goes to NIC 2 port 0, this is working fine.
I tried to place both QSFP28 modules into the same NIC and use back-to-back cabling. After I plugged in the second QSFP28 into port 1 I got the following error message:
[419335.854822] mlx5_core 0000:88:00.1: Port module event[error]: module 1, Cable error, Power budget exceeded
also the orange led is flashing.
My host running Ubuntu 18.04 with "MLNX_OFED_LINUX-4.3-22.214.171.124-ubuntu18.04-x86_64" driver version.
Do you have any hints how to bringup both QSFP28 in the same NIC ?
sudo mlxconfig -e -d 04:00.0 set ADVANCED_POWER_SETTINGS=True
sudo mlxconfig -e -d 04:00.0 set DISABLE_SLOT_POWER_LIMITER=True