I have several HCA's and SX6036G gateway systems.
Here is some test result.
FDR14 switch port mode + QDR 40Gb HCA -> 40Gb IB link-up
FDR14 switch port mode + FDR14 56Gb HCA -> 56Gb IB link-up
ETH 56Gb switch port mode + 56Gb ETH mode with modified firmware -> 56Gb ETH link-up
ETH 56Gb switch port mode + 40Gb ETH mode with un-modified firmware -> 40Gb ETH link-up
ETH 56Gb switch port mode + any 40Gb ETH Adapter -> 40Gb ETH link-up
I'm heard Mellanox doesn't limit 3rd party Adapter connection to Mellanox switches.
But you mest check compatibility about cable, HCA, Ethernet adapter list on MLNX-OS User Manuals.