Can you confirm what type of HCA card(s) installed within the ESXI server?
What is the port protocol configured as IB or Ethernet?
I have a similar problem: i cant get IP over IB working.
MT 27500 ConnectX-3 (PSID MT_1100120019) on ESX 6 (latest Updates 2 days ago). In the BIOS (SuperMicro) I set in the PCIe Configuration Page “Above 4G Decoding” as “Disabled” since otherwise the card was not recognized in older SuperMicro hardware.
First installed the 22.214.171.124 and also updated the firmware to 5150
Then had a vmnic2 with up to 40000, but link was down.
Then I uninstalled and installed OFED 126.96.36.199
Then had a vmnic2 with up to 1000GB, link down.
I also tried to install both drivers, same issue: link down, card is recognized as Ethernet and not IPoIB.
I already opened a support ticket with VMWare but the technician had no solution.
It seems the last know driver to work is 188.8.131.52 (see here VSphere 6.0 ) but this driver is not supported in 6.x
By the way: when you select now OFED 184.108.40.206. you are redirected to 220.127.116.11 for download (http://www.mellanox.com/page/products_dyn?product_family=36&mtag=vmware_drivers )
I had no problem with the same cards in similar (older) hardware and ESX 5.5.
1) Driver version 2.4.0 is an Ethernet driver only so if you have a VPI card, please make sure the port protocol is configured as 2 which is Ethernet.
#/opt/mellanox/bin/mlxconfig -d /dev/mt4099_pciconf0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2
2) Enable SRIOV with the MFT tool (Mellanox FW tool) as follow:
#/opt/mellanox/bin/mlxconfig -d /dev/mt4099_pciconf0 set SRIOV_EN=1 NUM_OF_VFS=16
3) Confirm BIOS setting supports SRIOV and/or virtualization.
4) Set the driver module parameters for VF:
#esxcli system module parameters set -m mlx4_core -p 'num_vfs=<VFs over Port1, VFs over port2, 0> port_type_array=2'
This is all documented in the driver UM.
we don't want to use it as Ethernet card but as IB over IP. That's the problem that I can't get it to work as a 40GB IBoIP card in ESX 6 anymore, which worked perfectly with ESX 5.5 and the 18.104.22.168 drivers (which are not supported anymore).
Would you please also link to the driver UM you are referring too?
Driver 2.4.0 include IPoIB driver and works!
Absolutlely! driver 2.4.0 located in ethernet driver section, but why this driver 2.4.0 include IPoIB driver & works?