0 Replies Latest reply on Jul 18, 2018 6:45 AM by stdenis

    The problem with RoCE connectivity between ConnectX-3 and ConnectX-4 Lx adapters

    stdenis

      Hello.

      I have Microsoft Windows 2012 R2 cluster and some nodes have ConnectX-3 adapters and some nodes have ConnectX-4 Lx adapters.

      There is RoCE connectivity between nodes with ConnectX-4 Lx adapters, but there isn’t connectivity between nodes with different adapters.

      I think it’s because ConnectX-3 adapters use RoCE  1.0 mode, but ConnectX-4 Lx adapters use RoCE 2.0 mode.

      I tred to change RoCE mode from 1.0 to 2.0 for ConnectX-3 adapters by “Set-MlnxDriverCoreSetting -RoceMode 2”, but had warning – “SingleFunc_2_0_0: RoCE v2.0 mode was requested, but it is not supported. The NIC starts in RoCE v1.5 mode” and RoCE connectivity doesn’t work.

      What the best way to fix my problem ?

      ConnectX-3 adapters don’t work in RoCE 2.0 mode at all ?    How about new FW ? Today I have WinOF-5_35 with FW:

       

      Image type:      FS2

      FW Version:      2.40.5032

      FW Release Date: 16.1.2017

      Product Version: 02.40.50.32

      Rom Info:        type=PXE version=3.4.747 devid=4099

      Device ID:       4099

      Description:     Node             Port1            Port2            Sys image

      GUIDs:           ffffffffffffffff ffffffffffffffff ffffffffffffffff ffffffffffffffff

      MACs: e41d2ddfa540     e41d2ddfa541

      VSD:

      PSID:            MT_1080120023

       

       

      I don’t find way to change to RoCE 1.0 mode for ConnectX-4 Lx adapters for Microsoft Windows environment.