2 Replies Latest reply on Feb 24, 2018 8:50 AM by justinclift

    Ethernet mode non-functional with recent CentOS7 kernels and ConnectX-2 cards?

    justinclift

      Hi all,

       

      Is anyone else around running CentOS 7 x64 and ConnectX (1/2/3) series cards in eth mode?

       

      Asking because after rebooting my CentOS 7 x64 (1708) desktop this evening after a yum update,

      the Mellanox ConnectX-2 card in it (set to run in ethernet mode) refused to come up correctly.

      That was with the latest kernel (installed yesterday) of kernel-3.10.0-693.17.1.

       

      Instead, it came up in Infiniband mode, with "ip addr" complaining about a potential bad address.

       

      Had a feeling it might be kernel related (something unforeseen from recent Meldown/Spectre

      patches maybe?), so tried the previous kernels to see if that's the cause. Short answer: Yep.

       

      My desktop has these kernels installed at the moment:

       

      • kernel-3.10.0-693.17.1.el7.x86_64 | ConnectX-2 card not working
      • kernel-3.10.0-693.11.6.el7.x86_64 | ConnectX-2 card not working
      • kernel-3.10.0-693.11.1.el7.x86_64 | ConnectX-2 card works
      • kernel-3.10.0-693.el7.x86_64 | ConnectX-2 card works

       

      So, kernel-3.10.0-693.11.6 and onwards are "busted" from this point of view. Reverting to either

      of the older two kernels and the card comes up fine, working as 10GbE as expected.

       

      For reference, this is using the CentOS 7 x64 Inbox driver.

       

      The address of the card in my desktop:

       

           $ lspci | grep Mellanox

           06:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

       

      And this is the entry for it in /etc/rdma/mlx4.conf telling it to operate in ethernet mode (it's a single port card):

       

           $ tail -2 /etc/rdma/mlx4.conf

           #

           0000:06:00.0 eth

       

      Note - Filed a bug report on bugs.centos.org, and will likely file one on the RedHat Bugzilla too for good

      measure.  But figured people here would probably be interested/impacted too.