2 Replies Latest reply on Dec 28, 2015 12:40 AM by swartz

    Extremely slow ethernet speed with ConnectX VPI card in EN mode.


      I picked up a pair of ConnectX VPI cards along with CX4->SPF+ cable as a cost-effective way to get 10gigE for shared storage.


      I initially plugged them into an switch (gigE switch with 4 10gigE uplink ports). To eliminate switch or the CX4->SFP+ cable as the cause of the problem, I took the two systems with same ConnectX cards and connected the two cards directly with a passive CX4 cable.


      With this direct card-to-card connection and tested via iperf I get these results.

      IPoIB:  Around 6-7Gbit/s with datagram mode and default 2044 MTU. Switching to connected mode and setting MTU to 65520 doesn't make any difference.

      10gig Ethernet: I'm lucky to get 200-300 Kbit/sec.


      Card firmware is latest available for these cards v2.9.1000.


      Both systems are Debian Wheezy 7.8 (well, Proxmox)


      Adapters are identified as:

      04:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX VPI PCIe 2.0 2.5GT/s - IB DDR / 10GigE] (rev a0)


      Module versions:

      mlx4_core: Mellanox ConnectX core driver v2.2-1 (Feb, 2014)

      mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.2-1 (Feb 2014)


      Possible causes? Solutions?