9 Replies Latest reply on Jul 12, 2013 3:27 PM by justinclift

    IPoIB not working on Windows 2008 r2 - need help

    opoplawski

      I'm trying for the first time to get IPoIB working on one of our Windows servers.  Details:

       

      InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA]

      Windows Server 2008 r2

      MLNX_WinOF_VPI_2_1_2_win7_x64.msi (as recommended by the mellanox download page for InfiniHost III adapters)

       

      I don't notice any errors, the adapter shows up fine and I can configure it with a static IP address.  After configuring it (or after boot) I can ping it from another machine for about 10 seconds before it stops responding.  When I ping out from the machine, the icmp packets are being sent out the main ethernet interface (which is a different IP network) and I can see them get to our router.  ibdiagnet does not report any errors.  ipconfig and netstart -r seem fine.

       

      I see the following in my opensm log:

       

      Jun 06 11:51:50 600282 [29FC1700] 0x02 -> log_notice: Reporting Generic Notice type:1 num:128 (Link state change) from LID:2 GID:fe80::1:6:6a00:d800:242
      Jun 06 11:51:51 011771 [1F5B0700] 0x02 -> osm_ucast_mgr_process: minhop tables configured on all switches
      Jun 06 11:51:51 016889 [1F5B0700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:64 (GID in service) from LID:1 GID:fe80::1:5:ad00:c:5ced
      Jun 06 11:51:51 016899 [1F5B0700] 0x02 -> state_mgr_report_new_ports: Discovered new port with GUID:0x0005ad00000c5ced LID range [16,16] of node: Topspin DDR-HCAe LX x8
      Jun 06 11:51:51 027491 [1F5B0700] 0x02 -> SUBNET UP
      Jun 06 11:51:56 333829 [213B3700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:1405:ffff::3333:0:1
      Jun 06 11:51:56 333875 [209B2700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:1405:ffff::3333:ff76:9ac6
      Jun 06 11:51:56 603980 [295C0700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:1405:ffff::3333:0:2
      Jun 06 11:51:56 604270 [245B8700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:1405:ffff::3333:0:16
      Jun 06 11:52:15 854497 [263BB700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:1405:ffff::3333:1:2
      Jun 06 11:52:15 857261 [213B3700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:1405:ffff::3333:1:3
      Jun 06 11:52:15 857968 [209B2700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:401b:ffff::16
      Jun 06 11:52:15 963577 [21DB4700] 0x02 -> log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:401b:ffff::fc
      Jun 06 12:04:56 535293 [26DBC700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:0:1 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 535870 [277BD700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:1:3 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 535908 [259BA700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:1:2 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 535942 [23BB7700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:401b:ffff::fc for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 535970 [281BE700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:401b:ffff::16 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 536014 [277BD700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:ff76:9ac6 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 536042 [209B2700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:0:16 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:04:56 536634 [227B5700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:0:2 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:06:29 959894 [295C0700] 0x01 -> mcmr_rcv_join_mgrp: ERR 1B13: validate_modify failed from port 0x0005ad00000c5ced (Topspin DDR-HCAe LX x8), sending IB_SA_MAD_STATUS_REQ_INVALID
      Jun 06 12:06:29 960518 [231B6700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:1405:ffff::3333:1:2 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:06:36 629355 [26DBC700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:401b:ffff::1 for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:06:36 629416 [259BA700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:401b:ffff::ffff:ffff for PortGID: fe80::5:ad00:c:5ced
      Jun 06 12:06:36 638659 [21DB4700] 0x01 -> mcmr_rcv_join_mgrp: ERR 1B13: validate_modify failed from port 0x0005ad00000c5ced (Topspin DDR-HCAe LX x8), sending IB_SA_MAD_STATUS_REQ_INVALID
      Jun 06 12:06:36 638853 [245B8700] 0x01 -> mcmr_rcv_leave_mgrp: ERR 1B25: Received an invalid delete request for MGID: ff12:401b:ffff::ffff:ffff for PortGID: fe80::5:ad00:c:5ced

       

      This last message repeats quite a bit within that second, and then stops.