3 Replies Latest reply on Oct 2, 2017 12:13 AM by marlon

    Why are IB Interfaces are running at 20gbps with Unexpected actual link speed 5

    jaimesanders

      TLDR: How do I resolve this error:

      Unexpected actual link speed 5 (enable_speed1="2.5 or 5 or 10", enable_speed2="2.5 or 5 or 10" therefore final speed should be 10)

       

       

      I am not sure if I am missing a step somewhere, but  the fabric works just with very high ping times

       

      C:\> ibdiagnet

      ...

      Speed / Width checks

      -I- Link Speed Check (Compare to supported link speed)

      -E- Links Speed Check finished with errors

      -E- Link: S24be05ffffcb19e3/N24be05ffffcb19e0(ECHO2)/P1<-->Sf4521403002220f0/Nf4521403002220f0(SwitchX -  Mellanox Technologies)/P3 - Unexpected actual link speed 5 (enable_speed1="2.5 or 5 or 10", enable_speed2="2.5 or 5 or 10" therefore final speed should be 10)

      -E- Link: Sf4521403002220f0/Nf4521403002220f0(SwitchX -  Mellanox Technologies)/P1<-->S24be05ffffcbb733/N24be05ffffcbb730(ECHOHCTL)/P1 - Unexpected actual link speed 5 (enable_speed1="2.5 or 5 or 10", enable_speed2="2.5 or 5 or 10" therefore final speed should be 10)

      -E- Link: Sf4521403002220f0/Nf4521403002220f0(SwitchX -  Mellanox Technologies)/P2<-->S24be05ffffaa8853/N24be05ffffaa8850(ECHO1)/P1 - Unexpected actual link speed 5 (enable_speed1="2.5 or 5 or 10", enable_speed2="2.5 or 5 or 10" therefore final speed should be 10)

       

       

      -I- Link Width Check (Compare to supported link width)

      -I- Links Width Check finished successfully

      ---------------------------------------------

      Cable Diagnostic (Plugin)

      -I- Cable Info retrieving finished successfully

       

       

      ---------------------------------------------

      Summary

      -I- Stage                     Warnings   Errors     Comment  

      -I- Discovery                 0          0        

      -I- Lids Check                0          0        

      -I- Links Check               0          0        

      -I- Subnet Manager            0          0        

      -I- Port Counters             0          0        

      -I- Nodes Information         0          0        

      -I- Speed / Width checks      0          3        

      -I- Partition Keys            0          0        

      -I- Alias GUIDs               0          0        

      -I- Cable Diagnostic (Plugin) 0          0        

       

      ***********************************************************************************

       

      C:\> ibstat 

      CA 'ibv_device0'

              CA type:

              Number of ports: 2

              Firmware version: 2.40.5030

              Hardware version: 0x0

              Node GUID: 0x24be05ffffcbb730

              System image GUID: 0x24be05ffffcbb733

          Port 1:

              State: Active

              Physical state: LinkUp

              Rate: 20

              Base lid: 6

              LMC: 0

              SM lid: 6

              Capability mask: 0x90580000

              Port GUID: 0x24be05ffffcbb731

              Link layer: IB

              Transport: IB

          Port 2:

              State: Down

              Physical state: Polling

              Rate: 40

              Base lid: 0

              LMC: 0

              SM lid: 0

              Capability mask: 0x90580000

              Port GUID: 0x24be05ffffcbb732

              Link layer: IB

              Transport: IB

      CA 'ibv_device0'

              CA type:

              Number of ports: 2

              Firmware version: 2.40.5030

              Hardware version: 0x0

              Node GUID: 0x24be05ffffaa8850

              System image GUID: 0x24be05ffffaa8853

          Port 1:

              State: Active

              Physical state: LinkUp

              Rate: 20

              Base lid: 5

              LMC: 0

              SM lid: 6

              Capability mask: 0x90580000

              Port GUID: 0x24be05ffffaa8851

              Link layer: IB

              Transport: IB

          Port 2:

              State: Down

              Physical state: Polling

              Rate: 40

              Base lid: 0

              LMC: 0

              SM lid: 0

              Capability mask: 0x90580000

              Port GUID: 0x24be05ffffaa8852

              Link layer: IB

              Transport: IB

       

      CA 'ibv_device0'

              CA type:

              Number of ports: 2

              Firmware version: 2.40.5030

              Hardware version: 0x0

              Node GUID: 0x24be05ffffcb19e0

              System image GUID: 0x24be05ffffcb19e3

          Port 1:

              State: Active

              Physical state: LinkUp

              Rate: 20

              Base lid: 7

              LMC: 0

              SM lid: 6

              Capability mask: 0x90580000

              Port GUID: 0x24be05ffffcb19e1

              Link layer: IB

              Transport: IB

          Port 2:

              State: Down

              Physical state: Polling

              Rate: 40

              Base lid: 0

              LMC: 0

              SM lid: 0

              Capability mask: 0x90580000

              Port GUID: 0x24be05ffffcb19e2

              Link layer: IB

              Transport: IB

       

       

      C:\>ibnetdiscover

      #

      # Topology file: generated on Fri Sep 29 11:29:00 2017

      #

      # Initiated from node 24be05ffffaa8850 port 24be05ffffaa8851

       

       

      vendid=0x2c9

      devid=0xc738

      sysimgguid=0xf4521403002220f0

      switchguid=0xf4521403002220f0(f4521403002220f0)

      Switch  18 "S-f4521403002220f0"         # "SwitchX -  Mellanox Technologies" base port 0 lid 2 lmc 0

      [1]     "H-24be05ffffcbb730"[1](24be05ffffcbb731)               # "ECHOHCTL" lid 6 4xDDR

      [2]     "H-24be05ffffaa8850"[1](24be05ffffaa8851)               # "ECHO1" lid 5 4xDDR

      [3]     "H-24be05ffffcb19e0"[1](24be05ffffcb19e1)               # "ECHO2" lid 7 4xDDR

       

       

      vendid=0x2c9

      devid=0x1003

      sysimgguid=0x24be05ffffcb19e3

      caguid=0x24be05ffffcb19e0

      Ca      2 "H-24be05ffffcb19e0"          # "ECHO2"

      [1](24be05ffffcb19e1)   "S-f4521403002220f0"[3]         # lid 7 lmc 0 "SwitchX -  Mellanox Technologies" lid 2 4xDDR

       

       

      vendid=0x2c9

      devid=0x1003

      sysimgguid=0x24be05ffffcbb733

      caguid=0x24be05ffffcbb730

      Ca      2 "H-24be05ffffcbb730"          # "ECHOHCTL"

      [1](24be05ffffcbb731)   "S-f4521403002220f0"[1]         # lid 6 lmc 0 "SwitchX -  Mellanox Technologies" lid 2 4xDDR

       

       

      vendid=0x2c9

      devid=0x1003

      sysimgguid=0x24be05ffffaa8853

      caguid=0x24be05ffffaa8850

      Ca      2 "H-24be05ffffaa8850"          # "ECHO1"

      [1](24be05ffffaa8851)   "S-f4521403002220f0"[2]         # lid 5 lmc 0 "SwitchX -  Mellanox Technologies" lid 2 4xDDR

       

      C:\>ibping -L 6

      Pong from ?hostname?.?domainname? (Lid 6): time 0.214 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.118 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.101 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.099 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.122 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.102 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.132 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.104 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.125 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.111 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.099 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.093 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.111 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.092 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.108 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.115 ms

      Pong from ?hostname?.?domainname? (Lid 6): time 0.123 ms

       

       

      --- ?hostname?.?domainname? (Lid 6) ibping statistics ---

      17 packets transmitted, 17 received, 0% packet loss, time 18548 ms

      rtt min/avg/max = 0.092/0.115/0.214 ms