13 Replies Latest reply on Nov 18, 2015 12:17 PM by cmm

    sx1012 link issue

    prieto

      Hi,

       

      So I have two new sx1012 connected to each other with fiber from two different buildings. They both have the same transceivers:

       

      [standalone: master] # show interfaces ethernet transceiver

      Port 1/1 state

              identifier                           : QSFP+

              cable/ module type           : Optical cable/ module - LR4

              ethernet speed and type  : 40GBASE - LR4

              vendor                              : OPNEXT, INC.

              supported cable length     : 10000m SMF

              part number                      : TRQ5B20MN-LF100W

              revision                              : X1

              serial number                    : Z13J00069

       

      [standalone: master] # show interfaces ethernet 1/12

      Eth1/12

        Admin state: Enabled

        Operational state: Down

        Description: N\A

        Mac address: e4:1d:2d:39:3b:54

        MTU: 1500 bytes(Maximum packet size 1522 bytes)

        Flow-control: receive off send off

        Actual speed: 40 Gbps

        Width reduction mode: disabled

        Switchport mode: access

        MAC learning mode: Enabled

        Last clearing of "show interface" counters : Never

        60 seconds ingress rate: 0 bits/sec, 0 bytes/sec, 0 packets/sec

        60 seconds egress rate: 0 bits/sec, 0 bytes/sec, 0 packets/sec

      Rx

        0                    packets

        0                    unicast packets

        0                    multicast packets

        0                    broadcast packets

        0                    bytes

        0                    error packets

        0                    discard packets

      Tx

        0                    packets

        0                    unicast packets

        0                    multicast packets

        0                    broadcast packets

        0                    bytes

        0                    discard packets

       

      We have measured the optical power levels on the fibers and check out fine but I don't get a connection. The port "operational state" shows as "Down" and the LED light doesn't come on. I'm wondering if that transceiver is compatible even though it recognizes it?

       

      Any help or suggestions would be greatly appreciate to troubleshoot that issue.

      Thank you,

      Daniel

        • Re: sx1012 link issue
          ophirmaor

          what is the length of the fiber?

          • Re: sx1012 link issue
            se-rulez

            We dont support third party LR4 or SR4 modules. That doesn't mean we do anything to prevent it from working but it may not work. If you provide us the length of the fiber in between, we can suggest a Mellanox optical module.

            • Re: sx1012 link issue
              prieto

              Addition question about that connection: Is there some mode that I need to configure on the port to be aware that it is a Ethernet connection and not IB?

               

              Thank you,

              Daniel

                • Re: sx1012 link issue
                  ophirmaor

                  You need to configure the port type to be ethernet or InfiniBand.

                   

                  See here, step 7.

                   

                  HowTo Configure InfiniBand Gateway HA (Proxy ARP)

                    • Re: sx1012 link issue
                      prieto

                      Ophir,

                       

                      That command from step 7 "show ports type" or "port..." is not recognized on my sx1012. I ran some of the other ones that are mentioned in that link that work:

                       

                      [standalone: master] (config) # show system capabilities

                      Ethernet: Supported, Full L2, L3

                      Ethernet Max licensed speed: 40Gb

                       

                      [standalone: master] (config) # show system profile

                      eth-single-switch

                       

                      [standalone: master] (config) # show port

                      % Unrecognized command "port".

                      Type "show ?" for help.

                       

                      [standalone: master] (config) # show ports

                      % Unrecognized command "ports".

                      Type "show ?" for help.

                       

                      The only option that 'port' gives me is "load-balance"

                        • Re: sx1012 link issue
                          ophirmaor

                          All ports are ethernet,

                          this command is applicable only for Gateway systems (you select which port is Ethernet and which ports are infiniBand)

                           

                          Ophir.

                           

                           

                           

                          Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

                            • Re: sx1012 link issue
                              prieto

                              Ophir,

                               

                              So, if all my ports are already ethernet and there is no specific settings that I need to configure, on my ethernet ports connected point to point with fiber, for me to see some traffic between them that falls back to some issue with the fiber or transceiver that don't seem to be supported by the sx1012.

                               

                              Does anyone know about any third party vendors (finisar FTLX1471D3BCL Duplex SFP with Mellanox adapter) that would work with the sx1012?

                               

                              Thank you,

                              Daniel

                      • Re: sx1012 link issue
                        cmm

                        Situation: Link is not UP on each end, when port speed is 40G Ethernet.


                        Two SX1012 switches connected by ~2500 meters of single mode fiber. Transcievers are QSFP+ LR4 40GE OPNEXT. Models:

                        TRQ5B20MN-LF069

                        TRQ5B20MN-LF100

                         

                        These transceivers support this ~2500 meter cable length.

                         

                        Light readings verified and acceptable  on both ends of the cable. They are within proper range, not saturating (too hot) and not below sensitivity threshold (too weak).

                         

                        Switch software is latest version - 3.4.3002.

                         

                        Switch configurations are proper.

                         

                        Root cause of link issue: The OPNEXT transceivers aren't working, at 40GE,  in these Mellanox switches, with this fiber length.

                         

                        Fix:
                        Mellanox recommends using this transceiver:

                        Module QSFP MC2210511-LR4 Active Optical Module 40GigE+FDR10

                        10km SM
                        wavelengths: 1270, 1290, 1310 and 1330nm,

                        Standard MSA Singlemode Transceiver for 10Gigabit Ethernet and 40Gigabit Ethernet with LC connector.

                         

                        Use with MetroX MTX6XX0 long haul platforms, SX1012, SX1710, SX1036/1700 ports 1,3,33,35 and SX1024 ports 50,52,54,56,58,60.

                        Mellanox guaranties this module functionality only in a Mellanox End-to-End solution.

                        Mellanox doesn't officially support third party LR4 or SR4 modules, though third party transceivers may or may not work in various solutions.

                         

                        Note: Due to a hardware limitation, SwitchX-2 units do support LR-4 modules, but SwitchX models do not. These switches referenced in this discussion are in fact SwitchX-2 models.
                        Additionally, LR4 modules will be supported by ConnectX-4 adapters,  and by the Spectrum family of switches (starting next  software release with ETA December 2015).