7 Replies Latest reply on Nov 26, 2014 9:14 AM by eric

    Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?

      Hello.

       

      I have Ubuntu 14.04. I have Mellanox Technologies MT27500 Family [ConnectX-3] HCAs. I was able to pass all the tests like ibping and ib_rdma_lat.

      Now I'm running MPI tests to verify that InfiniBand is working. How can I actually see that there is traffic going through the HCA's port?

       

      root@node3:~/ibdump-1.0.5-4-rpms/ibdump-1.0.5-4.x86_64-rhel6/usr/bin# ./ibdump
       ------------------------------------------------
       IB device                      : "mlx4_0"       
       IB port                        : 1              
       Dump file                      : sniffer.pcap   
       Sniffer WQEs (max burst size)  : 4096           
       ------------------------------------------------
                                                       
      Initiating resources ...                         
      searching for IB devices in host                 
      Warning: memory access to device mlx4_0 failed: Input/output error.
      Warning: Fallback on IO: much slower, and unsafe if device in use.
      -E- Failed to open mst device mlx4_0: No such file or directory   
      -E- failed to create resources                                    
      

       

      I've tried to execute the ibdump utility (from all the RPMs that were in the download). But it fails. with this:

       

      But there is actually the mlx4_0 device. As I said all the tests from the infiniband-diags package are passed.

       

      Is ibdump supported in MLNX_OFED 2 in Ubuntu 14.04? If no then what alternatives can I use?

       

       

      Thank you.

        • Re: Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?
          eddie.notz

          Hi Grigory,

           

          ibdump 1.0.5.4 doesn't support ConnectX-3, and currently there is no ibdump version that supports Ubuntu/ Debian.

           

          Currently there is no alternative for IB traffic on Ubuntu Machines.

           

          I suggest to check with the community or support@mellanox.com  in the coming future ( next few months) for any development around this feature

          • Re: Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?
            vangelis

            I was able to capture IB traffic with the ConnectX-3 adapters in Ubuntu 12.04, by using ibdump 2.0.0 from the Mellanox OFED RPMs.

            I just converted the rpm package to deb using alien.

              • Re: Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?
                eric

                Hi,

                 

                I try to capture the RDMA traffic also with ibdump, but Ibdump 2.0 doesn't work on my host.

                OS: SUSE (SLES11SP3), library OFED-3.12-1-rc3, Mellanox Connect X-3 NIC.

                 

                # ibdump  --mem-mode 65536

                Initiating resources ...

                searching for IB devices in host

                Port active_mtu=4096

                MR was registered with addr=0x6197e0, lkey=0xe8011900, rkey=0xe8011900, flags=0x1

                ------------------------------------------------

                Device                         : "mlx4_0"

                Physical port                  : 1

                Link layer                     : Infiniband

                Dump file                      : sniffer.pcap

                Sniffer WQEs (max burst size)  : 4096

                ------------------------------------------------

                 

                -E- Failed to set sniffer mode. ibv_create_flow failed: File exists

                #

                 

                It creates an empty 'sniffer.pcap' file, and that's all. No capture file.

                Does ibdump work with OFED library? I haven't installed the MLNX_OFED lib.Is it required?

                 

                thanks

                Eric

                  • Re: Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?
                    vangelis

                    I have only tried with MLNX_OFED and it worked nicely for me. I was able to capture and open the files in wireshark.
                    I haven't tried the OFED library.

                      • Re: Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?
                        eric

                        I've uninstalled OFED and installed MLNX_OFED_LINUX-2.3-2.0.0 on my SUSE host, but ibdump does not work better.

                         

                        # ibdump

                        Initiating resources ...

                        searching for IB devices in host

                        Port active_mtu=4096

                        MR was registered with addr=0x6198f0, lkey=0x88011300, rkey=0x88011300, flags=0x1

                        ------------------------------------------------

                        Device                         : "mlx4_0"

                        Physical port                  : 1

                        Link layer                     : Infiniband

                        Dump file                      : sniffer.pcap

                        Sniffer WQEs (max burst size)  : 4096

                        ------------------------------------------------

                         

                        -E- Failed to set sniffer mode. ibv_create_flow failed: Function not implemented

                         

                        any idea about the cause of this "function not implemented" ?

                         

                        Thanks

                        Eric

                  • Re: Is ibdump in Mellanox OFED 2 supported in Ubuntu 14.04?
                    vangelis

                    It was working for me on MLNX_OFED 2.0-3.0.0 on Ubuntu 12.04.

                    Now that I upgraded to the latest version of the drivers (2.3-2.0.0), I got the same error as the one you got.

                     

                    However, if I use the ibdump package included in the MLNX_OFED 2.0-3.0.0, it works even with the latest drivers (2.3-2.0.0)!

                    1 of 1 people found this helpful