1 Reply Latest reply on Oct 26, 2018 7:09 PM by yairi

    IPOIB and RDMA verification over InfiniBand


      Hello everyone;

      I am new to InfinBand and working on my final year project, in which at initial, I have to configure IPOIB and RDMA over InfiniBand. I have configured both of them in Oracle Linux and status of RDMA is "active" and it is "enabled" (see capture 1 and capture 2).

      I tried to test the bandwidth and run the command "ib_write_bw --reprt_gbits --iters=100000 --bidirectional", and waited for client to connect. The result of average bandwidth came out to be around 20 Gbps (see capture 3).

      Firstly, I want to know, whether my RDMA configuration is working fine or do I need to run other scripts or commands as well to verify RDMA ?

      Secondly, if it is working fine, but I am getting average bandwidth of 20 Gbps. However, in actual InfiniBand provides around 40 Gbps link. I have also increased the number of iterations, but it didn't effect the average bandwidth.

      I am utilizing one port of IB card on each system. So, is that reason of getting 20 Gbps bandwidth  or does each of two port supports 40 Gbps link ?