10 Replies Latest reply on Oct 4, 2018 9:28 AM by yaolin

    Line rate using Connect_X5 100G EN in Ubuntu; PCIe speed difference;

    h0ax_ki1l3r

      Hi,

      I am trying to get line rate with two machines (A and B) connected back to back using Connect_X5 EN 100G NICs.

       

      Machine A (Transmitting pkts)

      Run DPDK pktgen: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-5 -n 3 -w 04:00.0 -- -T -P -m "[1:2-5].0"

       

      Machine B (Receiving pkts)

      Run DPDK pktgen: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-5 -n 3 -w 04:00.0 -- -T -P -m "[1-4:5].0"

       

      Machine A is sending packets @ 52G

      Machine B is receiving packets @ 16G

       

       

      Here are some more information.

      Dell R620 with Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (6 cores)

      Only one socket is there.

      HT is disabled.

      isolcpus=1-5

      DPDK 18.08

      Ubuntu 16.04 LTS

      MLNX_OFED_LINUX-4.4-2.0.7.0-ubuntu16.04-x86_64 was installed

       

      My questions:

       

      Q1. Why is machine B not able to receive more than 16G packets?

       

      Q2. PCIE capacity and status speeds are diffrent.

      root:~$ sudo lspci -s 04:00.0 -vvv | grep Width

              LnkCap:    Port #0, Speed 16GT/s, Width x16, ASPM not supported, Exit Latency L0s unlimited, L1 unlimited

              LnkSta:    Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

             

           As you see, speed available is 16GT/s, but only 8GT/s is used. How can I increase this? The card is installed in SLOT2_G2_X16(CPU1).

       

      Thanks,

      A