2 Replies Latest reply on Jan 25, 2016 6:44 AM by derisavi

    Low bandwidth using DPDK2.2 and pktgen-dpdk on ConnectX-3

    derisavi

      Hello,

         I'm trying a very simple setup connecting a ConnectX-3 card on one host directly to another ConnectX-3 card on another. I'm using DPDK 2.2 and pktgen-2.9.7 to send packets from one to another. I have configured both sending and receiving ports to Ethernet.

      Things are functional, I can saturate 40Gb/s with packet sizes of 1024 or 512 bytes but not with 256 bytes or lower. Seems like the max throughput I get is 12 Mpps.

      I verified that I have set log_num_mgm_entry_size=-7.

      This is my pktgen command line: pktgen  -c 0xFFF00 -n 4  -w 0000:82:00.0 -m 2048 -- -T -P -p 0x0 -m "[10-11:12-13].0"

       

      Any idea why I can't saturate the line with small packets? Any help would be greatly appreciated.

        • Re: Low bandwidth using DPDK2.2 and pktgen-dpdk on ConnectX-3
          olgas

          Hi,

           

          In matter of fact we didn't run performance tests with pktgen.

          You can try to change rxq, txq and burst size.

          In the past I have tried to check if it can be done dynamically with pktgen, but didn't find a way.

          Changed the following values in app/pktgen-constants.h

                  DEFAULT_PKT_BURST = 64,           // Increasing this number consumes memory very fast

          //#ifdef RTE_LIBRTE_VMXNET3_PMD

          // DEFAULT_RX_DESC = (DEFAULT_PKT_BURST * 8 * 2),

          // DEFAULT_TX_DESC = DEFAULT_RX_DESC,

          //#else

          // DEFAULT_RX_DESC = (DEFAULT_PKT_BURST * 8),

          // DEFAULT_TX_DESC = DEFAULT_RX_DESC,

                  DEFAULT_RX_DESC = 256,

          DEFAULT_TX_DESC = 256,

          //#endif

           

          Try to check TX  first ...

          Hope it helps.

           

          Best Regards,

          Olga

            • Re: Low bandwidth using DPDK2.2 and pktgen-dpdk on ConnectX-3
              derisavi

              Hi Olga,

                 Thanks for your comments. Burst size is set to 32 by default, at least when I try it on 82599 or ConnectX-3. And it can be modified using "set <port#> burst <value>" command in pktgen. Pktgen doesn't accept a value larger than 32 for the burst size (unless I recompile the source code obviously as you mentioned). However, regardless of whether I set the burst size to 1 or 32, I still get the same ~12 Mpps.

               

              Regards,

              --Salem