4 Replies Latest reply on Mar 20, 2015 11:45 AM by jlusticky

    Receive Side Scaling

      Hello,

      I'm running RHEL 7 acting as a router and performing throughput tests.

      My machine is a 6 physical cores Xeon with Connect-X 3 EN dual-port.

      The Spirent generator sends packets from many different flows to one interface and the Linux kernel forwards them to the other interface (unidirectional test).

       

      By default there are 4 RX queues created by the driver for each interafce (largest power of 2 not greater then cpu count), so I changed it to 6 to get one queue per CPU.

      I disabled the irqbalance daemon and set my custom IRQ mapping to get the interrupts evenly distributed.

      However the load is still distributed among 4 softirqs with 100% load and the remaining 2 are sleeping.

      The distribution is perfect according to the /proc/interrupts file.

       

      When I turn hyper-threading ON, to have 12 CPUs and 8 RX queues by default, there are still only 4 softirqs using 100%, which seems strange to me.

      The /proc/interrupts file shows even distribution among all cores.

       

      The behaviour is the same with RHEL 7 kernel and upstream vanilla 3.18 kernel.

      Any advice how to make more softirqs do the work?