0 Replies Latest reply on Dec 4, 2015 7:34 AM by jwb

    How to get VF rate limiting working with VGT on ConnectX-3 ?

    jwb

      Hello,

       

      I am using a ConnectX-3 and SR-IOV to create some VFs that I give to different processes. One of them is used for a DPDK-based (dpdk.org) based program. I want to rate limit this VF but I cannot find a way to get this working. I have read:

      HowTo Configure QoS over SR-IOV

      and

      http://delivery04.dhe.ibm.com/sar/CMA/XSA/mellanox_sr-iov-kvm_cookbook_11feb2014.pdf

       

      The guide from this site seems to talk about VLANs instead of VLANs priorities which is quite confusing to me. From all my tests, I am quite confident in my conclusion that I can rate limit based on a VLAN priority, not a VLAN id. Anyway, to rate limit a VF in VST mode seems quite simple following the IBM reference. I simply do:

      • ip link set enp132s0 vf 0 vlan 0 qos 1
      • mlnx_qos -i enp132s0 -p 0,1,2,3,4,5,6,7 -s strict,strict,strict,strict,strict,strict,strict,strict -r 0,1,0,0,0,0,0,0

      which ratelimits all traffic from vf0 to 1gbit/s. Unfortunately, it also removes all VLAN tags that I add to the outgoing packets. To avoid this, I should be able to the same in VGT mode right ? like:

      • ip link set enp132s0 vf 0 vlan 4095 qos 1
      • mlnx_qos -i enp132s0 -p 0,1,2,3,4,5,6,7 -s strict,strict,strict,strict,strict,strict,strict,strict -r 0,1,0,0,0,0,0,0

      In this case the traffic from vf0 falls into user priority 0 and is thus not rate limited. Alternatively, I can do:

      • ip link set enp132s0 vf 0 vlan 3000 qos 1
      • mlnx_qos -i enp132s0 -p 0,1,2,3,4,5,6,7 -s strict,strict,strict,strict,strict,strict,strict,strict -r 0,1,0,0,0,0,0,0

      for example. This works - but it adds vlan 3000 to all packets transmitted, however, which is not what i want either.

       

      I can't really figure out if i'm missing anything or why this doesn't work in VGT mode. My mlx4_core.conf files looks like (i basically tried enabling a lot of this to see it helps):

      options mlx4_core port_type_array=2,2 num_vfs=3 probe_vf=3 log_num_mgm_entry_size=-1 fast_drop=1 use_prio=1 enable_qos=1 enable_vfs_qos=1

       

      Any help would be very appreciated. Regards,

      Jesper