This post discusses and presents the configuration for setting rate limits per Virtual Function (VF) using ConnectX-4/ConnectX-5 adapter cards families.

The max rate limit is supported starting from MLNX_OFED v3.4, while the min rate limit is supported starting from MLNX_OFED 4.0 and later.

References

• HowTo Set Virtual Network Attributes on a Virtual Function (SR-IOV)
• HowTo Configure Privileged VF on ConnectX-4
• HowTo Configure MAC Anti-Spoofing for VMs over SR-IOV

Configuration

As a prerequisite, follow HowTo Configure SR-IOV for ConnectX-4 with KVM (Ethernet). Make sure the Virtual Machine (VM) is configured with a MAC address.

In the following example, the administrator assigned this MAC address 00:22:33:44:55:66 to the VM located on VF-0.

# ip link show

...

9: ens785f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000

    link/ether e4:1d:2d:f2:a4:89 brd ff:ff:ff:ff:ff:ff

    vf 0 MAC 00:22:33:44:55:66, spoof checking off, link-state auto

    vf 1 MAC 00:00:00:00:00:00, spoof checking off, link-state auto
    vf 2 MAC 00:00:00:00:00:00, spoof checking off, link-state auto    vf 3 MAC 00:00:00:00:00:00, spoof checking off, link-state auto...

There are two options to set rate limit per VF:

1. Using the command ip link set.

2. Using the sysfs (In case MLNX_OFED 4.0 is installed).

See examples below:

ip link. (Option 1)

Use the standard ip link command (supported in Kernel version 3.10 and later)

Format:

# ip link set <PF_IF_NAME> vf <VF_IDX> max_tx_rate  <MAX_RATE_IN_MBIT/S> min_tx_rate <MIN_RATE_IN_MBIT/S>

Example to enable rate limiting of 1000Mb/s and minimum of 10Mb/s:

# ip link set ens785f1 vf 0 max_tx_rate 1000 min_tx_rate 10

The rate is measured in Mb/s.

To disable or stop the rate limit per VF, set the rate to 0.

# ip link set ens785f1 vf 0 max_tx_rate 0 min_tx_rate 0

Note: An older version of iproute2 may have only a rate option (this is equal to the max rate)

ip link set <interface_name> vf <vf_id> rate <value>

For more ip link examples, refer to  HowTo Set Virtual Network Attributes on a Virtual Function (SR-IOV).

To verify, run ip link show command.

Rate limit is Enabled (Max + Min).

# ip link show

...

9: ens785f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000

    link/ether e4:1d:2d:f2:a4:89 brd ff:ff:ff:ff:ff:ff

    vf 0 MAC 00:22:33:44:55:66, tx rate 1200 (Mbps), max_tx_rate 1200Mbps, min_tx_rate 1000Mbps spoof checking off, link-state auto

    vf 1 MAC 00:00:00:00:00:00, spoof checking off, link-state auto

    vf 2 MAC 00:00:00:00:00:00, spoof checking off, link-state auto

    vf 3 MAC 00:00:00:00:00:00, spoof checking off, link-state auto

sysfs (Option 2)

Run the echo command with the file located under

• /sys/class/net/<ETH_IF_NAME> /device/sriov/<VF index>/max_tx_rate
• /sys/class/net/<ETH_IF_NAME> /device/sriov/<VF index>/min_tx_rate

For example, to set the maximum rate to 1Gb/s (1000Mb/s) and the minimum rate to 10Mb/s, run:

# echo 1000 > /sys/class/net/ens785f1/vf/0/max_tx_rate

# echo 10  > /sys/class/net/ens785f1/vf/0/min_tx_rate

Note: This configuration is not persistent (it resets with driver restart).

Once you enable the rate limit, test the speed rate from the VM and see that the traffic is limited to the rate you configured.

To get the sysfs configuration, run:

# cat /sys/class/net/ens785f1/device/sriov/0/config

VF : 0

MAC : 00:22:33:44:55:66

VLAN : 0

QoS : 0

SpoofCheck : OFF

Trust : OFF

LinkState : Follow

MinTxRate : 10

MaxTxRate : 1000