mlnx_qos

Version 16

    This post discusses the different options supported for the mlnx_qos Linux tool (part of MLNX_OFED).

    This post applies to MLNX_OFED v4.2.

     

    References

     

    Parameters

    -i, --interface

    This parameter is used to get the current QoS configuration:

    # mlnx_qos -i eth2

    DCBX mode: OS controlled
    Priority trust state: dscp
    dscp2prio mapping:
    prio:0 dscp:07,06,05,04,03,02,01,00,
    prio:1 dscp:15,14,13,12,11,10,09,08,
    prio:2 dscp:23,22,21,20,19,18,17,16,
    prio:3 dscp:31,30,29,28,27,26,25,24,
    prio:4 dscp:39,38,37,36,35,34,33,32,
    prio:5 dscp:47,46,45,44,43,42,41,40,
    prio:6 dscp:55,54,53,52,51,50,49,48,
    prio:7 dscp:63,62,61,60,59,58,57,56,
    Cable len: 10
    PFC configuration:
    priority 0 1 2 3 4 5 6 7
    enabled 1 0 1 0 0 0 1 1

    tc: 0 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 1

    tc: 1 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 0

    tc: 2 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 2

    tc: 3 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 3

    tc: 4 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 4

    tc: 5 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 5

    tc: 6 ratelimit: unlimited, tsa: strict

    priority: 6

    tc: 7 ratelimit: unlimited, tsa: ets, bw: 40%

    priority: 7

     

    -f, --pfc

    This parameter is used to set the Priority Flow Control (PFC) configuration on a port. You can enable any subset of priorities 0...7.

    The priorities are provided using a comma-delimited list.

    For example, to enable priority 3, run:

    # mlnx_qos -i eth2 -f 0,0,0,1,0,0,0,0

    To check the configuration, run:

    # mlnx_qos -i eth2

    PFC configuration:

    priority 0 1 2 3 4 5 6 7

    enabled 0 0 0 1 0 0 0 0

     

    ...

    PFC is enabled on priority 3.

     

    See also, HowTo Configure PFC on ConnectX-4 .

     

    -p --prio_tc

    This parameter is used to map priorities to Egress Traffic Class (ETC).

    Note: By default, priority 0 is mapped to tc1, and priority 1 is mapped to tc0. All other priorities are mapped to the same TC.

     

    To change the mapping, add a command-delimited list. For example:

    # mlnx_qos -i eth2 -p 0,1,2,3,4,5,6,7

    Default mapping:

    # mlnx_qos -i eth2

    PFC configuration:

    priority 0 1 2 3 4 5 6 7

    enabled 0 0 0 1 0 0 0 0

     

    tc: 0 ratelimit: unlimited, tsa: vendor

    priority: 1

    tc: 1 ratelimit: unlimited, tsa: vendor

    priority: 0

    tc: 2 ratelimit: unlimited, tsa: vendor

    priority: 2

    tc: 3 ratelimit: unlimited, tsa: vendor

    priority: 3

    tc: 4 ratelimit: unlimited, tsa: vendor

    priority: 4

    tc: 5 ratelimit: unlimited, tsa: vendor

    priority: 5

    tc: 6 ratelimit: unlimited, tsa: vendor

    priority: 6

    tc: 7 ratelimit: unlimited, tsa: vendor

    priority: 7

     

    -s --tsa , -t -t-cbw

     

    -s is used to set the Egress scheduling method, either strict priority or ETS per traffic class.

    -t is used to set the bandwidth per traffic class (TC) (relevant when this TC egress scheduling method is ETS). The sum must be 100, while strict priority must get 0 bw.

    These configurations must come together.

     

    For example, to set strict priority to tc6, while all the rest are ETS, run:

    # mlnx_qos -i eth2 -s ets,ets,ets,ets,ets,ets,strict,ets -t 10,10,10,10,10,10,0,40

    Priority trust mode: pcp

    PFC configuration:

    priority 0 1 2 3 4 5 6 7

    enabled 0 0 0 0 0 0 0 0

    tc: 0 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 1

    tc: 1 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 0

    tc: 2 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 2

    tc: 3 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 3

    tc: 4 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 4

    tc: 5 ratelimit: unlimited, tsa: ets, bw: 10%

    priority: 5

    tc: 6 ratelimit: unlimited, tsa: strict

    priority: 6

    tc: 7 ratelimit: unlimited, tsa: ets, bw: 40%

    priority: 7

     

    -r --ratelimit

    This parameter is used to add a rate limit for each TC. For example, to limit tc0 to 5G, tc3 to 10G, and tc6 to 7G, run:

    # mlnx_qos -i eth2 -r 5,0,0,10,0,0,7,0

     

    -d --dcbx

    This parameter is used to get/set the DCBX mode (firmware or OS controlled). For more information, see HowTo Auto-Config PFC and ETS on ConnectX-4 via LLDP DCBX.

     

    --trust

    This parameter is used to set the trust level (pcp or dscp). See also HowTo Configure Trust State on Mellanox Adapters.

    Note that for this setting to take effect on VF, please reload the VF driver or restart the VM.

    # mlnx_qos -i eth2 --trust=dscp

    DCBX mode: OS controlled
    Priority trust state: dscp
    dscp2prio mapping:
    prio:0 dscp:07,06,05,04,03,02,01,00,
    prio:1 dscp:15,14,13,12,11,10,09,08,
    prio:2 dscp:23,22,21,20,19,18,17,16,
    prio:3 dscp:31,30,29,28,27,26,25,24,
    prio:4 dscp:39,38,37,36,35,34,33,32,
    prio:5 dscp:47,46,45,44,43,42,41,40,
    prio:6 dscp:55,54,53,52,51,50,49,48,
    prio:7 dscp:63,62,61,60,59,58,57,56,

    --dscp2prio

    This parameter is used to set dscp to priority mapping when the trust state is "dscp". Note that for this setting to take effect on VF, please reload the VF driver or restart the VM.

    For example, to map dscp 30 to priority 6, run:

    # mlnx_qos -i eth2 --dscp2prio set,30,6

    DCBX mode: OS controlled

    Priority trust state: dscp

    dscp2prio mapping:

    prio:0 dscp:07,06,05,04,03,02,01,00,

    prio:1 dscp:15,14,13,12,11,10,09,08,

    prio:2 dscp:23,22,21,20,19,18,17,16,

    prio:3 dscp:31,29,28,27,26,25,24,

    prio:4 dscp:39,38,37,36,35,34,33,32,

    prio:5 dscp:47,46,45,44,43,42,41,40,

    prio:6 dscp:30,55,54,53,52,51,50,49,

    48,

    prio:7 dscp:63,62,61,60,59,58,57,56,

    --buffer_size

    Support from OFED 4.4. This parameter is used to set receive buffer size. Note that to avoid traffic loss, lossless buffer size should be at least (xoff + 8 * MTU) and lossy buffer size should not be 0.

    For example, this below command set buffer size 0,1,2,3,4,5,6,7 to the size of 80296,80296,1000,80296,1000,1000,1000,1000 respectively.

    # mlnx_qos -i ens2f0 --buffer_size 80296,80296,1000,80296,1000,1000,1000,1000

    DCBX mode: OS controlled

    Priority trust state: pcp

    Receive buffer size (bytes): 80256,80256,896,80256,896,896,896,896,

    Cable len: 7

    PFC configuration:

    priority 0 1 2 3 4 5 6 7

    enabled 0 0 0 0 0 0 0 0

    buffer 0 0 0 0 0 0 0 0

    --prio2buffer

    Support from OFED 4.4. This parameter is used to manually map a priority to a receive buffer.

    For example, this below command set priority 0,1,2,3,4,5,6,7 to buffer 0,2,5,7,1,2,3,6 respectively.

    # mlnx_qos -i ens2f0 --prio2buffer 0,2,5,7,1,2,3,6

    DCBX mode: OS controlled

    Priority trust state: pcp

    Receive buffer size (bytes): 80256,80256,896,80256,896,896,896,896,

    Cable len: 7

    PFC configuration:

    priority 0 1 2 3 4 5 6 7

    enabled 0 0 0 0 0 0 0 0

    buffer 0 2 5 7 1 2 3 6

    -h

    This parameter is used to display help information.

    # mlnx_qos -h

    Usage: mlnx_qos -i <interface> [options]

    Options:
    --version show program's version number and exit
    -h, --help show this help message and exit
    -f LIST, --pfc=LIST Set priority flow control for each priority. LIST is
    comma separated value for each priority starting from
    0 to 7. Example: 0,0,0,0,1,1,1,1 enable PFC on TC4-7
    -p LIST, --prio_tc=LIST
    maps UPs to TCs. LIST is 8 comma separated TC numbers.
    Example: 0,0,0,0,1,1,1,1 maps UPs 0-3 to TC0, and UPs
    4-7 to TC1
    -s LIST, --tsa=LIST Transmission algorithm for each TC. LIST is comma
    separated algorithm names for each TC. Possible
    algorithms: strict, ets and vendor. Example:
    vendor,strict,ets,ets,ets,ets,ets,ets sets TC0 to
    vendor, TC1 to strict, TC2-7 to ets.
    -t LIST, --tcbw=LIST Set minimal guaranteed %BW for ETS TCs. LIST is comma
    separated percents for each TC. Values set to TCs that
    are not configured to ETS algorithm must be zero.
    Example: if TC0,TC2 are set to ETS, then
    10,0,90,0,0,0,0,0 will set TC0 to 10% and TC2 to 90%.
    Percents must sum to 100.
    -r LIST, --ratelimit=LIST
    Rate limit for TCs (in Gbps). LIST is a comma
    separated Gbps limit for each TC. Example: 1,8,8 will
    limit TC0 to 1Gbps, and TC1,TC2 to 8 Gbps each.
    -d DCBX, --dcbx=DCBX set dcbx mode to firmware controlled(fw) or OS
    controlled(os). Note, when in OS mode, mlnx_qos should
    not be used in parallel with other dcbx tools such as
    lldptool
    --trust=TRUST set priority trust state to pcp or dscp
    --dscp2prio=DSCP2PRIO
    set/del a (dscp,prio) mapping. Example 'set,30,2' maps
    dscp 30 to priority 2. 'del,30,2' deletes the mapping
    and resets the dscp 30 mapping back to the default
    setting priority 0.
    --cable_len=CABLE_LEN
    set cable_len for buffer's xoff and xon thresholds
    --prio2buffer=LIST maps priority to receive buffer. Example:
    0,2,5,7,1,2,3,6 maps priorities 0,1,2,3,4,5,6,7 to
    receive buffer 0,2,5,7,1,2,3,6
    --buffer_size=LIST Set receive buffer size LIST is comma separated
    percents for each buffer.For pfc enabled buffer, the
    buffer size must be larger than the xoff_threshold.
    Example: 87296,87296,0,87296,0,0,0,0 sets receive
    buffer size for buffer 0,1,2,3,4,5,6,7 respectively
    -i INTF, --interface=INTF
    Interface name
    -a Show all interface's TCs