Understanding TC Scheduling on Mellanox Spectrum Switches (WRR, SP)

Version 4

    This post discusses the Traffic Class (TC) scheduling option on Mellanox Spectrum switches.

     

    References

     

    Overview

    TC scheduling is performed per egress port to enable the assignment of proper QoS values to each TC.

    Eight TCs can be mapped either to Strict Priority (SP) or to Weighted Round Robin (WRR) scheduler options.

    In the default configuration, all eight TCs are mapped to WRR with the same rate of 12/13.

    switch (config) # show dcb ets interface ethernet 1/1

    Eth1/1

    Interface Bandwidth Shape [Mbps]: N/A

    Multicast unaware mapping : disabled

     

    ETS per TC :

    TC Scheduling Mode Weight Weight (%)

    -- --------------- ------ ----------

    0  WRR             12     12

    1  WRR             13     13

    2  WRR             12     12

    3  WRR             13     13

    4  WRR             12     12

    5  WRR             13     13

    6  WRR             12     12

    7  WRR             13     13

     

    Bandwidth Shape per TC:

    TC Bandwidth Shape [Mbps]

    -- ----------------------

    0  N/A

    1  N/A

    2  N/A

    3  N/A

    4  N/A

    5  N/A

    6  N/A

    7  N/A

     

    Bandwidth Guarantee per TC:

    TC Bandwidth Guaranteed [Mbps]

    -- ---------------------------

    0  0

    1  0

    2  0

    3  0

    4  0

    5  0

    6  0

    7  0

     

    Switch Priority to TC mapping:

    Switch Priority TC

    --------------- --

    0               0

    1               1

    2               2

    3               3

    4               4

    5               5

    6               6

    7               7

     

    Configuration Example

    Refer to HowTo Configure Mellanox Spectrum Switch for Lossless RoCE as a basic example.

    In this example we have 3 TCs as follows:

    • TC0 - TCP traffic
    • TC3 - RDMA traffic
    • TC6 - CNP traffic

    We will configure TC6 to SP, and TC0 and TC3 to WRR with the following BW weights:

    • TC0 - 40% BW
    • TC3 - 60% BW

     

    Run the following commands to set the configuration described above on the required interface:

     

    1. Set TC6 with SP

    switch (config) # interface ethernet 1/1 traffic-class 6 dcb ets strict

     

    2. Set TC0 to WRR and bandwidth weight of 40%, and TC3 to WRR with weight of 60%.

    switch (config) # interface ethernet 1/1 traffic-class 0 dcb ets wrr 40
    switch (config) # interface ethernet 1/1 traffic-class 3 dcb ets wrr 60

     

    3. Set all other unused TCs (1,2,4,5,7) to weigtht 0.

    switch (config) # interface ethernet 1/1 traffic-class 1 dcb ets wrr 0

    switch (config) # interface ethernet 1/1 traffic-class 2 dcb ets wrr 0

    switch (config) # interface ethernet 1/1 traffic-class 4 dcb ets wrr 0

    switch (config) # interface ethernet 1/1 traffic-class 5 dcb ets wrr 0

    switch (config) # interface ethernet 1/1 traffic-class 7 dcb ets wrr 0

     

    4. Show the configuration to see TC0, TC3 and TC6 configuration.

    switch (config) # show dcb ets interface ethernet 1/1

    Eth1/1

    Interface Bandwidth Shape [Mbps]: N/A

    Multicast unaware mapping : disabled

     

    ETS per TC :

    TC Scheduling Mode Weight Weight (%)

    -- --------------- ------ ----------

    0  WRR             40     40

    1  WRR             0      0

    2  WRR             0      0

    3  WRR             60     60

    4  WRR             0      0

    5  WRR             0      0

    6  Strict          0      0

    7  WRR             0      0

     

    Bandwidth Shape per TC:

    TC Bandwidth Shape [Mbps]

    -- ----------------------

    0  N/A

    1  N/A

    2  N/A

    3  N/A

    4  N/A

    5  N/A

    6  N/A

    7  N/A

     

    Bandwidth Guarantee per TC:

    TC Bandwidth Guaranteed [Mbps]

    -- ---------------------------

    0  0

    1  0

    2  0

    3  0

    4  0

    5  0

    6  0

    7  0

     

    Switch Priority to TC mapping:

    Switch Priority TC

    --------------- --

    0               0

    1               1

    2               2

    3               3

    4               4

    5               5

    6               6

    7               7