Lossless RoCE Configuration for Onyx Switches in PCP-Based QoS Mode

Version 9

    This post provides a configuration example for Mellanox Spectrum installed with MLNX-OS and running RoCE over a lossless network, in PCP-based QoS mode.




    Step 1 - Enable ECN Marking

    Note - For a fair sharing of switch buffer with other traffic classes, it is recommended to configure ECN on all other traffic classes.

    switch (config) # interface ethernet 1/1-1/32 traffic-class 3 congestion-control ecn minimum-absolute 150 maximum-absolute 1500


    Step 2 - Create the RoCE pool and set QoS

    Configure the traffic pool for RoCE

    switch (config) # traffic pool roce type lossless

    switch (config) # traffic pool roce memory percent 50.00
    switch (config) # traffic pool roce map switch-priority 3

    Set a strict priority to CNPs over traffic class 6

    switch (config) # interface ethernet 1/1-1/32 traffic-class 6 dcb ets strict

    [Optional] Enable DCBX LLDP

    Note: This is required in case the adapter card relies on LLDP configuration in the switch for setting priority for PFC

    switch (config) #lldp


    Step 3 - Per port configuration

    Configure the switchport

       interface ethernet 1/1-1/32 switchport mode trunk

       interface ethernet 1/1-1/32 switchport trunk allowed-vlan [vlan-id]