RoCE Configuration for MLNX-OS Switches in PCP-Based QoS Mode

Version 4

    This post provides a configuration example of RoCE over lossy networks for MLNX-OS switches in PCP-based QoS mode.

    Notes

     

    References

     

    Overview

    This solutions offers the following network setup:

     

    Configuration

    1. Enable ECN for RoCE traffic that runs over traffic class 3 in order to get DCQCN congestion control to work.

    switch (config) # interface ethernet 1/1-1/32 traffic-class 3 congestion-control ecn minimum-absolute 150 maximum-absolute 1500

    Note: For a fair sharing of switch buffer with other traffic classes, it is recommended to configure ECN on all other traffic classes.

     

    2. Configure the buffer pool - allocate buffer pool 0 for lossy traffic.

    switch (config) # traffic pool roce type lossy

    switch (config) # traffic pool roce memory percent 50.00
    switch (config) # traffic pool roce map switch-priority 3

     

    3. Set a strict priority to CNPs over traffic class 6.

    interface ethernet 1/1-1/32 traffic-class 6 dcb ets strict

     

    4. Change port mode to hybrid.

    Note: This is required for preserving VLAN tag and PCP value when forwarding the packet.

    switch (config) # interface ethernet 1/1-1/32 switchport mode hybrid

     

    5. Add the relevant VLAN to the switch.

    switch (config) # vlan 10
    switch (config vlan 10) exit
    switch (config) # interface ethernet 1/1-1/32 switchport hybrid allowed-vlan all

     

    For more information about switch port mode, see HowTo Configure Switch Port Types with MLNX-OS.

     

    For end to end configuration example and troubleshooting, see HowTo Configure Resilient RoCE (ECN) End-to-End Using ConnectX-4 and Spectrum (Trust L2).