Recommended Network Configuration Examples for RoCE Deployment

Version 40

    This post provides recommended network configuration examples for deploying RoCE with Mellanox adapters.

    The below information is applicable for ConnectX-4 and above adapter cards, with MLNX_OFED version 4.1 or WinOF-2 version 1.70 and above.

     

    References

     

    Network Profiles

     

    Configuration Mode

    Trust

    VLAN Required

    ECN

    PFC

    RDMA Priority

    Adapter

    Switch

     

     

     

     

     

     

    Linux

    Windows

    MLNX_OS

    Cumulus

    Cisco
    Lossy fabric

    N/A

    No

    Yes

    No

    Any

    link

    link

    link

    soon

    soon

    Lossy fabric with QoS

    Layer 3

    No

    Yes

    No

    DSCP = 26

    link

    link

    link

    soon

    soon

    Layer 2

    Yes

    Yes

    No

    PCP = 3

    link

    N/A

    link

    soon

    soon

    Lossless fabric

    Layer 3

    No

    Yes

    Yes

    DSCP = 26

    link

    link

    link

    soon

    soon

    Layer 2

    Yes

    Yes

    Yes

    PCP = 3

    link

    N/A

    link

    soon

    soon

     

    To learn more about the network characteristics and profile selection, see: Understanding QoS Configuration for RoCE