RDMA/RoCE Considerations for Windows 2016 on ConnectX-3 Adapters

Version 4

    This post discuss the RDMA/RoCE Considerations when using Windows 2016 inbox WinOF driver.

    This post is basic and and meant for IT managers and field engineers.

     

    References

     

    In Windows 2012, RDMA was enabled by default on ConnectX-3 adapters. However, in many cases, the IT managers of the networks did not create lossless network (PFC/QoS) environment to enable RDMA (RDMA requires lossless network). This behavior caused the SMB to switch from RDMA to TCP when congestion happened in the network. Unlike Windows 2012, RDMA capabilities will be disabled in Windows 2016 in case QoS and PFC will be disabled as well.

     

    To enable RDMA on Windows 2016, simply follow HowTo Configure SMB Direct (RoCE) over PFC on Windows 2012 Server. It is the same procedure.

     

    In order to fall-back to Windows 2012 behavior we added a new registry key NdkWithGlobalPause to the network adapter advanced property.

    This key will roll the driver behavior back to the old behavior which means that RDMA will be turned on by default.

     

    To enable this registry key via PowerShell, run:

    PS C:> Set-NetAdapterAdvancedProperty –Name <NetAdapter Name> –RegistryKeyword " NdkWithGlobalPause " –RegistryValue "1"

     

    The default value of this parameter is 0 (disbaled).