RoCEv1 Addressing

Version 6

    This post discusses RoCEv1 Addresses (GIDs) in various operating systems.

     

    RoCEv1 packets carry a GRH (L3 header) with source and destination addresses denoted as SGID and DGID respectively.

    RoCEv1 implementations (Operating Systems, Appliances, etc) derive their GID values using one of two methods:

     

    1. GIDs derived from the the MAC Address and VLAN ID of the corresponding network interface.

    2. GIDs derived from the IP Address of the corresponding network interface.

     

    Both RoCEv1 endnodes must derive their GIDs using the same method in order to connect properly.

     

    Here is a table that summarizes the support for each method per OS Distro / kernel:

     

    OS Distro/ KernelMethod
    MLNX_OFED earlier versions to 2.11
    MLNX_OFED 2.1 or later2
    kernel.org earlier versions to 3.141
    kernel.org 3.14 or later2
    SLES earlier versions to 121
    SLES 12 or later2
    Ubuntu earlier versions to 14.041
    Ubuntu 14.04 or later2

    WinOF 3.20 version up to WinOF 4.80

    WinOF 4.80 added support for Method 2 (the default is Method 1)

    1
    Windows 2012 R2 Inbox1
    VMWare MLNX_OFED_ESX drivers1

     

     

    For example:

    - Two endnodes that one is running MLNX_OFED 2.1 and the other one inbox kernel 3.13 will not be connected using RoCEv1.

    - Two endnodes that one is running WinOF 4.80 and the other Windows 2012 R2 inbox driver will be able to connect using method 1 (configurable in WinOF driver).