LRH and GRH InfiniBand Headers

Version 2

    This post provides a glimpse into how the Local Route Headers (LRH) and Global Route Headers (GRH) headers of InfiniBand work.

    The post is basic and is meant for beginners.

     

    References

    • InfiniBand Specification version 1.3
    Figure 1: IBA Data Packet Format*

    IBA_data_packet_format.jpg

    * Graphic courtesy of the InfiniBand Trade Association.

    Local Route Headers

     

    21.PNG

     

    The addressing in the Link Layer is the Local Identifier (LID). Please note the presence of the Source LID (SLID) and Destination LID (DLID).

    The 16-bit LID is local to the subnet and assigned by the SM (unlike MAC address which is fixed).

     

    Global Route Headers

     

    21.PNG

    In general, the GRH header does not need to accompany the InfiniBand packet. It is needed when the packet needs to be routed via the IB router, however. See IB Router Architecture and Functionality .

    The addressing in the Network Layer is the 128-bit Global Identifier.

     

    * From the Spec:

     

    Unicast GID scoping is defined as:

    a) Link-local - A unicast GID used within a local subnet using the default GID prefix. Routers must not forward any packets with either link-local source or destination GIDs outside the local subnet. A link-local GID has the following format:

    1.png

     

    b) Site-local - A unicast GID used within a collection of subnets which is unique within that collection (e.g. a data center or campus) but is not necessarily globally unique. Routers must not forward any packets with either a site-local Source GID (SGID) or a site-local Destination GID (DGID) outside of the site.

    132.PNG

    The IB Router uses the site local GID.