Understanding show_gids Script

Version 7

    Once you understand HowTo Configure RoCE on ConnectX-4, you can search for methods to make it easier.

    Here is a nice script that shows you a summary table with the device, port, index GID, IP address, RoCE version and the device name.

    Note: you need to use OFED 3.2 or later.

     

    *** Update: The show_gids script was added to MLNX_OFED 3.4

     

    References

     

    Output Example

    Here is the script output:

    # ./show-gids

    DEV     PORT INDEX GID                                      IPv4           VER   DEV

    ---     ---- ----- ---                                      ------------   ---   ---

    mlx4_0  1    0     fe80:0000:0000:0000:e61d:2dff:fe26:3ce1                 V1    ens817

    mlx4_0  2    0     fe80:0000:0000:0000:e61d:2dff:fe26:3ce2                 V1    ens817d1

    mlx4_0  2    1     0000:0000:0000:0000:0000:ffff:0303:0306  3.3.3.6        V1    ens817d1

    mlx5_0  1    0     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V2    ens785f0

    mlx5_0  1    1     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V1.5  ens785f0

    mlx5_0  1    2     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V1.5  ens785f0

    mlx5_0  1    3     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V1    ens785f0

    mlx5_0  1    4     0000:0000:0000:0000:0000:ffff:0909:0906  9.9.9.6        V1    ens785f0

    mlx5_0  1    5     0000:0000:0000:0000:0000:ffff:0909:0906  9.9.9.6        V2    ens785f0

    mlx5_0  1    6     0000:0000:0000:0000:0000:ffff:0909:0906  9.9.9.6        V1.5  ens785f0

    mlx5_0  1    7     0000:0000:0000:0000:0000:ffff:6363:6306  99.99.99.6     V1    ens785f0.100

    mlx5_0  1    8     0000:0000:0000:0000:0000:ffff:6363:6306  99.99.99.6     V2    ens785f0.100

    mlx5_0  1    9     0000:0000:0000:0000:0000:ffff:6363:6306  99.99.99.6     V1.5  ens785f0.100

    mlx5_1  1    0     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V2    ens785f1

    mlx5_1  1    1     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V1.5  ens785f1

    mlx5_1  1    2     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V1.5  ens785f1

    mlx5_1  1    3     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V1    ens785f1

    mlx5_1  1    4     0000:0000:0000:0000:0000:ffff:0c0c:0c06  12.12.12.6     V1    ens785f1

    mlx5_1  1    5     0000:0000:0000:0000:0000:ffff:0c0c:0c06  12.12.12.6     V2    ens785f1

    mlx5_1  1    6     0000:0000:0000:0000:0000:ffff:0c0c:0c06  12.12.12.6     V1.5  ens785f1

    n_gids_found=20

     

    Note: In many cases, such as performance benchmarks, you will need the GID index, marked with INDEX column. For example, Index 8 in the above example, suit RoCEv2 running on VLAN 100, interface ens785f0.100, ConnectX-4.

     

    The Script

    #!/bin/bash

     

    black='\E[30;50m'

    red='\E[31;50m'

    green='\E[32;50m'

    yellow='\E[33;50m'

    blue='\E[34;50m'

    magenta='\E[35;50m'

    cyan='\E[36;50m'

    white='\E[37;50m'

    bold='\033[1m'

    gid_count=0

     

    # cecho (color echo) prints text in color.

    # first parameter should be the desired color followed by text

    function cecho ()

    {

      echo -en $1

      shift

      echo -n $*

      tput sgr0

    }

     

    # becho (color echo) prints text in bold.

    becho ()

    {

      echo -en $bold

      echo -n $*

      tput sgr0

    }

     

    function print_gids()

    {

      dev=$1

      port=$2

      for gf in /sys/class/infiniband/$dev/ports/$port/gids/* ; do

      gid=$(cat $gf);

      if [ $gid = 0000:0000:0000:0000:0000:0000:0000:0000 ] ; then

      continue

      fi

      echo -e $(basename $gf) "\t" $gid

      done

    }

     

    echo -e "DEV\tPORT\tINDEX\tGID\t\t\t\t\tIPv4  \t\tVER\tDEV"

    echo -e "---\t----\t-----\t---\t\t\t\t\t------------  \t---\t---"

    DEVS=$1

    if [ -z "$DEVS" ] ; then

      DEVS=$(ls /sys/class/infiniband/)

    fi

    for d in $DEVS ; do

      for p in $(ls /sys/class/infiniband/$d/ports/) ; do

      for g in $(ls /sys/class/infiniband/$d/ports/$p/gids/) ; do

                   gid=$(cat /sys/class/infiniband/$d/ports/$p/gids/$g);

                    if [ $gid = 0000:0000:0000:0000:0000:0000:0000:0000 ] ; then

                            continue

                    fi

                    if [ $gid = fe80:0000:0000:0000:0000:0000:0000:0000 ] ; then

                            continue

                    fi

      _ndev=$(cat /sys/class/infiniband/$d/ports/$p/gid_attrs/ndevs/$g 2>/dev/null)

      __type=$(cat /sys/class/infiniband/$d/ports/$p/gid_attrs/types/$g 2>/dev/null)

      _type=$(echo $__type| grep -o "[Vv].*")

      if [ $(echo $gid | cut -d ":" -f -1) = "0000" ] ; then

      ipv4=$(printf "%d.%d.%d.%d" 0x${gid:30:2} 0x${gid:32:2} 0x${gid:35:2} 0x${gid:37:2})

                      echo -e "$d\t$p\t$g\t$gid\t$ipv4  \t$_type\t$_ndev"

      else

      echo -e "$d\t$p\t$g\t$gid\t\t\t$_type\t$_ndev"

      fi

      gid_count=$(expr 1 + $gid_count)

            done #g (gid)

      done #p (port)

    done #d (dev)

     

    echo n_gids_found=$gid_count

     

    Programming using libibverbs

    Use ibv_query_gid and ibv_find_gid_index functions defined in libibverbs to get the desired GID index.

    Refer to here: