Understanding show_gids Script

Version 9

    show_gids is a nice script that provides you a summary table listing the device, port, index GID, IP address, RoCE version and the device name. It is available with OFED v3.4 or later.

     

     

    References

     

    Output Example

    Here is the script output:

    # ./show-gids

    DEV     PORT INDEX GID                                      IPv4           VER   DEV

    ---     ---- ----- ---                                      ------------   ---   ---

    mlx4_0  1    0     fe80:0000:0000:0000:e61d:2dff:fe26:3ce1                 V1    ens817

    mlx4_0  2    0     fe80:0000:0000:0000:e61d:2dff:fe26:3ce2                 V1    ens817d1

    mlx4_0  2    1     0000:0000:0000:0000:0000:ffff:0303:0306  3.3.3.6        V1    ens817d1

    mlx5_0  1    0     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V2    ens785f0

    mlx5_0  1    1     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V1.5  ens785f0

    mlx5_0  1    2     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V1.5  ens785f0

    mlx5_0  1    3     fe80:0000:0000:0000:e61d:2dff:fef2:a45c                 V1    ens785f0

    mlx5_0  1    4     0000:0000:0000:0000:0000:ffff:0909:0906  9.9.9.6        V1    ens785f0

    mlx5_0  1    5     0000:0000:0000:0000:0000:ffff:0909:0906  9.9.9.6        V2    ens785f0

    mlx5_0  1    6     0000:0000:0000:0000:0000:ffff:0909:0906  9.9.9.6        V1.5  ens785f0

    mlx5_0  1    7     0000:0000:0000:0000:0000:ffff:6363:6306  99.99.99.6     V1    ens785f0.100

    mlx5_0  1    8     0000:0000:0000:0000:0000:ffff:6363:6306  99.99.99.6     V2    ens785f0.100

    mlx5_0  1    9     0000:0000:0000:0000:0000:ffff:6363:6306  99.99.99.6     V1.5  ens785f0.100

    mlx5_1  1    0     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V2    ens785f1

    mlx5_1  1    1     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V1.5  ens785f1

    mlx5_1  1    2     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V1.5  ens785f1

    mlx5_1  1    3     fe80:0000:0000:0000:e61d:2dff:fef2:a45d                 V1    ens785f1

    mlx5_1  1    4     0000:0000:0000:0000:0000:ffff:0c0c:0c06  12.12.12.6     V1    ens785f1

    mlx5_1  1    5     0000:0000:0000:0000:0000:ffff:0c0c:0c06  12.12.12.6     V2    ens785f1

    mlx5_1  1    6     0000:0000:0000:0000:0000:ffff:0c0c:0c06  12.12.12.6     V1.5  ens785f1

    n_gids_found=20

     

    Note: In case you're running performance benchmarks, you will need the GID index, denoted as the INDEX column in the table output. For example, Index 8 in the above example suits RoCE (v2) running on VLAN 100, interface ens785f0.100, ConnectX-4.

     

    The Script

    #!/bin/bash

     

    black='\E[30;50m'

    red='\E[31;50m'

    green='\E[32;50m'

    yellow='\E[33;50m'

    blue='\E[34;50m'

    magenta='\E[35;50m'

    cyan='\E[36;50m'

    white='\E[37;50m'

    bold='\033[1m'

    gid_count=0

     

    # cecho (color echo) prints text in color.

    # first parameter should be the desired color followed by text

    function cecho ()

    {

      echo -en $1

      shift

      echo -n $*

      tput sgr0

    }

     

    # becho (color echo) prints text in bold.

    becho ()

    {

      echo -en $bold

      echo -n $*

      tput sgr0

    }

     

    function print_gids()

    {

      dev=$1

      port=$2

      for gf in /sys/class/infiniband/$dev/ports/$port/gids/* ; do

      gid=$(cat $gf);

      if [ $gid = 0000:0000:0000:0000:0000:0000:0000:0000 ] ; then

      continue

      fi

      echo -e $(basename $gf) "\t" $gid

      done

    }

     

    echo -e "DEV\tPORT\tINDEX\tGID\t\t\t\t\tIPv4  \t\tVER\tDEV"

    echo -e "---\t----\t-----\t---\t\t\t\t\t------------  \t---\t---"

    DEVS=$1

    if [ -z "$DEVS" ] ; then

      DEVS=$(ls /sys/class/infiniband/)

    fi

    for d in $DEVS ; do

      for p in $(ls /sys/class/infiniband/$d/ports/) ; do

      for g in $(ls /sys/class/infiniband/$d/ports/$p/gids/) ; do

                   gid=$(cat /sys/class/infiniband/$d/ports/$p/gids/$g);

                    if [ $gid = 0000:0000:0000:0000:0000:0000:0000:0000 ] ; then

                            continue

                    fi

                    if [ $gid = fe80:0000:0000:0000:0000:0000:0000:0000 ] ; then

                            continue

                    fi

      _ndev=$(cat /sys/class/infiniband/$d/ports/$p/gid_attrs/ndevs/$g 2>/dev/null)

      __type=$(cat /sys/class/infiniband/$d/ports/$p/gid_attrs/types/$g 2>/dev/null)

      _type=$(echo $__type| grep -o "[Vv].*")

      if [ $(echo $gid | cut -d ":" -f -1) = "0000" ] ; then

      ipv4=$(printf "%d.%d.%d.%d" 0x${gid:30:2} 0x${gid:32:2} 0x${gid:35:2} 0x${gid:37:2})

                      echo -e "$d\t$p\t$g\t$gid\t$ipv4  \t$_type\t$_ndev"

      else

      echo -e "$d\t$p\t$g\t$gid\t\t\t$_type\t$_ndev"

      fi

      gid_count=$(expr 1 + $gid_count)

            done #g (gid)

      done #p (port)

    done #d (dev)

     

    echo n_gids_found=$gid_count

     

    Programming using libibverbs

    Use ibv_query_gid and ibv_find_gid_index functions defined in libibverbs to get the desired GID index.

    You can refer to the following posts for details.