HowTo Configure TGT enabled with iSER Transport for Ubuntu

Version 15

    This post shows how to configure set tgt ISCSI client and target to work in iSER transport mode for Ubuntu distribution.

     

    References

     

    Setup

     

    Network

    The connectivity between the iSCSI client and target should be on lossless network required for RoCE. PFC or global pause should be enabled across the network.

     

    Notes:

    (1) MLNX_OFED installation install by default all relevant modules to enable iSER.

    (2) For servers without MLNX_OFED, you need to install several modules using the command:

    # apt-get install libmlx4-1 tgt

     

    (3) For debug purposes install

    # apt-get install infiniband-diags ibutils ibverbs-utils rdmacm-utils perftest

     

    (4) Make sure the following modules are loaded in the kernel (use lsmod command)

    • mlx4_core
    • mlx4_ib
    • mlx4_en
    • ib_core
    • ib_addr
    • ib_sa
    • ib_cm
    • rdma_cm
    • rdma_ucm
    • ib_iser
    • ib_isert

     

    (4) The only configuration change between regular iSCSI and iSER is a simple flag  More information on the commands can be found on the man pages of the application.

     

    (5) The basic management utility is ‘iscsiadm’, which is part of iSCSI package on any distro, and is capable of configuring iSER.

     

    iSCSI target (Server) configuration:

    Server configuration is generic and fairly simple. In the following link you can find example for Ubuntu

    http://www.server-world.info/en/note?os=Ubuntu_12.04&p=iscsi

     

    iSCSI client configuration:

    Client configuration (Initiator) is generic and fairly simple as well. In the following link you can find example for Ubuntu 14.04

    https://help.ubuntu.com/14.04/serverguide/iscsi-initiator.html

    http://www.server-world.info/en/note?os=Ubuntu_12.04&p=iscsi&f=2

     

    Server configuration example (with null device):

    # apt-get install infiniband-diags ibutils ibverbs-utils rdmacm-utils perftest libmlx4-1 tgt

    # tgtd

    # tgt-setup-lun -n tgt-2 -d /tmp/null -b null -t iser

     

    To verify that the target exist run:

    # tgtadm -m target -o show | grep -i target

    Target 1: iqn.2001-04.com.reg-r-vrt-003.tgt-2

    Client configuration example:

    # apt-get install infiniband-diags ibutils ibverbs-utils rdmacm-utils perftest libmlx4-1 open-iscsi-utils open-iscsi

    # modprobe ib_iser

    # iscsiadm -m discovery -t st -p 11.11.11.3:3260

    11.11.11.3:3260,1 iqn.2001-04.com.reg-r-vrt-003-tgt-2

    # iscsiadm -m node

    11.11.11.3:3260,1 iqn.2001-04.com.reg-r-vrt-003-tgt-2

    # iscsiadm -m node -T iqn.2001-04.com.reg-r-vrt-003-tgt-2 | grep transport

    iface.transport_name = tcp

    # iscsiadm -m node -T iqn.2001-04.com.reg-r-vrt-003-tgt-2 -o update -n iface.transport_name -v iser

    # iscsiadm -m node -T iqn.2001-04.com.reg-r-vrt-003-tgt-2 | grep transport

    iface.transport_name = iser

    # iscsiadm -m node -l

    Logging in to [iface: iser, target: iqn.2001-04.com.reg-r-vrt-003-tgt-2, portal: 11.11.11.3,3260] (multiple)

    Login to [iface: iser, target: iqn.2001-04.com.reg-r-vrt-003-tgt-2, portal: 11.11.11.3,3260] successful.

    # fdisk -l

    Disk /dev/sda: 82.3 GB, 82348277760 bytes
    255 heads, 63 sectors/track, 10011 cylinders, total 160836480 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00052dfe

       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *        2048     2000895      999424   83  Linux
    /dev/sda2         2000896    41062399    19530752   83  Linux
    /dev/sda3        41062400    44967935     1952768   82  Linux swap / Solaris
    /dev/sda4        44969982   160835583    57932801    5  Extended

    Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
    255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000