mlx_fs_dump

Version 4

    This post describes the mlx_fs_dump Python tool. This tool dumps the steering rules in a readable, easy-to-understand manner.

    This tool is available starting with MLNX_OFED rev. 4.1 and above for mlx5 drivers (for ConnectX-4 and above).

     

    References

     

    Prerequisites

    • Python 2.7 or above + pip, anytree and termcolor libraries
    • The latest Mellanox Firmware Tool (MFT) package

     

    When using this tool for the first time, you will need to verify that the two Python libraries (anytree and termcolor) are installed. Because it depends on Python configurations, users should install the packages by running commands similar to these shown below:

    # yum install yum install python2-pip.noarch

    # pip install anytree

    # pip install termcolor

     

    Basic Usage

    1. Start mst and get the device ID using mst status.

    # mst start

    ...

    # mst status -v

    MST modules:

    ------------

        MST PCI module loaded

        MST PCI configuration module loaded

    PCI devices:

    ------------

    DEVICE_TYPE             MST                           PCI       RDMA            NET                       NUMA 

    ConnectX5(rev:0)        /dev/mst/mt4119_pciconf0.1    81:00.1   mlx5_3          net-ens801f1              1    

    ConnectX5(rev:0)        /dev/mst/mt4119_pciconf0      81:00.0   mlx5_2          net-ens801f0              1    

    ConnectX4(rev:0)        /dev/mst/mt4115_pciconf0.1    05:00.1   mlx5_1          net-eth2                  0    

    ConnectX4(rev:0)        /dev/mst/mt4115_pciconf0      05:00.0   mlx5_0          net-eth1                  0    

    ConnectX3Pro(rev:0)     /dev/mst/mt4103_pciconf0     

    ConnectX3Pro(rev:0)     /dev/mst/mt4103_pci_cr0       82:00.0   mlx4_0          net-ib0,net-ib1           1    

     

    2. The system is now ready to run the script. The script can also display brief help information if you include the -h option.

    # ./ofed_scripts/utils/mlx_fs_dump -h

    usage: mlx_fs_dump [-h] -d DEVICE [-g GVMI] [-c COLOR] [-f IN_FILE]

     

    -----------------------------------------------------------------

    Dump flow table in a fancy way.

     

    Package/Tool required:

      - mst

      - python packages (anytree, termcolor)

    -----------------------------------------------------------------

     

    optional arguments:

      -h, --help  show this help message and exit

      -d DEVICE   Device name from #mst status, currently supports ConnectX-4 (/dev/mst/mt4115_pciconf) and ConnectX-4 LX (/dev/mst/mt4117_pciconf)

      -g GVMI     Gvmi number

      -c COLOR    Print with color

      -f IN_FILE  Parse a file from mlxdump

     

    GVMI Number:

    First, GVMIs are all the Physical Functions (PFs), then the Virtual functions (VFs) of PF0, VFs of PF1, etc. For example, if we have two PFs (physical function per port) and two VFs per PF, so:

    ------------------------PFs----------------------------

    Gvmi0 = host0.pf0  , port 1 --> vport 0

    Gvmi1 = host0.pf1  , port 2 --> vport 0

     

    ------------------------VFs----------------------------

    Gvmi2 = host0.pf0.vf0  , port 0 --> vport 1

    Gvmi3 = host0.pf0.vf1  , port 0 --> vport 2

    Gvmi4 = host0.pf1.vf0  , port 1 --> vport 1

    Gvmi5 = host0.pf1.vf1  , port 1 --> vport 2

     

    3. By performing a simple run you will see the flow steering rules tree. It will display the flow tables with color highlights for gvmi 0 as default (-g). From the output, we you can see that this device has eight flow tables (FT). Let us look at flow table 8 (FT: 8) as example. It is a flow table that belongs to NIC_RX, and has three flow groups (FG: 0x12, 0x13, 0x14) with the type designated as MISC. Flow group 0x13 has one flow table entry (FTE 0x1000), which will forward (FWD) the traffic to flow table 7, and so on.

    # ./ofed_scripts/utils/mlx_fs_dump -d /dev/mst/mt4119_pciconf0.1

    FT: 8 (level: 0x1f, type: NIC_RX)

    |-- FG: 0x12 (MISC)

    |-- FG: 0x13 (MISC)

    |   +-- FTE: 0x1000 (FWD) to (FLOW_TABLE:0x7)

    |       +-- FT: 7 (level: 0x20, type: NIC_RX)

    |           |-- FG: 0xf (OUT_HDR)

    |           |-- FG: 0x10 (MISC)

    |           +-- FG: 0x11 (MISC)

    +-- FG: 0x14 (MISC)

    FT: 6 (level: 0x22, type: NIC_RX)

    |-- FG: 0xc ()

    |   |-- FTE: 0x0 (FWD) to (TIR:0x6b) in.ethtype:IPv4 in.ip_prot:TCP

    |   |-- FTE: 0x1 (FWD) to (TIR:0x6d) in.ethtype:IPv6 in.ip_prot:TCP

    |   |-- FTE: 0x2 (FWD) to (TIR:0x6f) in.ethtype:IPv4 in.ip_prot:UDP

    |   |-- FTE: 0x3 (FWD) to (TIR:0x71) in.ethtype:IPv6 in.ip_prot:UDP

    |   |-- FTE: 0x4 (FWD) to (TIR:0x73) in.ethtype:IPv4 in.ip_prot:AH

    |   |-- FTE: 0x5 (FWD) to (TIR:0x75) in.ethtype:IPv6 in.ip_prot:AH

    |   |-- FTE: 0x6 (FWD) to (TIR:0x77) in.ethtype:IPv4 in.ip_prot:ESP

    |   +-- FTE: 0x7 (FWD) to (TIR:0x79) in.ethtype:IPv6 in.ip_prot:ESP

    |-- FG: 0xd ()

    |   |-- FTE: 0x8 (FWD) to (TIR:0x7b) in.ethtype:IPv4

    |   +-- FTE: 0x9 (FWD) to (TIR:0x7d) in.ethtype:IPv6

    +-- FG: 0xe (OUT_HDR)

        +-- FTE: 0xa (FWD) to (TIR:0x7e)

    FT: 5 (level: 0x21, type: NIC_RX)

    |-- FG: 0x8 (MISC)

    |-- FG: 0x9 (MISC)

    |   |-- FTE: 0x10 (FWD) to (TIR:0x6a) out.ethtype:IPv4 out.ip_prot:TCP

    |   |-- FTE: 0x11 (FWD) to (TIR:0x6c) out.ethtype:IPv6 out.ip_prot:TCP

    |   |-- FTE: 0x12 (FWD) to (TIR:0x6e) out.ethtype:IPv4 out.ip_prot:UDP

    |   |-- FTE: 0x13 (FWD) to (TIR:0x70) out.ethtype:IPv6 out.ip_prot:UDP

    |   |-- FTE: 0x14 (FWD) to (TIR:0x72) out.ethtype:IPv4 out.ip_prot:AH

    |   |-- FTE: 0x15 (FWD) to (TIR:0x74) out.ethtype:IPv6 out.ip_prot:AH

    |   |-- FTE: 0x16 (FWD) to (TIR:0x76) out.ethtype:IPv4 out.ip_prot:ESP

    |   +-- FTE: 0x17 (FWD) to (TIR:0x78) out.ethtype:IPv6 out.ip_prot:ESP

    |-- FG: 0xa (MISC)

    |   |-- FTE: 0x18 (FWD) to (TIR:0x7a) out.ethtype:IPv4

    |   +-- FTE: 0x19 (FWD) to (TIR:0x7c) out.ethtype:IPv6

    +-- FG: 0xb (OUT_HDR)

        +-- FTE: 0x1a (FWD) to (TIR:0x7e)

    FT: 4 (level: 0x23, type: NIC_RX)

    |-- FG: 0x6 (MISC)

    +-- FG: 0x7 (OUT_HDR)

        +-- FTE: 0x1000 (FWD) to (TIR:0x70)

    FT: 3 (level: 0x23, type: NIC_RX)

    |-- FG: 0x4 (MISC)

    +-- FG: 0x5 (OUT_HDR)

        +-- FTE: 0x1000 (FWD) to (TIR:0x6e)

    FT: 2 (level: 0x23, type: NIC_RX)

    |-- FG: 0x2 (MISC)

    +-- FG: 0x3 (OUT_HDR)

        +-- FTE: 0x1000 (FWD) to (TIR:0x6c)

    FT: 1 (level: 0x23, type: NIC_RX)

    |-- FG: 0x0 (MISC)

    +-- FG: 0x1 (OUT_HDR)

        +-- FTE: 0x1000 (FWD) to (TIR:0x6a)

    FT: 0 (level: 0x25, type: NIC_RX)

     

    Example

    Here is another example with a Precision Time Protocol (PTP) flow steering configuration.

    Configure PTP steering flows (see HowTo Steer PTP traffic to single RX ring (via ethtool) for more details).

     

    1. Get the ring configuration.

    # ethtool -u eth1

    56 RX rings available

    Total 0 rules

     

    2. Create rules - one for each of the UDP ports.

    # ethtool -U eth1 flow-type udp4 dst-port 319 action 0 loc 1

    # ethtool -U eth1 flow-type udp4 dst-port 320 action 0 loc 2

     

    3. Check the rules.

    # ethtool -u eth1

    56 RX rings available

    Total 2 rules

     

     

    Filter: 1

      Rule Type: UDP over IPv4

      Src IP addr: 0.0.0.0 mask: 255.255.255.255

      Dest IP addr: 0.0.0.0 mask: 255.255.255.255

      TOS: 0x0 mask: 0xff

      Src port: 0 mask: 0xffff

      Dest port: 320 mask: 0x0

      Action: Direct to queue 0

     

    Filter: 2

      Rule Type: UDP over IPv4

      Src IP addr: 0.0.0.0 mask: 255.255.255.255

      Dest IP addr: 0.0.0.0 mask: 255.255.255.255

      TOS: 0x0 mask: 0xff

      Src port: 0 mask: 0xffff

      Dest port: 319 mask: 0x0

      Action: Direct to queue 0

     

    4. Run the mlx_fs_dump script to see the rules.

    ./ofed_scripts/utils/mlx_fs_dump -d /dev/mst/mt4115_pciconf0

    FT: 9 (level: 0x18, type: NIC_RX)

    +-- FG: 0x15 (MISC)

        |-- FTE: 0x0 (FWD) to (TIR:0x7e) out.ethtype:IPv4 out.ip_prot:UDP out.udp_dport:0x140

        +-- FTE: 0x1 (FWD) to (TIR:0x7e) out.ethtype:IPv4 out.ip_prot:UDP out.udp_dport:0x13f

    ...