Mlx5Cmd.exe

Version 4

    The Mlx5Cmd tool is used to configure the adapter and to collect information utilized by Windows driver (WinOF-2), which supports Mellanox ConnectX-4, ConnectX-4 Lx and ConnectX-5 adapters.

    The output displayed below was obtained via WinOF-2 v1.60 driver.

     

    The main categories are:

     

    References

     

    Configuration Usage

    Usage (Categories)

    PS C:\> Mlx5Cmd.exe -h

    Usage:

    Mlx5cmd -Version | -Trace | -PerfTuning | -Stat | -Mstdump | -QoSConfig | -Sniffer | -RssSniffer |-RegKeys | -LinkSpeed | -help | -h

    Options:

      -help | -h

        Shows this help screen

     

      -Version

        Shows the version of the tool

     

      -Trace [-Start | -Stop] [-FileName <file name>] [-level <number>] [-MaxFileSize <Megabytes>]

        Start/Stop ETW WPP tracing

        Options:

          -Start       starts trace into given file (This is the default)

          -Stop        stops trace into given file

          -FileName    <file name> (default mlx5trace.etl)

          -level       <number> (default 7 - error, warning and info)

                       bit 1 - error, 2 - warning, 3 - info

          -MaxFileSize <Megabytes> (default 16, buffer is circular)

        For example:

          Start tracing: mlx5cmd.exe -Trace -start

          Stop tracing: mlx5cmd.exe -Trace -stop

     

      -PerfTuning -f | -r -c1 <first connection name> [-c2 <second connection name>]

        Configures IPForwarding traffic

        This command is allowed only for a physical adapter

        Options:

          -f           forward traffic scenario

          -r           restores default settings

          -c1          <first connection name>

          -c2          <second connection name>

        For example:

          mlx5cmd.exe -PerfTuning -f -c1 "Local Area Connection 6" -c2 "Local Area Connection 7"

     

      -Stat [-json]

        Shows adapters' attributes

        Options:

          -json          shows output in JSON format. (default is standard format)

          -Indentation   chooses indentation for the output

        For example:

          mlx5cmd.exe -Stat

     

      -Mstdump [-bdf <pci-bus#> <pci-device#> <pci-function#>] | -hh

        Create MSTDUMP files (in %SystemRoot%\temp folder)

        This command is allowed only for a physical adapter

        Options:

          -hh          Show this help screen

          -bdf         <pci-bus#> <pci-device#> <pci-function#> - Physical location. (Optional)

                       If PCI is not provided, the information is shown for all the devices.

                       The PCI information of a device can be found using device manager, in

                       the device's "Properties" dialog, "General" tab, "Location" property.

        For example

          If "Location" is "PCI Slot 3 (PCI bus 8, device 1, function 0)" type:

            mlx5cmd.exe -Mstdump -bdf 8.1.0

     

      -QoSConfig -hh | -Name <adapter name> <-DefaultUntaggedPriority | -Dcqcn>

        Manages QoS settings

        For further details use the following command:

          mlx5cmd.exe -QoSConfig -hh

     

      -Sniffer -name <adapter name> -start [-filename <Filename>] [-buffer_size <Memory buffer size>] | -stop

        Options:

          -Start       starts sniffer

          -Stop        stops sniffer into given file

        Start sniffer

        Options:

          -filename <file name> (default is \%systemroot%\temp\mlx5sniffer.pcap)

          -buffer_size <Memory buffer size> cash buffer size in MB, buffer is circular (default is 100MB)

        For example:

          mlx5cmd.exe -Sniffer -name "Ethernet 3" -start -filename c:\sniffer\trace.pcap

     

      -RssSniffer -name <adapter name> -vport <vPort ID> -start [-filename <Filename>] [-buffer_size <Memory buffer size>] [-lookahead <Packet Lookahead Size>] [-rate <Sampling Rate>] | -stop | -hh    Options:

          -Start       starts non-RSS traffic capture

          -Stop        stops non-RSS traffic capture

          -name        Adapter name

          -vport       vPort ID - mandatory when virtualization enabled only

        Start

        Options:

          -filename <file name> (default is \%systemroot%\temp\mlx5sniffer.pcap)

          -buffer_size <Memory buffer size> cash buffer size in MB, buffer is circular (default is 100MB)

          -lookahead <size in Bytes> number of stored bytes of each sampled packet(default is 40)

          -rate <in packets> 1 / number of packets will be captured(default 1)

        For example:

          mlx5cmd.exe -RssSniffer -name "Ethernet 3" -vport 2 -start -rate 5 -filename c:\sniffer\trace.pcap

     

      -RegKeys -hh | [-bdf <pci-bus#> <pci-device#> <pci-function#>]

        Shows the registry keys that were read by the driver

        Options:

          -hh          Show this help screen

          -bdf         <pci-bus#> <pci-device#> <pci-function#> - Physical location. (Optional)

                       If PCI is not provided, the information will be shown for all the devices.

                       The PCI information of a device can be found using the device manager, in

                       the device's "Properties" dialog, "General" tab, "Location" property.

                       * This option is allowed only for a physical adapter.

        For example

          If "Location" is "PCI Slot 3 (PCI bus 8, device 1, function 0)" type:

            mlx5cmd.exe -RegKeys -bdf 8.1.0

     

      -LinkSpeed -hh -name <adapter name> -Query | -Set <value>

        Options:

          -Query       Display all supported linkspeeds

          -Set         Force linkspeed to specified value

        For example:

          mlx5cmd.exe -LinkSpeed -name "Ethernet 3" -Query

          mlx5cmd.exe -LinkSpeed -name "Ethernet 3" -Set 10

     

    Trace

    PS C:\> Mlx5Cmd.exe -trace -hh

    Usage:

      -Trace [-Start | -Stop] [-FileName <file name>] [-level <number>] [-MaxFileSize <Megabytes>]

        Start/Stop ETW WPP tracing

        Options:

          -Start       starts trace into given file (This is the default)

          -Stop        stops trace into given file

          -FileName    <file name> (default mlx5trace.etl)

          -level       <number> (default 7 - error, warning and info)

                       bit 1 - error, 2 - warning, 3 - info

          -MaxFileSize <Megabytes> (default 16, buffer is circular)

        For example:

          Start tracing: mlx5cmd.exe -Trace -start

          Stop tracing: mlx5cmd.exe -Trace -stop

     

    PerfTuning

     

    PS C:\> Mlx5Cmd.exe -perftuning -hh

     

    Usage of program for WinOF-2 driver:

     

    -f/-r  Forwarding traffic scenario / Restore default settings.

           This option must be followed by two connection names, the adapters must have the same configuration.

           This option automatically sets:

              2 RSS cores, one for recieve core and the other for send

    -c1    First connection name

    -c2    Second connection name

    -info  Query current adapters mode

    Usage example:

     

    For example, if the adapter is represented by "Local Area Connection 6" and "Local Area Connection 7"

    -perftuning -PerfTuning -f -c1 "Local Area Connection 6" -c2 "Local Area Connection 7"

     

    * This command is allowed only for a physical adapter

     

    Stat

    PS C:\> Mlx5Cmd.exe -stat -hh

    Usage: MlxStat [-Help | h | hh | ?] [-json] [-Indentation <num>]

            -h | h | hh | Help       - shows help

            -json                    - shows output in JSON format. default is standard format

            -Indentation             - chooses indentation for the output

            -verbose                 - verbose output

    MstDump

    PS C:\> Mlx5Cmd.exe -mstdump -hh

    Usage:

    Mlx5cmd -Mstdump [-bdf <pci-bus#> <pci-device#> <pci-function#>] | -hh

      Create MSTDUMP files (in %SystemRoot%\temp folder)

      This command is allowed only for a physical adapter

      Options:

        -hh          Show this help screen

        -bdf         <pci-bus#> <pci-device#> <pci-function#> - Physical location. (Optional)

                     If PCI is not provided, the information will be shown for all the devices.

                     The PCI information of a device can be found using the device manager, in

                     the device's "Properties" dialog, "General" tab, "Location" property.

      For example

        If "Location" is "PCI Slot 3 (PCI bus 8, device 1, function 0)" type:

          mlx5cmd.exe -Mstdump -bdf 8.1.0

     

    QoSConfig (DCQCN/ECN)

    For configuration example, see HowTo Configure DCQCN (ECN) Parameters for Windows Driver (WinOF-2) .

    PS C:\> Mlx5Cmd.exe -Qosconfig -hh

    Usage:

    Mlx5cmd -QoSConfig -hh | -Name <adapter name> <-DefaultUntaggedPriority | -Dcqcn>

      Manages QoS settings

      Options:

        -hh                         Show this help screen

     

        -Name                       <adapter name> - Network adapter name

     

        -DefaultUntaggedPriority <-Set <prio> | -Get>

          Configures the port default priority (PCP defined in IEEE 802.1p) that is used for untagged frames.

          Options:

            -Get                    Queries the default priority

            -Set                    <prio> - Priority to set, valid values: [0-7]

                                    Set command is allowed only for the physical adapter

        For example:

          Set priority 3 as default priority:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -DefaultUntaggedPriority -Set 3

     

        -Dcqcn <-Enable [prio] | -Disable [prio] | -Restore | -Defaults | -Get | -Set <Parameters...>>

          Configures DCQCN (Data Center QCN), this is allowed only for the physical adapter.

          DCQCN is enabled by default

          Options:

            -Enable                 [prio] Enables congestion control of frames with this priority, valid values: [0-7]

                                    If priority is not provided, congestion control will be enabled for all the priorities

            -Disable                [prio] Disables congestion control of frames with this priority, valid values: [0-7]

                                    If priority is not provided, congestion control will be disabled for all the priorities

            -Restore                Restores the default settings of DCQCN as defined by Mellanox

            -Defaults               Queries congestion control default parameters

            -Get                    Queries congestion control parameters

            -Set                    Sets congestion control parameters, all parameters are shared for all priorities

              Parameters:

                -DcqcnClampTgtRate                If set, whenever a CNP is processed, the target rate will be updated to be the

                                                  current rate (Optional)

                                                  Valid values: [0,1]

                -DcqcnClampTgtRateAfterTimeInc    When receiving a CNP, the target rate will be updated if the transmission

                                                  rate was increased due to the timer, and not only due to the byte counter (Optional)

                                                  Valid values: [0,1]

                -DcqcnCnpDscp                     Changes the priority value on the IP level that can be set for CNPs

                                                  Valid values: [0-7]

                -DcqcnCnpPrioMode                 If this parameter is set to '0', then use DcqcnCnp802pPrio as the priority value (802.1p) on the Ethernet

                                                  header of generated CNPs for this adapter. Otherwise, the priority value of CNPs will be taken from received

                                                  packets that were marked as DCQCN packets (Optional)

                                                  Valid values: [0-1]

                -DcqcnCnp802pPrio                 Changes the priority value (802.1p) on the Ethernet header of generated CNPs for this adapter (Optional)

                                                  Also, set DcqcnCnpPrioMode to '0' in order to use this priority value

                                                  Valid values: [0-7]

                -DcqcnRpgTimeReset                Time between each rate increase if no CNPs are received. Given in useconds (Optional)

                                                  Valid values: [0-131071]

                -DcqcnRpgByteReset                Transmitted data between rate increases if no CNPs are received. Given in 64 Bytes (Optional)

                                                  Valid values: [0-32767]

                -DcqcnRpgThreshold                The number of times rpByteStage or rpTimeStage can count before the RP rate control state

                                                  machine advances states. (Optional)

                                                  Valid values: [1-31]

                -DcqcnRpgAiRate                   The rate, in Mbits per second, used to increase rpTargetRate in the RPR_ACTIVE_INCREASE. (Optional)

                                                  Valid values: [1-line rate]

                -DcqcnRpgHaiRate                  The rate, in Mbits per second, used to increase rpTargetRate in the RPR_HYPER_INCREASE state (Optional)

                                                  Valid values: [1-line rate]

                -DcqcnAlphaToRateShift            Defines the decrement rate for a given alpha (Optional)

                                                  Valid values: [0-11]

                -DcqcnRpgMinDecFac                Defines the maximal ratio of decrease in a single step (Optional)

                                                  Valid values: [0-100]

                -DcqcnRpgMinRate                  In addition to the DcqcnRpgMinDecFac, the DcqcnRpgMinRate parameter defines the minimal rate value

                                                  for the entire single flow. (Optional)

                                                  Note: Setting it to a line rate will disable the congestion control.

                                                  Valid values: [0-line rate]

                -DcqcnRateToSetOnFirstCnp         The rate that is set for the flow when a rate limiter is allocated to it upon the first CNP

                                                  received, in Mbps (Optional)

                                                  Valid values: [0-line rate]

                -DcqcnDceTcpG                     Used to maintain alpha. Expressed in a fixed point fraction of 2^10 (Optional)

                                                  Valid values: [0-1023] (fixed point fraction of 1024)

                -DcqcnDceTcpRtt                   Used to maintain alpha. The time between updates of the alpha value in microseconds (Optional)

                                                  Valid values: [0-131071]

                -DcqcnRateReduceMonitorPeriod     The minimum time between 2 consecutive rate reductions for a single flow. Rate reduction will

                                                  occur only if a CNP is received during the relevant time interval. (Optional)

                                                  Valid values: [0-UINT32]

                -DcqcnInitialAlphaValue           The initial value of alpha to use when receiving the first CNP for a flow. Expressed in a fixed

                                                  point fraction of 2^10. (Optional)

                                                  Valid values: [0-1023] (fixed point fraction of 1024)

        For example:

          Enable DCQCN on priority 3:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Enable 3

          Disable DCQCN on all priorities:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Disable

          Query DCQCN configuration:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Get

          Set DcqcnInitialAlphaValue parameter to 3:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Set -DcqcnInitialAlphaValue 3

     

     

        * For further details about DCQCN parameters, please refer to Mellanox WinOF-2 User Manual.

     

    Sniffer

    PS C:\> Mlx5Cmd.exe -sniffer -hh

    Usage:

    Mlx5cmd -Sniffer -hh | -Name <adapter name> <-start [-filename <Filename>] [-buffer_size <Memory buffer size>] | -stop>

      Manages Sniffer

      Options:

        -hh                            Show this help screen

     

        -Name                          <adapter name> - Network adapter name

     

        -start                         Start capture

          Options:

            -filename <Filename>       .pcap file name, default is %systemroot%\temp\mlx5sniffer.pcap

            -buffer_size <size in MB>  the cyclic buffer size for captured data temporary storage until saved to file, default is 100MB. Increase this size for heavy traffic captures

        -stop                          Stop capture

     

        For example:

          Capture "Ethernet 2" interface traffic to eth2.pcap file :

          mlx5cmd.exe -Sniffer -Name "Ethernet 2" -start -filename .\eth2.pcap

     

    For example, see HowTo Capture RDMA traffic on mlx5 driver using mlx5cmd (Windows) .

     

    RSS Sniffer

    PS C:\> Mlx5Cmd.exe -rsssniffer -hh

    Usage:

    Mlx5cmd -RssSniffer -hh | -Name <adapter name>  -vport <vPort ID> <-start [-filename <Filename>] [-buffer_size <Memory buffer size>] [-lookahead <Packet Lookahead Size>] [-rate <Sampling Rate>] | -stop>

      Manages Sniffer

      Options:

        -hh                            Show this help screen

        -Name                          <adapter name> - Network adapter name

        -vport                         <vPort ID> - vPort number mandatory when virtualization enabled only

        -start                         Start non-RSS traffic capture

          Options:

            -filename <Filename>       .pcap file name, default is %systemroot%\temp\mlx5sniffer.pcap

            -buffer_size <size in MB>  the cyclic buffer size for captured data temporary storage until saved to file,                                   default is 100MB

                                                                                                    Increase this size for heavy traffic captures

            -lookahead <size in Bytes> number of stored bytes of each sampled packet(default is 40)

            -rate <in packets>         1 / number of packets will be captured (default is 1)

     

        -stop                          Stop non-RSS traffic capture

     

        For example:

          Capture  every 5-th packet of non-RSS traffic on "Ethernet 2" interface to eth2-vport001.pcap file :

          mlx5cmd.exe -RssSniffer -Name "Ethernet 2" -vport 1 -start -rate 5 -filename .\eth2-vport001.pcap

     

     

    Registry keys

    PS C:\> Mlx5Cmd.exe -regkeys -hh

    Usage:

    Mlx5cmd -RegKeys [-bdf <pci-bus#> <pci-device#> <pci-function#>] | -hh

      Shows the registry keys that were read by the driver

      Options:

        -hh          Show this help screen

        -bdf         <pci-bus#> <pci-device#> <pci-function#> - Physical location. (Optional)

                     If PCI is not provided, the information will be shown for all the devices.

                     The PCI information of a device can be found using the device manager, in

                     the device's "Properties" dialog, "General" tab, "Location" property.

                     * This option is allowed only for a physical adapter.

      For example

        If "Location" is "PCI Slot 3 (PCI bus 8, device 1, function 0)" type:

          mlx5cmd.exe -RegKeys -bdf 8.1.0

     

    PS C:\> Mlx5Cmd.exe -linkspeed -hh

    Usage:

    Mlx5cmd -LinkSpeed -hh | -Name <adapter name> | -Query | -Set <value>>

        For example:

          Query supported linkspeeds by adapter:

            mlx5cmd.exe -LinkSpeed -Name "Ethernet" -Query

          Force Linkspeed to 10Gbps:

            mlx5cmd.exe -LinkSpeed -Name "Ethernet" -Set 10

          Force Linkspeed to be autonegotiated:

            mlx5cmd.exe -LinkSpeed -Name "Ethernet" -Set 0