HowTo Configure DCQCN (ECN) Parameters for Windows Driver (WinOF-2)

Version 3

    This post illustrates how to set the DCQCN (ECN) parameters for WinOF-2 driver (ConnectX-4 and ConnectX-5 adapter cards).

    To set DCQCN/ECN parameters on WinOF-2 use Mlx5Cmd.exe tool with QoSConfig flag.

    The output of this tool is taken using WinOF-2 driver v1.60

     

    References

     

    Configuration

    Usage

    PS C:\> Mlx5Cmd.exe -Qosconfig -hh

    Usage:

    Mlx5cmd -QoSConfig -hh | -Name <adapter name> <-DefaultUntaggedPriority | -Dcqcn>

      Manages QoS settings

      Options:

        -hh                         Show this help screen

     

     

        -Name                       <adapter name> - Network adapter name

     

     

        -DefaultUntaggedPriority <-Set <prio> | -Get>

          Configures the port default priority (PCP defined in IEEE 802.1p) that is used for untagged frames.

          Options:

            -Get                    Queries the default priority

            -Set                    <prio> - Priority to set, valid values: [0-7]

                                    Set command is allowed only for the physical adapter

        For example:

          Set priority 3 as default priority:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -DefaultUntaggedPriority -Set 3

     

     

        -Dcqcn <-Enable [prio] | -Disable [prio] | -Restore | -Defaults | -Get | -Set <Parameters...>>

          Configures DCQCN (Data Center QCN), this is allowed only for the physical adapter.

          DCQCN is enabled by default

          Options:

            -Enable                 [prio] Enables congestion control of frames with this priority, valid values: [0-7]

                                    If priority is not provided, congestion control will be enabled for all the priorities

            -Disable                [prio] Disables congestion control of frames with this priority, valid values: [0-7]

                                    If priority is not provided, congestion control will be disabled for all the priorities

            -Restore                Restores the default settings of DCQCN as defined by Mellanox

            -Defaults               Queries congestion control default parameters

            -Get                    Queries congestion control parameters

            -Set                    Sets congestion control parameters, all parameters are shared for all priorities

              Parameters:

                -DcqcnClampTgtRate                If set, whenever a CNP is processed, the target rate will be updated to be the

                                                  current rate (Optional)

                                                  Valid values: [0,1]

                -DcqcnClampTgtRateAfterTimeInc    When receiving a CNP, the target rate will be updated if the transmission

                                                  rate was increased due to the timer, and not only due to the byte counter (Optional)

                                                  Valid values: [0,1]

                -DcqcnCnpDscp                     Changes the priority value on the IP level that can be set for CNPs

                                                  Valid values: [0-7]

                -DcqcnCnpPrioMode                 If this parameter is set to '0', then use DcqcnCnp802pPrio as the priority value (802.1p) on the Ethernet

                                                  header of generated CNPs for this adapter. Otherwise, the priority value of CNPs will be taken from received

                                                  packets that were marked as DCQCN packets (Optional)

                                                  Valid values: [0-1]

                -DcqcnCnp802pPrio                 Changes the priority value (802.1p) on the Ethernet header of generated CNPs for this adapter (Optional)

                                                  Also, set DcqcnCnpPrioMode to '0' in order to use this priority value

                                                  Valid values: [0-7]

                -DcqcnRpgTimeReset                Time between each rate increase if no CNPs are received. Given in useconds (Optional)

                                                  Valid values: [0-131071]

                -DcqcnRpgByteReset                Transmitted data between rate increases if no CNPs are received. Given in 64 Bytes (Optional)

                                                  Valid values: [0-32767]

                -DcqcnRpgThreshold                The number of times rpByteStage or rpTimeStage can count before the RP rate control state

                                                  machine advances states. (Optional)

                                                  Valid values: [1-31]

                -DcqcnRpgAiRate                   The rate, in Mbits per second, used to increase rpTargetRate in the RPR_ACTIVE_INCREASE. (Optional)

                                                  Valid values: [1-line rate]

                -DcqcnRpgHaiRate                  The rate, in Mbits per second, used to increase rpTargetRate in the RPR_HYPER_INCREASE state (Optional)

                                                  Valid values: [1-line rate]

                -DcqcnAlphaToRateShift            Defines the decrement rate for a given alpha (Optional)

                                                  Valid values: [0-11]

                -DcqcnRpgMinDecFac                Defines the maximal ratio of decrease in a single step (Optional)

                                                  Valid values: [0-100]

                -DcqcnRpgMinRate                  In addition to the DcqcnRpgMinDecFac, the DcqcnRpgMinRate parameter defines the minimal rate value

                                                  for the entire single flow. (Optional)

                                                  Note: Setting it to a line rate will disable the congestion control.

                                                  Valid values: [0-line rate]

                -DcqcnRateToSetOnFirstCnp         The rate that is set for the flow when a rate limiter is allocated to it upon the first CNP

                                                  received, in Mbps (Optional)

                                                  Valid values: [0-line rate]

                -DcqcnDceTcpG                     Used to maintain alpha. Expressed in a fixed point fraction of 2^10 (Optional)

                                                  Valid values: [0-1023] (fixed point fraction of 1024)

                -DcqcnDceTcpRtt                   Used to maintain alpha. The time between updates of the alpha value in microseconds (Optional)

                                                  Valid values: [0-131071]

                -DcqcnRateReduceMonitorPeriod     The minimum time between 2 consecutive rate reductions for a single flow. Rate reduction will

                                                  occur only if a CNP is received during the relevant time interval. (Optional)

                                                  Valid values: [0-UINT32]

                -DcqcnInitialAlphaValue           The initial value of alpha to use when receiving the first CNP for a flow. Expressed in a fixed

                                                  point fraction of 2^10. (Optional)

                                                  Valid values: [0-1023] (fixed point fraction of 1024)

        For example:

          Enable DCQCN on priority 3:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Enable 3

          Disable DCQCN on all priorities:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Disable

          Query DCQCN configuration:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Get

          Set DcqcnInitialAlphaValue parameter to 3:

            mlx5cmd.exe -QoSConfig -Name "Ethernet" -Dcqcn -Set -DcqcnInitialAlphaValue 3

     

     

        * For further details about DCQCN parameters, please refer to Mellanox WinOF-2 User Manual.

     

    Defaults

    PS C:\> Mlx5Cmd.exe -Qosconfig -Name R1 -Dcqcn -defaults

    DCQCN default parameters for adapter "R1":

            Paramter                           Default Value

            DcqcnEnablePrio0                   1

            DcqcnEnablePrio1                   1

            DcqcnEnablePrio2                   1

            DcqcnEnablePrio3                   1

            DcqcnEnablePrio4                   1

            DcqcnEnablePrio5                   1

            DcqcnEnablePrio6                   1

            DcqcnEnablePrio7                   1

            DcqcnClampTgtRate                  0

            DcqcnClampTgtRateAfterTimeInc      1

            DcqcnRpgTimeReset                  600

            DcqcnRpgByteReset                  32767

            DcqcnRpgThreshold                  5

            DcqcnRpgAiRate                     5

            DcqcnRpgHaiRate                    50

            DcqcnAlphaToRateShift              11

            DcqcnRpgMinDecFac                  50

            DcqcnRpgMinRate                    1

            DcqcnRateToSetOnFirstCnp           100

            DcqcnDceTcpG                       4

            DcqcnDceTcpRtt                     1

            DcqcnRateReduceMonitorPeriod       4

            DcqcnInitialAlphaValue             0

            DcqcnCnpDscp                       0

            DcqcnCnpPrioMode                   1

            DcqcnCnp802pPrio                   7

     

    The command was executed successfully

     

    Examples

    Most of the parameters are used for debug and tuning and require high understanding of the DC-QCN standard.

    The following parameters are imported to a standard user.

    • DcqcnCnpDscp                 
    • DcqcnCnpPrioMode           
    • DcqcnCnp802pPrio            
    • DcqcnEnablePrio<X>

     

    For detailed information about the parameters, see DC-QCN Parameters

     

    1. To Enable DCQCN on priority 3, and set the DcqcnCnpDSCP on value 6 run the following command:

    PS:\>  Mlx5Cmd.exe -Qosconfig -Name R1 -Dcqcn -Enable 3 -DcqcnCnpDscp 6

    The command was executed successfully

     

    2. To get the current DCQCN configuration, run:

    PS C:\> mlx5cmd.exe -QoSConfig -Name R1 -Dcqcn -Get

    DCQCN RP attributes for adapter "R1":

            DcqcnRPEnablePrio0: 1

            DcqcnRPEnablePrio1: 1

            DcqcnRPEnablePrio2: 1

            DcqcnRPEnablePrio3: 1

            DcqcnRPEnablePrio4: 1

            DcqcnRPEnablePrio5: 1

            DcqcnRPEnablePrio6: 1

            DcqcnRPEnablePrio7: 1

            DcqcnClampTgtRate: 0

            DcqcnClampTgtRateAfterTimeInc: 1

            DcqcnRpgTimeReset: 600

            DcqcnRpgByteReset: 32767

            DcqcnRpgThreshold: 5

            DcqcnRpgAiRate: 5

            DcqcnRpgHaiRate: 50

            DcqcnAlphaToRateShift: 11

            DcqcnRpgMinDecFac: 50

            DcqcnRpgMinRate: 1

            DcqcnRateToSetOnFirstCnp: 100

            DcqcnDceTcpG: 4

            DcqcnDceTcpRtt: 1

            DcqcnRateReduceMonitorPeriod: 4

            DcqcnInitialAlphaValue: 0

     

    DCQCN NP attributes for adapter "R1":

            DcqcnNPEnablePrio0: 1

            DcqcnNPEnablePrio1: 1

            DcqcnNPEnablePrio2: 1

            DcqcnNPEnablePrio3: 1

            DcqcnNPEnablePrio4: 1

            DcqcnNPEnablePrio5: 1

            DcqcnNPEnablePrio6: 1

            DcqcnNPEnablePrio7: 1

            DcqcnCnpDscp: 6

            DcqcnCnpPrioMode: 1

            DcqcnCnp802pPrio: 7

    The command was executed successfully