RoCE Configuration for Cisco switches

Version 2

    This post provides a configuration example for Cisco switches running RoCE over a lossy or a lossless network, in DSCP-based QoS mode.

     

    Notes:

    • Configuration was derived and tested on Cisco 3232C, version 7.0(3)I3(1)

     

    Configuration

    Step 1 - Ingress traffic classification

    class-map type qos match-all CNP

      match dscp 48

    class-map type qos match-all RDMA

      match dscp 26

    policy-map type qos QOS_MARKING

      class RDMA

        set qos-group 3

      class CNP

        set qos-group 6

    Step 2 - Configure QoS policies

    policy-map type network-qos QOS_NETWORK

      class type network-qos c-8q-nq3

        pause pfc-cos 3

        mtu 2240

    policy-map type queuing QOS_QUEUEING

      class type queuing c-out-8q-q3

        random-detect minimum-threshold 150 kbytes maximum-threshold 1500 kbytes drop-probability 100 weight 0 ecn

        bandwidth remaining percent 20

      class type queuing c-out-8q-q6

        priority level 1

    policy-map type queuing INPUT_QOS_QUEUEING

      class type queuing c-in-q3

        queue-limit dynamic 3

    system qos

      service-policy type queuing input INPUT_QOS_QUEUEING

      service-policy type queuing output QOS_QUEUEING

      service-policy type network-qos QOS_NETWORK

    Step 3 - Configure resource allocation

    hardware access-list tcam region e-racl 0

    hardware access-list tcam region vpc-convergence 0

    hardware access-list tcam region racl-lite 768

    hardware access-list tcam region l3qos-intra-lite 0

    hardware access-list tcam region qos 256

    hardware access-list tcam region e-qos 256

    Step 4 - Per port configuration

    interface Ethernet1/1-1/32

      service-policy type qos input QOS_MARKING

    [If lossless] Set PFC

    interface Ethernet1/1-1/32

      priority-flow-control mode on

     

    References