This post provides a configuration example for Cisco switches running RoCE over a lossy or a lossless network, in DSCP-based QoS mode.

Notes:

• Configuration was derived and tested on Cisco 3232C, version 7.0(3)I3(1)

Configuration

Step 1 - Ingress traffic classification

class-map type qos match-all CNP

  match dscp 48

class-map type qos match-all RDMA

  match dscp 26

policy-map type qos QOS_MARKING

  class RDMA

    set qos-group 3

  class CNP

    set qos-group 6

Step 2 - Configure QoS policies

policy-map type network-qos QOS_NETWORK

  class type network-qos c-8q-nq3

    pause pfc-cos 3

    mtu 2240

policy-map type queuing QOS_QUEUEING

  class type queuing c-out-8q-q3

    random-detect minimum-threshold 150 kbytes maximum-threshold 1500 kbytes drop-probability 100 weight 0 ecn

    bandwidth remaining percent 20

  class type queuing c-out-8q-q6

    priority level 1

policy-map type queuing INPUT_QOS_QUEUEING

  class type queuing c-in-q3

    queue-limit dynamic 3

system qos

  service-policy type queuing input INPUT_QOS_QUEUEING

  service-policy type queuing output QOS_QUEUEING

  service-policy type network-qos QOS_NETWORK

Step 3 - Configure resource allocation

hardware access-list tcam region e-racl 0

hardware access-list tcam region vpc-convergence 0

hardware access-list tcam region racl-lite 768

hardware access-list tcam region l3qos-intra-lite 0

hardware access-list tcam region qos 256

hardware access-list tcam region e-qos 256

Step 4 - Per port configuration

interface Ethernet1/1-1/32

  service-policy type qos input QOS_MARKING

[If lossless] Set PFC

interface Ethernet1/1-1/32

  priority-flow-control mode on

References

• Recommended Network Configuration Examples for RoCE Deployment