HowTo Configure BGP Unnumbered on Cumulus Linux

Version 5

    This post discusses the Border Gateway Protocol (BGP) unnumbered and describes the configuration procedures for the Mellanox Spectrum installed with Cumulus Linux OS.

     

    References

     

    BGP Unnumbered

    Unnumbered L3 interfaces are interfaces without unique IP addresses. In BGP, you configure unnumbered interfaces using Extended Next Hop Encoding (ENHE), which is defined by RFC 5549. BGP unnumbered interfaces provide a means of advertising an IPv4 route with an IPv6 next-hop. Prior to RFC 5549, an IPv4 route could be advertised only with an IPv4 next-hop.

    BGP unnumbered interfaces are particularly useful in deployments where IPv4 prefixes are advertised through BGP over a section without any IPv4 address configuration on links. As a result, the routing entries are also IPv4 for destination lookup and have IPv6 next-hops for forwarding purposes.

     

    Once enabled and active, BGP makes use of the available IPv6 next-hops for advertising any IPv4 prefixes. BGP learns the prefixes, calculates the routes, and installs them in IPv4 AFI-to-IPv6 AFI format. However, ENHE in a Mellanox switch (running Cumulus Linux) does not install routes into the kernel in IPv4 prefix-to-IPv6 next-hop format. For link-local peering enabled by dynamically learning the other end's link-local address using IPv6 neighbor discovery router advertisements, an IPv6 next-hop is converted into an IPv4 link-local address and a static neighbor entry is installed for this IPv4 link-local address with the MAC address derived from the link-local address of the other end.

     

    Setup

    Configuration

     

    Switch - 1

     

    1. Enable the IPv6 neighbor discovery.

    # net add interface swp1 ipv6 nd ra-interval 5

    # net del interface swp1 ipv6 nd suppress-ra

     

    2. Create a loopback interface (router ID).

    # net add loopback lo ip address 1.1.1.1/32

     

    3. Create a loopback interface for adding a network, which can be advertised over the BGP connection.

    # net add loopback lo ip address 10.10.10.1/32

     

    4. Configure BGP with an Autonomous System (AS) number of 65000.

    # net add bgp autonomous-system 65000

    # net add bgp router-id 1.1.1.1

     

    5. Configure BGP unnumbered with “group name as Fabric”.

    # net add bgp bestpath as-path multipath-relax

    # net add bgp bestpath compare-routerid

    # net add bgp neighbor fabric peer-group

    # net add bgp neighbor fabric remote-as external

    # net add bgp neighbor fabric description Internal Fabric Network

    # net add bgp neighbor fabric capability extended-nexthop

    # net add bgp neighbor swp1 interface peer-group fabric

     

    6. Advertise a second loopback network.

    # net add bgp ipv4 unicast network 10.10.10.1/32

     

    7. Activate the group.

    # net add bgp ipv6 unicast neighbor fabric activate

     

    Switch – 2

     

    1. Enable an IPv6 neighbor discovery.

    # net add interface swp1 ipv6 nd ra-interval 5

    # net del interface swp1 ipv6 nd suppress-ra

     

    2. Create a loopback interface for the router ID.

    # net add loopback lo ip address 2.2.2.2/32

     

    3. Create a loopback for adding a network, which can be advertised over the BGP connection.

    # net add loopback lo ip address 20.20.20.1/32

     

    4. Configure BGP with an AS number of 65002.

    # net add bgp autonomous-system 65002

    # net add bgp router-id 2.2.2.2

     

    5. Configure the BGP unnumbered with “group name as Fabric”.

    # net add bgp bestpath as-path multipath-relax

    # net add bgp bestpath compare-routerid

    # net add bgp neighbor fabric peer-group

    # net add bgp neighbor fabric remote-as external

    # net add bgp neighbor fabric description Internal Fabric Network

    # net add bgp neighbor fabric capability extended-nexthop

    # net add bgp neighbor swp1 interface peer-group fabric

     

    6. Advertise a second loopback network.

    # net add bgp ipv4 unicast network 20.20.20.1/32

     

    7. Activate the group.

    # net add bgp ipv6 unicast neighbor fabric activate

     

    Verification

     

    Use the following show commands to validate the setup.

     

     

    1. Show a BGP neighbor on Switch - 1.

    cumulus@cumulus:~$ net show bgp neighbor swp1

    BGP neighbor on swp1: fe80::7efe:90ff:fefc:7bd8, remote AS 65002, local AS 65000, external link >> Neighbor learned on SWP1 with its IPv6 link local address

    Hostname: cumulus

    Member of peer-group fabric for session parameters

      BGP version 4, remote router ID 2.2.2.2

      BGP state = Established, up for 00:03:17

      Last read 00:00:02, Last write 00:01:07

      Hold time is 9, keepalive interval is 3 seconds

      Neighbor capabilities:

        4 Byte AS: advertised and received

        AddPath:

          IPv4 Unicast: RX advertised IPv4 Unicast and received

          IPv6 Unicast: RX advertised IPv6 Unicast and received

        Extended nexthop: advertised and received

          Address families by peer:

                       IPv4 Unicast

        Route refresh: advertised and received(old & new)

        Address family IPv4 Unicast: advertised and received

        Address family IPv6 Unicast: advertised and received

        Hostname Capability: advertised and received

        Graceful Restart Capabilty: advertised and received

          Remote Restart timer is 120 seconds

          Address families by peer:

            none

      Graceful restart informations:

        End-of-RIB send: IPv4 Unicast, IPv6 Unicast

        End-of-RIB received: IPv4 Unicast, IPv6 Unicast

      Message statistics:

        Inq depth is 0

        Outq depth is 0

                             Sent       Rcvd

        Opens:                  7          2

        Notifications:          0          0

        Updates:               10         10

        Keepalives:           762        760

        Route Refresh:          0          0

        Capability:             0          0

        Total:                779        772

      Minimum time between advertisement runs is 0 seconds

     

    For address family: IPv4 Unicast

      fabric peer-group member

      Update group 3, subgroup 3

      Packet Queue length 0

      Community attribute sent to this neighbor(both)

      1 accepted prefixes

     

    For address family: IPv6 Unicast

      fabric peer-group member

      Update group 4, subgroup 4

      Packet Queue length 0

      Community attribute sent to this neighbor(both)

      0 accepted prefixes

     

      Connections established 2; dropped 1

      Last reset 00:04:07, due to Interface down

    Local host: fe80::7efe:90ff:fefc:7ad8, Local port: 55204

    Foreign host: fe80::7efe:90ff:fefc:7bd8, Foreign port: 179

    Nexthop: 1.1.1.1

    Nexthop global: fe80::7efe:90ff:fefc:7ad8

    Nexthop local: fe80::7efe:90ff:fefc:7ad8

    BGP connection: shared network

    BGP Connect Retry Timer in Seconds: 10

    Read thread: on  Write thread: off

     

    cumulus@cumulus:~$

     

    2. "Show" a BGP neighbor on Switch - 2.

    cumulus@cumulus:~$ net show bgp neighbor swp1

    BGP neighbor on swp1: fe80::7efe:90ff:fefc:7ad8, remote AS 65000, local AS 65002, external link >> Neighbor learned on SWP1 with its IPv6 link local address

    Hostname: cumulus

    Member of peer-group fabric for session parameters

      BGP version 4, remote router ID 1.1.1.1

      BGP state = Established, up for 00:04:46

      Last read 00:00:01, Last write 00:02:36

      Hold time is 9, keepalive interval is 3 seconds

      Neighbor capabilities:

        4 Byte AS: advertised and received

        AddPath:

          IPv4 Unicast: RX advertised IPv4 Unicast and received

          IPv6 Unicast: RX advertised IPv6 Unicast and received

        Extended nexthop: advertised and received

          Address families by peer:

                       IPv4 Unicast

        Route refresh: advertised and received(old & new)

        Address family IPv4 Unicast: advertised and received

        Address family IPv6 Unicast: advertised and received

        Hostname Capability: advertised and received

        Graceful Restart Capabilty: advertised and received

          Remote Restart timer is 120 seconds

          Address families by peer:

            none

      Graceful restart informations:

        End-of-RIB send: IPv4 Unicast, IPv6 Unicast

        End-of-RIB received: IPv4 Unicast, IPv6 Unicast

      Message statistics:

        Inq depth is 0

        Outq depth is 0

                             Sent       Rcvd

        Opens:                  2          2

        Notifications:          0          0

        Updates:               10         10

        Keepalives:           792        790

        Route Refresh:          0          0

        Capability:             0          0

        Total:                804        802

      Minimum time between advertisement runs is 0 seconds

     

    For address family: IPv4 Unicast

      fabric peer-group member

      Update group 3, subgroup 3

      Packet Queue length 0

      Community attribute sent to this neighbor(both)

      1 accepted prefixes

     

    For address family: IPv6 Unicast

      fabric peer-group member

      Update group 4, subgroup 4

      Packet Queue length 0

      Community attribute sent to this neighbor(both)

      0 accepted prefixes

     

      Connections established 2; dropped 1

      Last reset 00:05:36, due to Interface down

    Local host: fe80::7efe:90ff:fefc:7bd8, Local port: 179

    Foreign host: fe80::7efe:90ff:fefc:7ad8, Foreign port: 55204

    Nexthop: 2.2.2.2

    Nexthop global: fe80::7efe:90ff:fefc:7bd8

    Nexthop local: fe80::7efe:90ff:fefc:7bd8

    BGP connection: shared network

    BGP Connect Retry Timer in Seconds: 10

    Read thread: on  Write thread: off

     

    cumulus@cumulus:~$

     

    3. Show an IP route on Switch - 1.

    cumulus@cumulus:~$ net show route

     

    show ip route

    =============

    Codes: K - kernel route, C - connected, S - static, R - RIP,

           O - OSPF, I - IS-IS, B - BGP, P - PIM, T - Table, v - VNC,

           V - VPN,

           > - selected route, * - FIB route

     

     

    K>* 0.0.0.0/0 via 10.20.0.251, eth0

    C>* 1.1.1.1/32 is directly connected, lo

    C>* 10.10.10.1/32 is directly connected, lo

    C>* 10.20.0.0/16 is directly connected, eth0

    B>* 20.20.20.1/32 [20/0] via fe80::7efe:90ff:fefc:7bd8, swp1, 00:03:37 >> Network 20.20.20.1 of Switch - 2 learned via swp1 on Switch - 1

     

    show ipv6 route

    ===============

    Codes: K - kernel route, C - connected, S - static, R - RIPng,

           O - OSPFv6, I - IS-IS, B - BGP, T - Table, v - VNC,

           V - VPN,

           > - selected route, * - FIB route

     

     

    C * fe80::/64 is directly connected, swp1

    C * fe80::/64 is directly connected, swid0_eth

    C>* fe80::/64 is directly connected, eth0

    cumulus@cumulus:~$

     

    4. Show a route on Switch - 2.

    cumulus@cumulus:~$ net show route

     

    show ip route

    =============

    Codes: K - kernel route, C - connected, S - static, R - RIP,

           O - OSPF, I - IS-IS, B - BGP, P - PIM, T - Table, v - VNC,

           V - VPN,

           > - selected route, * - FIB route

     

    K>* 0.0.0.0/0 via 10.20.0.251, eth0

    C>* 2.2.2.2/32 is directly connected, lo

    B>* 10.10.10.1/32 [20/0] via fe80::7efe:90ff:fefc:7ad8, swp1, 00:05:16 >> Network 10.10.10.1 of Switch - 1 learned via swp1 on Switch - 2

    C>* 10.20.0.0/16 is directly connected, eth0

    C>* 20.20.20.1/32 is directly connected, lo

     

    show ipv6 route

    ===============

    Codes: K - kernel route, C - connected, S - static, R - RIPng,

           O - OSPFv6, I - IS-IS, B - BGP, T - Table, v - VNC,

           V - VPN,

           > - selected route, * - FIB route

     

    C * fe80::/64 is directly connected, swp1

    C * fe80::/64 is directly connected, swid0_eth

    C * fe80::/64 is directly connected, swp12

    C>* fe80::/64 is directly connected, eth0

    cumulus@cumulus:~$

     

    5. Show a run of Switch - 1.

    cumulus@cumulus:~$ net show configuration

     

    hostname cumulus

     

    interface lo

      address 1.1.1.1/32

      address 10.10.10.1/32

     

    interface eth0

      address dhcp

     

    interface swp1

      ipv6 nd ra-interval 5

      no ipv6 nd suppress-ra

     

    interface swp12

      link-speed 40000

     

    username cumulus nopassword

     

    service integrated-vtysh-config

     

    log file /var/log/quagga/quagga.log

     

    log timestamp precision 6

     

    router bgp 65000

      bgp router-id 1.1.1.1

      bgp bestpath as-path multipath-relax

      bgp bestpath compare-routerid

      neighbor fabric peer-group

      neighbor fabric remote-as external

      neighbor fabric description Internal Fabric Network

      neighbor fabric capability extended-nexthop

      neighbor swp1 interface peer-group fabric

      neighbor swp12 interface peer-group fabric

     

      address-family ipv4 unicast

        network 10.10.10.1/32

     

      address-family ipv6 unicast

        neighbor fabric activate

     

    line vty

    cumulus@cumulus:~$

     

    6. Show a run of switch - 2.

    cumulus@cumulus:~$ net show configuration

     

    hostname cumulus

     

    interface lo

      address 2.2.2.2/32

      address 20.20.20.1/32

     

    interface eth0

      address dhcp

     

    interface swp1

      ipv6 nd ra-interval 5

      no ipv6 nd suppress-ra

     

    interface swp12

      link-speed 40000

     

    username cumulus nopassword

     

    service integrated-vtysh-config

     

    log file /var/log/quagga/quagga.log

     

    log timestamp precision 6

     

    router bgp 65002

      bgp router-id 2.2.2.2

      bgp bestpath as-path multipath-relax

      bgp bestpath compare-routerid

      neighbor fabric peer-group

      neighbor fabric remote-as external

      neighbor fabric description Internal Fabric Network

      neighbor fabric capability extended-nexthop

      neighbor swp1 interface peer-group fabric

      neighbor swp12 interface peer-group fabric

     

      address-family ipv4 unicast

        network 20.20.20.1/32

     

      address-family ipv6 unicast

        neighbor fabric activate

     

    line vty

    cumulus@cumulus:~$