AfNOG - Advanced Routing - Day 3

Diarmuid O'Briain
12/06/2019, version 1.0

Last updated: 12-06-2019 21:42



BGP Attributes

Carry various information about or characteristics of the prefix being propagated. There are Transitive BGP Attributes which are allowed to be sent to other peers and Non-Transitive Attributes that are not allowed to be sent to other peers. BGP community strings are an example of optional Transitive Attribute. If such an attribute is received then it can be passed on to peer neighbours. The Multi-Exit Discriminator (MED) is an example of an optional Non-Transitive Attribute that if received then it must be removed from the Network Layer Reachability Information (NLRI) before forwarding the update on to peering neighbours.

AS-PATH

The AS_PATH is a sequence of ASes a route has traversed, Loop detection and applying policy. Mandatory non-transitive attribute.

NEXT-HOP

This is a mandatory Non-Transitive Attribute.

ORIGIN

The ORIGIN is a transitive and mandatory attribute that conveys the origin of the prefix. There are three possible values:

AGGREGATOR

An optional and transitive attribute that conveys the IP address of the router or BGP speaker generating the aggregate route. Does not influence best path selection. Useful for debugging, not really used for routing.

  Router(config)# router bgp 100
  Router(config-router)# aggregate-address 100.1.0.0 255.255.0.0
  

LOCAL_PREFERENCE

For outbound decisions (Higher is best). THe LOCAL_PREF is a Non-Transitive and optional attribute with local significance only. It is used to influence BGP path selection and determines the best path for outbound traffic by the higher local preference.

  Router(config)# router bgp 400
  Router(config-router)# neighbor 120.5.1.1 remote-as 300
  Router(config-router)# neighbor 120.5.1.1 route-map LOCAL-PREF in
  
  Router(config)# route-map LOCAL-PREF permit 10
  Router(config-route-map)# match ip address prefix-list MATCH
  Router(config-route-map)# set local-preference 800
  
  Router(config)# route-map LOCAL-PREF permit 20
  
  Router(config)# ip prefix-list MATCH permit 160.10.0.0/16
  

Multi-Exit Discriminator

Multi-Exit Discriminator (MED) is used for inbound decisions (Lower is best). MED is a hint to external neighbors about the preferred path into an Autonomous System (AS) that has multiple entry points. Inter-AS, Non-Transitive and optional attribute used to convey the relative preference of entry points that determines the best path for inbound traffic. The path with lowest MED wins. The default is 0 (RFC4271).

WEIGHT

This is a Cisco local only setting. It is not really an attribute. The highest weight wins and is applied to all routes from a neighbour or weight can be assigned to routes based on filter.

  Router(config-router)# neighbor 120.5.7.1 weight 100
  
  Router(config-router)# neighbor 120.5.7.3 filter-list 3 weight 50
   

COMMUNITY

A COMMUNITY is a Transitive and optional attribute described in RFC1997. This is a 32-bit integer in the format <local-ASN>:xx, 0:0 to 0:65535 and 65535:0 to 65535:65535 are reserved. It is used to group destinations and each destination could be part of many communities. It is a tag to group similar destinations, customers of particular types for example.

A transit ISP can develop a list of communities that they share with lower tier ISP customers, this allows the ISP customer to perform self traffic engineering. Example at: SEACOM BGP Communities

Extended COMMUNITIES

The addition of a new Transitive-optional attribute (Type 16) to give additional flexibility. This is useful to deal with 32-bit ASNs.

Reverse Path Forwarding (RPF)

Reverse-path forwarding (RPF) is a technique used in modern routers for the purposes of ensuring loop-free forwarding of multicast packets in multicast routing and to help prevent IP address spoofing in Unicast routing (uRPF). uRPF is a security feature that works by enabling a router to verify the reachability of the source address in packets being forwarded. This capability can limit the appearance of spoofed addresses on a network. If the source IP address is not valid, the packet is discarded.

BGP attribute order of preference

The attribute order of preference is:

  1. Prefer the path with the highest WEIGHT (Local to the router, it is not passed).
  2. Prefer the path with the highest LOCAL_PREF.
  3. Prefer the path that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP.
  4. Prefer the path with the shortest AS_PATH.
  5. Prefer the path with the lowest ORIGIN type, i.e. IGP < EGP < incompete.
  6. Prefer the path with the lowest MED.
  7. Prefer eBGP over iBGP paths. If bestpath is selected, go to Step 9.
  8. Prefer the path with the lowest IGP metric to the BGP next hop. Continue, even if best path is already selected.
  9. Determine if multiple paths require installation in the routing table for BGP Multipath. Continue, if bestpath is not yet selected.
  10. When both paths are external, prefer the path that was received first.
  11. Prefer the route that comes from the BGP router with the lowest ROUTER-ID.
  12. If the originator or ROUTER-ID is the same for multiple paths, prefer the path with the minimum cluster list length.
  13. Prefer the path that comes from the lowest neighbour address.

BGP Best Current practices

SPs should start off with the following BGP commands as a basic template:

  Router(config)# router bgp 64511
  Router(config-router)# bgp deterministic-med
  Router(config-router)# distance bgp 200 200 200
  Router(config-router)# no synchronisation
  Router(config-router)# no auto-summary
  

If supporting more than just IPv4 unicast neighbours. Turn off the IOS assumption that all neighbours will exchange IPv4 prefixes.

  Router(config-router)# no bgp default ipv4-unicast
  

It is good practice is to ensure that each eBGP neighbour has inbound and outbound filter applied.

  Router(config)# router bgp 64511
  Router(config-router)# neighbor 1.2.3.4 remote-as 64510
  Router(config-router)# neighbor 1.2.3.4 prefix-list as64510-in in
  Router(config-router)# neighbor 1.2.3.4 prefix-list as64510-out out
  

Internal Routing Protocols (IGPs)

Examples are ISIS and OSPF, used for carrying infrastructure addresses and NOT used for carrying Internet prefixes or customer prefixes. The design goal is to minimise number of prefixes in IGP to aid scalability and rapid convergence.

BGP

BGP is used bith internally (iBGP) and externally (eBGP).

BGP versus OSPF/ISIS

DO NOT distribute BGP prefixes into an IGP, distribute IGP routes into BGP or use an IGP to carry customer prefixes otherwise the network will not scale.

Aggregation

Aggregation means announcing the address block received from the RIR to the other ASes connected to your network. Sub-prefixes of this aggregate may be used internally in the ISP network and is announced to other ASes to aid with multihoming. Too many operators are still thinking about class Cs, resulting in a proliferation of /24s in the Internet routing table, as of February 2015: 287000 /24s existed in IPv4 table of 533000 prefixes. The same is happening for /48s with IPv6, as of February 2015: 9600 /48s existed in IPv6 table of 22000 prefixes.

Configuration example ISP has 101.10.0.0/19 address block, to put into BGP as an aggregate.

  Router(config)# router bgp 64511
  Router(config-router)# network 101.10.0.0 mask 255.255.224.0
  Router(config-router)# exit
  Router(config)# ip route 101.10.0.0 255.255.224.0 null0
  

The static route is a pull up route. More specific prefixes within this address block ensure connectivity to ISP's customers with Longest match lookup.

Address block should be announced to the Internet as an aggregate. Subprefixes of address block should NOT be announced to Internet unless for traffic engineering. Aggregate should be generated internally and not on the network borders.

  Router(config)# router bgp 64511
  Router(config-router)# network 101.10.0.0 mask 255.255.224.0
  Router(config-router)# neighbor 102.102.10.1 remote-as 101
  Router(config-router)# neighbor 102.102.10.1 prefix-list out-filter out
  Router(config-router)# exit
  Router(config)# ip route 101.10.0.0 255.255.224.0 null0
  Router(config)# ip prefix-list out-filter permit 101.10.0.0/19
  Router(config)# ip prefix-list out-filter deny 0.0.0.0/0 le 32
  

ISPs who don't and won't aggregate are held in poor regard by community. Registries publish their minimum allocation size. IPv4 now ranging from a /20 to a /24 depending on RIR wit different sizes for different address blocks. For IPv6 /48 for assignment, /32 for allocation. Until recently there was no real reason to see anything longer than a /22 IPv4 prefix in the Internet.

Separation of iBGP and eBGP

Many ISPs do not understand the importance of separating iBGP and eBGP. iBGP is where all customer prefixes are carried and eGP is used for announcing aggregate to Internet and for Traffic Engineering. Do NOT do traffic engineering with customer originated iBGP prefixes which leads to instability. Even though aggregate is announced, a flapping subprefix will lead to instability for the customer concerned. Generate traffic engineering prefixes on the Border Router.

Receiving Prefixes

There are three scenarios for receiving prefixes from other ASNs:

Each has different filtering requirements and need to be considered separately. ISPs should only accept prefixes which have been assigned or allocated to their downstream customer If ISP has assigned address space to its customer, then the customer IS entitled to announce it back to his ISP. If the ISP has NOT assigned address space to its customer, then check in the five RIR databases to see if this address space really has been assigned to the customer.

  $ whois -h whois.afrinic.net 196.223.25.0/24
  % This is the AfriNIC Whois server.
  
  % Note: this output has been filtered.
  %       To receive output for a database update, use the "-B" flag.
  
  % Information related to '196.223.25.0 - 196.223.25.255'
  
  % No abuse contact registered for 196.223.25.0 - 196.223.25.255
  
  inetnum:        196.223.25.0 - 196.223.25.255
  netname:        UIXP-Peering
  descr:          Uganda Internet Exchange Point
  country:        UG
  org:            ORG-UIEP1-AFRINIC
  admin-c:        KS26-AFRINIC
  tech-c:         MBK2-AFRINIC
  status:         ASSIGNED PI
  mnt-by:         AFRINIC-HM-MNT
  mnt-lower:      UIXP-MNT
  source:         AFRINIC # Filtered
  parent:         196.223.0.0 - 196.223.31.255
  
  

For Example, downstream has 100.50.0.0/20 block and should only announce this to upstreams. Upstreams should only accept this from them. Configure on the upstream:

  Router(config)# router bgp 100
  Router(config-router)# neighbor 102.102.10.1 remote-as 101
  Router(config-router)# neighbor 102.102.10.1 prefix-list customer in
  Router(config-router)# neighbor 102.102.10.1 prefix-list default out
  Router(config-router)# exit
  Router(config)# ip prefix-list customer permit 100.50.0.0/20
  Router(config)# ip prefix-list default permit 0.0.0.0/0
  

A peer is an ISP with whom there is an agreement to exchange prefixes the are originated into the Internet routing table. Prefixes accepted from a peer are only those they have indicated they will announce. Prefixes announced to a peer are only those that have been indicated will be announced. It is agreeing what each will announce to the other via direct communication of documentation or use of the Internet Routing Registry and configuration tools such as the IRRToolSet: http://irrtoolset.isc.org.

Example: a peer has 220.50.0.0/16, 61.237.64.0/18 and 81.250.128.0/17 address blocks. Configuration on local router:

  Router(config)# router bgp 100
  Router(config-router)# neighbor 102.102.10.1 remote-as 101
  Router(config-router)# neighbor 102.102.10.1 prefix-list my-peer in
  Router(config-router)# exit
  Router(config)# ip prefix-list my-peer permit 220.50.0.0/16
  Router(config)# ip prefix-list my-peer permit 61.237.64.0/18
  Router(config)# ip prefix-list my-peer permit 81.250.128.0/17
  Router(config)# ip prefix-list my-peer deny 0.0.0.0/0 le 32
  

Receiving prefixes from Upstream/Transit Provider

Upstream/Transit Provider is an ISP who you pay to give you transit to the WHOLE Internet. Receiving prefixes from them is not desirable unless really necessary. Ask upstream/transit provider to either originate a default-route or announce one prefix you can use as default.

Downstream Router Configuration

  Router(config)# router bgp 100
  Router(config-router)# network 101.10.0.0 mask 255.255.224.0
  Router(config-router)# neighbor 101.5.7.1 remote-as 101
  Router(config-router)# neighbor 101.5.7.1 prefix-list infilter in
  Router(config-router)# neighbor 101.5.7.1 prefix-list outfilter out
  Router(config-router)# exit
  Router(config)# ip prefix-list infilter permit 0.0.0.0/0
  Router(config)# ip prefix-list outfilter permit 101.10.0.0/19
  

Upstream Router Configuration

  Router(config)# router bgp 101
  Router(config-router)# neighbor 101.5.7.2 remote-as 100
  Router(config-router)# neighbor 101.5.7.2 default-originate
  Router(config-router)# neighbor 101.5.7.2 prefix-list cust-in in
  Router(config-router)# neighbor 101.5.7.2 prefix-list cust-out out
  Router(config-router)# exit
  Router(config)# ip prefix-list cust-in permit 101.10.0.0/19
  Router(config)# ip prefix-list cust-out permit 0.0.0.0/0
  

If necessary to receive prefixes from any provider, care is required.

Prefixes into iBGP

To inject prefixes into iBGP use iBGP to carry customer prefixes, do not use an IGP. Point a static pull-up route to customer interface. Use BGP network statements. As long as static route exists (interface active), prefix will be in BGP.

  Router(config)# interface loopback 0
  Router(config-if)# ip address 215.17.3.1 255.255.255.255
  
  Router(config-if)# interface Serial 5/0
  Router(config-if)# ip unnumbered loopback 0
  Router(config-if)# ip verify unicast reverse-path
  Router(config-if)# exit
  Router(config)# ip route 215.34.10.0 255.255.252.0 Serial 5/0
  
  Router(config)# router bgp 100
  Router(config-if)# network 215.34.10.0 mask 255.255.252.0
  

Interface flap will result in prefix withdraw and reannounce so use ip route . . . permanent. Many ISPs redistribute static routes into BGP rather than using the network statement; however, only do this if you understand why.

  Router(config)# ip route 215.34.10.0 255.255.252.0 Serial 5/0
  
  Router(config)# router bgp 100
  Router(config-router)# redistribute static route-map static-to-bgp
  
  Router(config)# route-map static-to-bgp permit 10
  Router(config-route-map)# match ip address prefix-list ISP-block
  Router(config-route-map)# set origin igp
  Router(config-route-map)# set community 100:1000
  
  Router(config)# ip prefix-list ISP-block permit 215.34.10.0/22 le 30
  

Route-map ISP-block can be used for many things like setting communities, other attributes and setting origin code to IGP, etc.. Be careful with prefix-lists and route-maps as the absence of either/both means all statically routed prefixes go into iBGP.

Some other tips