Last updated: 14-06-2019 00:56

• Multihoming
  • Multihoming Definition
  • Private-AS
• Policy
  • Policy tools
  • Transits
    • Mistakes
  • Peers
    • Mistakes
• Multihoming Scenarios
  • Stub network
  • Multi-homed stub network
  • Multi-homed network
• Principles of multi-homing
  • Basic Multihoming
    • Two links to the same ISP (one as backup only)
    • Two links to the same ISP with Loadsharing
    • Multiple Dualhomed Customer End-sites
  • Multihoming to multiple to more than one AS (ISP)
    • Multiple Dualhomed End-sites, with Loadsharing

Multihoming

Why multihome?

• Redundancy.
• Reliability.
• Supplier Diversity
  • Many businesses demand supplier diversity
  • Internet connection from two or more suppliers
  • Changing upstream provider
  • Leverage, playing one ISP off against the other.

Multihoming Definition

More than one link external to the local network

• Two or more links to the same ISP
• Two or more links to different ISPs.
  Usually two external facing routers, one router gives link and provider redundancy only.
  Implementation detail may be different
• End site → ISP # ISP controls config
• ISP1 → ISP2 # ISPs share config

Private-AS

Private ASNs MUST be removed from all prefixes announced to the public Internet.

• Router(config-router)# neighbor x.x.x.x remove-private-AS
  = Definitions =

• Transit
  • Carrying traffic across a network
  • Usually for a fee
• Peering
  • Exchanging routing information and traffic
  • Usually for no fee
  • Sometimes called settlement free peering
• Default
  • Where to send traffic when there is no explicit match in the routing table

Policy

• Assumptions
  • Prefix-lists are used throughout
  • Easier/better/faster than access-lists
• Three principles
  • Prefix-lists to filter prefixes
  • Filter-lists to filter ASNs
  • Route-maps to apply policy

Route-maps can be used for filtering, but this is more advanced Configuration.

Policy tools

• Local preference
  • Outbound traffic flows
• Metric (MED)
  • Inbound traffic flows (local scope)
• AS-PATH prepend
  • Inbound traffic flows (Internet scope)
• Communities
  • Specific inter-provider peering

It is a MUST to announce assigned address block to Internet. MAY also announce subprefixes as reachability is not guaranteed. The current minimum IPv4 allocation is /24 and IPv6 /48.

Transits

A Transit provider is another autonomous system (AS) which is used to provide the local network with access to other networks. A transit providers need to be chosen wisely and it is recommended to have at least two, no more than three.

Mistakes

• ISPs sign up with too many transit providers
• No diversity.

Peers

A peer is another autonomous system with which the local network has agreed to exchange locally sourced routes and traffic. There are public and private peers. Peer as much as possible.

Mistakes

• Mistaking a transit provider’s “Exchange” business for a no-cost public peering point
• Not working hard to get as much peering as possible
• Ignoring/avoiding competitors because they are competition

Multihoming Scenarios

Stub network

• No need for BGP
• Point static default to upstream ISP
• Upstream ISP advertises stub network
• Policy confined within upstream ISP’s policy

Multi-homed stub network

• Use BGP (not IGP or static) to loadshare
• Use private AS (ASN > 64511)
• Upstream ISP advertises stub network
• Policy confined within upstream ISP’s policy

Multi-homed network

• Many situations possible
  • Multiple sessions to same ISP
    • Several options
      • eBGP multihop (use loopback address and BGP prefixes learned with loopback address as next hop). Try to avoid.

          Router(config)# router bgp 100
          Router(config-router)# neighbor 1.1.1.1 remote-as 200
          Router(config-router)# neighbor 1.1.1.1 ebgp-multihop 2
          Router(config-router)# exit
          Router(config)# ip route 1.1.1.1 255.255.255.255 serial 1/0
          Router(config)# ip route 1.1.1.1 255.255.255.255 serial 1/1
          Router(config)# ip route 1.1.1.1 255.255.255.255 serial 1/2
          
        

      • bgp multipath
        • Three BGP sessions required, platform limit on number of paths (could be as little as 6). Full BGP feed makes this unwieldy as 3 copies of Internet Routing Table goes into the FIB.

            Router(config)# router bgp 100
            Router(config-router)# neighbor 1.1.2.1 remote-as 200
            Router(config-router)# neighbor 1.1.2.5 remote-as 200
            Router(config-router)# neighbor 1.1.2.9 remote-as 200
            Router(config-router)# exit
            Router(config)# maximum-paths 3
            
          

      • CEF loadsharing (equal cost links)
      • BGP attribute manipulation
  • Secondary for backup only.
  • Load-share between primary and secondary.
  • Selectively use different ISPs.
  • Multiple Sessions to another AS
    • eBGP multihop
    • BGP multipath.

Principles of multi-homing

To split traffic between two links announce the aggregate on both links to ensure redundancy, announce one half of the address space on each link. In this case traffic for first half of address space comes in first link and traffic for second half of address space comes in second link. If either link fails, the fact that the aggregate is announced ensures there is a backup path.

The keys to successful multihoming configuration:

• Keeping traffic engineering prefix announcements independent of customer iBGP.
• Understanding how to announce aggregates.
• Understanding the purpose of announcing subprefixes of aggregates.
• Understanding how to manipulate BGP attributes.
• Too many upstreams/external paths makes multihoming harder (2 or 3 is enough!).

Basic Multihoming

Two links to the same ISP (one as backup only)

Applies when end-site has bought a large primary WAN link to their upstream and a small secondary WAN link as the backup. For example, primary path might be an 10G, backup might be 1G. AS100 removes private AS and any customer subprefixes from Internet announcement.

Announce /19 aggregate on each link. On the primary link: Outbound – announce /19 unaltered, inbound – receive default route. On the backup link, outbound – announce /19 with increased metric and inbound – received default, and reduce LOCAL_PREF. When one link fails, the announcement of the /19 aggregate via the other link ensures continued connectivity.

Router A Configuration

  RouterA(config)# router bgp 65534
  RouterA(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterA(config-router)# neighbor 122.102.10.2 remote-as 100
  RouterA(config-router)# neighbor 122.102.10.2 description RouterC
  RouterA(config-router)# neighbor 122.102.10.2 prefix-list aggregate out
  RouterA(config-router)# neighbor 122.102.10.2 prefix-list default in
  RouterA(config-router)# exit
  RouterA(config)# ip prefix-list aggregate permit 121.10.0.0/19
  RouterA(config)# ip prefix-list default permit 0.0.0.0/0
  RouterA(config)# ip route 121.10.0.0 255.255.224.0 null0
  

Router B Configuration

  RouterB(config)# router bgp 65534
  RouterB(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterB(config-router)# neighbor 122.102.10.6 remote-as 100
  RouterB(config-router)# neighbor 122.102.10.6 description RouterD
  RouterB(config-router)# neighbor 122.102.10.6 prefix-list aggregate out
  RouterB(config-router)# neighbor 122.102.10.6 route-map med10-out out
  RouterB(config-router)# neighbor 122.102.10.6 prefix-list default in
  RouterB(config-router)# neighbor 122.102.10.6 route-map lp-low-in in
  RouterB(config-router)# exit
  RouterB(config)# ip prefix-list aggregate permit 121.10.0.0/19
  RouterB(config)# ip prefix-list default permit 0.0.0.0/0
  RouterB(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterB(config)# route-map med10-out permit 10
  RouterB(config-route-map)# set metric 10
  RouterB(config-route-map)# route-map lp-low-in permit 10
  RouterB(config-route-map)# set local-preference 90
  

Router C Configuration (main link)

  RouterC(config)# router bgp 100
  RouterC(config-router)# neighbor 122.102.10.1 remote-as 65534
  RouterC(config-router)# neighbor 122.102.10.1 default-originate
  RouterC(config-router)# neighbor 122.102.10.1 prefix-list Customer in
  RouterC(config-router)# neighbor 122.102.10.1 prefix-list default out
  RouterC(config-router)# exit
  RouterC(config)# ip prefix-list Customer permit 121.10.0.0/19
  RouterC(config)# ip prefix-list default permit 0.0.0.0/0
  

Router D Configuration (backup link)

  RouterD(config)# router bgp 100
  RouterD(config-router)# neighbor 122.102.10.5 remote-as 65534
  RouterD(config-router)# neighbor 122.102.10.5 default-originate
  RouterD(config-router)# neighbor 122.102.10.5 prefix-list Customer in
  RouterD(config-router)# neighbor 122.102.10.5 prefix-list default out
  RouterD(config-router)# exit
  RouterD(config)# ip prefix-list Customer permit 121.10.0.0/19
  RouterD(config)# ip prefix-list default permit 0.0.0.0/0
  

Router E

Router E removes the private AS and customer’s subprefixes from external announcements Private AS still visible inside AS100.

  RouterE(config)# router bgp 100
  RouterE(config-router)# neighbor 122.102.10.17 remote-as 110
  RouterE(config-router)# neighbor 122.102.10.17 remove-private-AS
  RouterE(config-router)# neighbor 122.102.10.17 prefix-list Customer out
  RouterE(config-router)# exit
  RouterE(config)# ip prefix-list Customer permit 121.10.0.0/19
  

Two links to the same ISP with Loadsharing

This is the more common case where end sites tend not to buy circuits and leave them idle, only used for backup as in previous example. This example assumes equal capacity circuits. Unequal capacity circuits require more refinement.

• Announce /19 aggregate on each link.
• Split /19 and announce as two /20s, one on each link.
• Vary the split until “perfect” loadsharing achieved.
• Accept the default from upstream.

Router A Configuration

  RouterA(config)# router bgp 65534
  RouterA(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterA(config-router)# network 121.10.0.0 mask 255.255.240.0
  RouterA(config-router)# neighbor 122.102.10.2 remote-as 100
  RouterA(config-router)# neighbor 122.102.10.2 prefix-list as100-a out
  RouterA(config-router)# neighbor 122.102.10.2 prefix-list default in
  RouterA(config-router)# exit
  RouterA(config)# ip prefix-list default permit 0.0.0.0/0
  RouterA(config)# ip prefix-list as100-a permit 121.10.0.0/20
  RouterA(config)# ip prefix-list as100-a permit 121.10.0.0/19
  RouterA(config)# ip route 121.10.0.0 255.255.240.0 null0
  RouterA(config)# ip route 121.10.0.0 255.255.224.0 null0
  

Router B Configuration

  RouterB(config)# router bgp 65534
  RouterB(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterB(config-router)# network 121.10.16.0 mask 255.255.240.0
  RouterB(config-router)# neighbor 122.102.10.6 remote-as 100
  RouterB(config-router)# neighbor 122.102.10.6 prefix-list as100-b out
  RouterB(config-router)# neighbor 122.102.10.6 prefix-list default in
  RouterB(config-router)# exit
  RouterB(config)# ip prefix-list default permit 0.0.0.0/0
  RouterB(config)# ip prefix-list as100-b permit 121.10.16.0/20
  RouterB(config)# ip prefix-list as100-b permit 121.10.0.0/19
  RouterB(config)# ip route 121.10.16.0 255.255.240.0 null0
  RouterB(config)# ip route 121.10.0.0 255.255.224.0 null0
  

Router C and Router D Configurations

• Router D configuration is identical to Router C.
• Router C and D only allows in /19 and /20 prefixes from customer block.

  RouterC(config)# router bgp 100
  RouterC(config-router)# neighbor 122.102.10.1 remote-as 65534
  RouterC(config-router)# neighbor 122.102.10.1 default-originate
  RouterC(config-router)# neighbor 122.102.10.1 prefix-list Customer in
  RouterC(config-router)# neighbor 122.102.10.1 prefix-list default out
  RouterC(config-router)# exit
  RouterC(config)# ip prefix-list Customer permit 121.10.0.0/19 le 20  # le = lessthan
  RouterC(config)# ip prefix-list default permit 0.0.0.0/0
  

Router E Configuration

Private AS still visible inside AS100.

  RouterE(config)# router bgp 100
  RouterE(config-router)# neighbor 122.102.10.17 remote-as 110
  RouterE(config-router)# neighbor 122.102.10.17 remove-private-AS
  RouterE(config-router)# neighbor 122.102.10.17 prefix-list Customer out
  RouterE(config-router)# exit
  RouterE(config)# ip prefix-list Customer permit 121.10.0.0/19
  

Default route for outbound traffic?

  Router(config)# router isis as65534
  Router(config-router)# default-information originate
  

Multiple Dualhomed Customer End-sites

Border router E in AS100 removes private AS and any customer subprefixes from Internet announcement.

Customer announcements as per previous exampl, this is documented in RFC2270. Address space is not overlapping and each customer hears default only. Router An and Bn configuration same as Router A and B previously.

Router A1 Configuration

  RouterA1(config)# router bgp 65534
  RouterA1(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterA1(config-router)# network 121.10.0.0 mask 255.255.240.0
  RouterA1(config-router)# neighbor 122.102.10.2 remote-as 100
  RouterA1(config-router)# neighbor 122.102.10.2 prefix-list as100-a out
  RouterA1(config-router)# neighbor 122.102.10.2 prefix-list default in
  RouterA1(config-router)# exit
  RouterA1(config)# ip prefix-list default permit 0.0.0.0/0
  RouterA1(config)# ip prefix-list as100-a permit 121.10.0.0/20
  RouterA1(config)# ip prefix-list as100-a permit 121.10.0.0/19
  RouterA1(config)# ip route 121.10.0.0 255.255.240.0 null0
  RouterA1(config)# ip route 121.10.0.0 255.255.224.0 null0
  

Router B1 Configuration

  RouterB1(config)# router bgp 65534
  RouterB1(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterB1(config-router)# network 121.10.16.0 mask 255.255.240.0
  RouterB1(config-router)# neighbor 122.102.10.6 remote-as 100
  RouterB1(config-router)# neighbor 122.102.10.6 prefix-list as100-b out
  RouterB1(config-router)# neighbor 122.102.10.6 prefix-list default in
  RouterB1(config-router)# exit
  RouterB1(config)# ip prefix-list default permit 0.0.0.0/0
  RouterB1(config)# ip prefix-list as100-b permit 121.10.16.0/20
  RouterB1(config)# ip prefix-list as100-b permit 121.10.0.0/19
  RouterB1(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterB1(config)# ip route 121.10.16.0 255.255.240.0 null0

Router C Configuration

  RouterC(config)# router bgp 100
  RouterC(config-router)# neighbor bgp-customers peer-group
  RouterC(config-router)# neighbor bgp-customers remote-as 65534
  RouterC(config-router)# neighbor bgp-customers default-originate
  RouterC(config-router)# neighbor bgp-customers prefix-list default out
  RouterC(config-router)# neighbor 122.102.10.1 peer-group bgp-customers
  RouterC(config-router)# neighbor 122.102.10.1 description Customer One
  RouterC(config-router)# neighbor 122.102.10.1 prefix-list Customer1 in
  RouterC(config-router)# neighbor 122.102.10.9 peer-group bgp-customers
  RouterC(config-router)# neighbor 122.102.10.9 description Customer Two
  RouterC(config-router)# neighbor 122.102.10.9 prefix-list Customer2 in
  RouterC(config-router)# neighbor 122.102.10.17 peer-group bgp-customers
  RouterC(config-router)# neighbor 122.102.10.17 description Customer Three
  RouterC(config-router)# neighbor 122.102.10.17 prefix-list Customer3 in
  RouterC(config-router)# exit
  RouterC(config)# ip prefix-list Customer1 permit 121.10.0.0/19 le 20
  RouterC(config)# ip prefix-list Customer2 permit 121.16.64.0/19 le 20
  RouterC(config)# ip prefix-list Customer3 permit 121.14.192.0/19 le 20
  RouterC(config)# ip prefix-list default permit 0.0.0.0/0
  

Router D Configuration

  RouterD(config)# router bgp 100
  RouterD(config-router)# neighbor bgp-customers peer-group
  RouterD(config-router)# neighbor bgp-customers remote-as 65534
  RouterD(config-router)# neighbor bgp-customers default-originate
  RouterD(config-router)# neighbor bgp-customers prefix-list default out
  RouterD(config-router)# neighbor 122.102.10.5 peer-group bgp-customers
  RouterD(config-router)# neighbor 122.102.10.5 description Customer One
  RouterD(config-router)# neighbor 122.102.10.5 prefix-list Customer1 in
  RouterD(config-router)# neighbor 122.102.10.13 peer-group bgp-customers
  RouterD(config-router)# neighbor 122.102.10.13 description Customer Two
  RouterD(config-router)# neighbor 122.102.10.13 prefix-list Customer2 in
  RouterD(config-router)# neighbor 122.102.10.21 peer-group bgp-customers
  RouterD(config-router)# neighbor 122.102.10.21 description Customer Three
  RouterD(config-router)# neighbor 122.102.10.21 prefix-list Customer3 in
  RouterD(config-router)# exit
  RouterD(config)# ip prefix-list Customer1 permit 121.10.0.0/19 le 20
  RouterD(config)# ip prefix-list Customer2 permit 121.16.64.0/19 le 20
  RouterD(config)# ip prefix-list Customer3 permit 121.14.192.0/19 le 20
  RouterD(config)# ip prefix-list default permit 0.0.0.0/0
  

Router E Configuration

Assumes customer address space is not part of upstream’s address block.

  RouterE(config)# router bgp 100
  RouterE(config-router)# neighbor 122.102.10.17 remote-as 110
  RouterE(config-router)# neighbor 122.102.10.17 remove-private-AS
  RouterE(config-router)# neighbor 122.102.10.17 prefix-list Customers out
  RouterE(config-router)# exit
  RouterE(config)# ip prefix-list Customers permit 121.10.0.0/19
  RouterE(config)# ip prefix-list Customers permit 121.16.64.0/19
  RouterE(config)# ip prefix-list Customers permit 121.14.192.0/19
  

Multihoming to multiple to more than one AS (ISP)

• Use a Public AS.
• Address space comes from both upstreams or the Regional Internet Registry (RIR).

• Announce /19 aggregate on each link, primary link makes standard announcement and backup link lengthens the AS PATH by using AS PATH prepend.
• When one link fails, the announcement of the /19 aggregate via the other link ensures continued connectivity.

Router A Configuration

  RouterA(config)# router bgp 130
  RouterA(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterA(config-router)# neighbor 122.102.10.1 remote-as 100
  RouterA(config-router)# neighbor 122.102.10.1 prefix-list aggregate out
  RouterA(config-router)# neighbor 122.102.10.1 prefix-list default in
  RouterE(config-router)# exit
  RouterA(config)# ip prefix-list aggregate permit 121.10.0.0/19
  RouterA(config)# ip prefix-list default permit 0.0.0.0/0
  RouterA(config)# ip route 121.10.0.0 255.255.224.0 null0
  

Router B Configuration

Preprend AS path to reduce its desirability as a path.

  RouterB(config)# router bgp 100
  RouterB(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterB(config-router)# neighbor 120.1.5.1 remote-as 120
  RouterB(config-router)# neighbor 120.1.5.1 prefix-list aggregate out
  RouterB(config-router)# neighbor 120.1.5.1 route-map as120-prepend out
  RouterB(config-router)# neighbor 120.1.5.1 prefix-list default in
  RouterB(config-router)# neighbor 120.1.5.1 route-map lp-low in
  RouterB(config-router)# exit
  RouterB(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterB(config)# ip prefix-list aggregate permit 121.10.0.0/19
  RouterB(config)# ip prefix-list default permit 0.0.0.0/0
  RouterB(config)# route-map as120-prepend permit 10
  RouterB(config-route-map)# set as-path prepend 100 100 100
  RouterB(config-route-map)# route-map lp-low permit 10
  RouterB(config-route-map)# set local-preference 80

Multiple Dualhomed End-sites, with Loadsharing

Announce /19 aggregate on each link. Split /19 and announce as two /20s, one on each link for basic inbound loadsharing. When one link fails, the announcement of the /19 aggregate via the other ISP ensures continued connectivity.

Router A Configuration

  RouterA(config)# router bgp 100
  RouterA(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterA(config-router)# network 121.10.0.0 mask 255.255.240.0
  RouterA(config-router)# neighbor 122.102.10.1 remote-as 110
  RouterA(config-router)# neighbor 122.102.10.1 prefix-list as110-out out
  RouterA(config-router)# neighbor 122.102.10.1 prefix-list default in
  RouterA(config-router)# exit
  RouterA(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterA(config)# ip route 121.10.0.0 255.255.240.0 null0
  RouterA(config)# ip prefix-list default permit 0.0.0.0/0
  RouterA(config)# ip prefix-list as110-out permit 121.10.0.0/20
  RouterA(config)# ip prefix-list as110-out permit 121.10.0.0/19
  

Router B Configuration

  RouterB(config)# router bgp 100
  RouterB(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterB(config-router)# network 121.10.16.0 mask 255.255.240.0
  RouterB(config-router)# neighbor 120.1.5.1 remote-as 120
  RouterB(config-router)# neighbor 120.1.5.1 prefix-list as120-out out
  RouterB(config-router)# neighbor 120.1.5.1 prefix-list default in
  RouterB(config-router)# exit
  RouterB(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterB(config)# ip route 121.10.16.0 255.255.240.0 null0
  RouterB(config)# ip prefix-list default permit 0.0.0.0/0
  RouterB(config)# ip prefix-list as120-out permit 121.10.0.0/19
  RouterB(config)# ip prefix-list as120-out permit 121.10.16.0/20
  

A more controlled loadsharing

Announce /19 aggregate on each link, on the first link, announce /19 as normal while on the second link, announce /19 with longer AS PATH, and announce one /20 subprefix. This controls loadsharing between upstreams and the Internet.

Vary the subprefix size and AS PATH length until perfect loadsharing achieved.

This example is more commonplace and shows how ISPs and end-sites subdivide address space frugally, as well as use the AS-PATH prepend concept to optimise the load sharing between different ISPs. Notice that the /19 aggregate block is ALWAYS announced.

Router A Configuration

  RouterA(config)# router bgp 100
  RouterA(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterA(config-router)# neighbor 122.102.10.1 remote-as 110
  RouterA(config-router)# neighbor 122.102.10.1 prefix-list default in
  RouterA(config-router)# neighbor 122.102.10.1 prefix-list as110-out out
  Routera(config-router)# exit
  RouterA(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterA(config)# ip prefix-list as110-out permit 121.10.0.0/19
  RouterA(config)# ip prefix-list default permit 0.0.0.0/0
  

Router B Configuration

  RouterB(config)# router bgp 100
  RouterB(config-router)# network 121.10.0.0 mask 255.255.224.0
  RouterB(config-router)# network 121.10.16.0 mask 255.255.240.0
  RouterB(config-router)# neighbor 120.1.5.1 remote-as 120
  RouterB(config-router)# neighbor 120.1.5.1 prefix-list default in
  RouterB(config-router)# neighbor 120.1.5.1 prefix-list as120-out out
  RouterB(config-router)# neighbor 120.1.5.1 route-map agg-prepend out
  RouterB(config-router)# exit
  RouterB(config)# ip route 121.10.0.0 255.255.224.0 null0
  RouterB(config)# ip route 121.10.16.0 255.255.240.0 null0
  RouterB(config)# route-map agg-prepend permit 10
  RouterB(config-route-map)# match ip address prefix-list aggregate
  RouterB(config-route-map)# set as-path prepend 100 100
  RouterB(config-route-map)# route-map agg-prepend permit 20
  RouterB(config-router)# exit
  RouterB(config)# ip prefix-list default permit 0.0.0.0/0
  RouterB(config)# ip prefix-list as120-out permit 121.10.0.0/19
  RouterB(config)# ip prefix-list as120-out permit 121.10.16.0/20
  RouterB(config)# ip prefix-list aggregate permit 121.10.0.0/19