In this post I’ll cover in more detail the route monitoring capabilities of ThousandEyes, already touched in previous blog posts on how routing changes impact performance, and BGP for DDoS mitigation. Routing is a key determinant of network performance; each route that packets take has varying latencies and throughput. And when routing goes wrong, it can prevent packets from getting to their destination. Therefore, understanding routing across networks, specifically using Border Gateway Protocol (BGP), is critical to troubleshooting traffic flows that traverse large corporate networks or the public Internet.
A Brief Intro to the Border Gateway Protocol (BGP)
The Internet consists of a myriad of independent networks organized into Autonomous Systems (AS). Each AS typically represents an independent administrative domain managed by a single organization and identified by a 4-byte number, e.g. AS 7018 is AT&T, AS 701-703 is Verizon, etc. Inside each AS there are a series of border routers (e.g. 2a-2c in Figure 1) that typically connect to each other in a full mesh using iBGP (i=internal; reflectors and confederations can be used to relax this constraint). Border routers in different ASes connect to each other through eBGP (e=external) sessions. BGP is used to announce reachability to a chunk of IP addresses (or prefix). BGP defines more than just physical interconnections; it is used to advertise which routes are possible based on policies defined by other considerations such as traffic engineering, maintenance, and commercial transit and peering agreements.
For example, pinterest.com resolves to the IP address 23.23.131.240. If we look at routing tables for announced address blocks that cover this IP address, we find it falls under address block 23.22.0.0/15 announced by AS 14618 belonging to Amazon.
$whois -h whois.cymru.com " -v 23.23.131.240 " AS | IP | BGP Prefix | CC | Registry | Allocated | AS Name 14618 | 23.23.131.240 | 23.22.0.0/15 | US | arin | 2011-09-19 | AMAZON-AES - Amazon.com, Inc.
Looking at RouteViews route server (telnet://route-views.routeviews.org) we can check the different AS paths available to reach 23.22.0.0/15. In the case below there are 31 available routes to reach the destination, but the router only picks one, the BGP best path, which is selected after looking at several route attributes, including BGP Local Preference and AS Path length.
route-views> sh ip bgp 23.23.131.240
BGP routing table entry for 23.22.0.0/15, version 636446191
Paths: (31 available, best #8, table Default-IP-Routing-Table)
Not advertised to any peer
3277 39710 9002 16509 14618
194.85.102.33 from 194.85.102.33 (194.85.4.4)
Origin IGP, localpref 100, valid, external
Community: 3277:39710 9002:9002 9002:64789
852 16509 14618
154.11.98.225 from 154.11.98.225 (154.11.98.225)
Origin IGP, metric 0, localpref 100, valid, external
Community: 852:180
3356 16509 14618
4.69.184.193 from 4.69.184.193 (4.69.184.193)
Origin IGP, metric 0, localpref 100, valid, external
Community: 3356:3 3356:22 3356:100 3356:123 3356:575 3356:2006 65000:0 65000:7843
...
External BGP Visibility (outside-in)
Public sources of BGP data, including RIPE-RIS in Europe and RouteViews in the U.S. establish eBGP sessions with hundreds of routers across the world (monitors) and provide a comprehensive picture of global routing reachability for a certain prefix (outside-in). This is the picture ThousandEyes typically represents in our BGP Route Visualization. For instance, in Figure 2 AS 36175 (ancestry.com) is announcing prefix 66.43.20.0/22 to 2 upstream providers XO Communications (AS 2828) and American Fiber (AS 31993). Each of the small green circles represent a router (or monitor) that is proving public BGP feeds. In the timeline, we are representing the average number of path changes per monitor; other metrics such as reachability and number of updates are also available. In this case, we noticed there’s a bump in the number of path changes at 6:00 UTC. If we zoom into that instant of time (Figure 3), we can see that there was a route change from AS 2828 (XO) to AS 31993 (American Fiber).
Internal BGP Visibility (inside-out)
We recently released the capability of visualizing both public and private eBGP routes for our customers. This means that any of our customers can setup a multi-hop eBGP session between one of their BGP speakers and our route collectors. There are two main benefits:
- Internal prefixes: for prefixes originated inside the network, the private feed is useful to triage problems whose root cause is inside the network versus problems that originate outside; users will be provided with a single view of public and private feeds.
- External prefixes: for prefixes belonging to a third party (e.g. a SaaS provider), the private feed is useful to detect cases where the route to the destination is sub-optimal (which affects performance of the application), or the route is taking a detour to a malicious destination (route hijacking).
Figure 4 below shows an example of one of our customers’ internal prefixes as seen by a combination of public and private BGP monitors. The small green double circle is a private BGP monitor. We can see that there are two origin Autonomous Systems in this view (the big green circles in the middle), but private AS 64999 in this case is only seen by the private monitor, and it’s not exposed to the other monitors.
Setting Up Private BGP Feeds in ThousandEyes
Setting up a private BGP feed with us is pretty straightforward. You just need to go to “Settings -> My Domains & Networks -> Private BGP Monitors” and complete the form indicating your router IP address and ASN, and we will coordinate with you to bring the session up (Figure 5). You can check the status of your sessions in the table at the bottom of this page as well.
With the combination of both public and private eBGP visibility, ThousandEyes provides a greater understanding of routing issues that occur within a corporate network as well as issues with external prefixes. This information can help reduce latencies, spot inefficient routes, troubleshoot incorrect routing changes, and detect hijacked routes. Start monitoring BGP routes in ThousandEyes by signing up for a free trial.