Understanding the Meta, Comcast, and LinkedIn Outages

Product Updates

Visualizing BGP Routing for Better Network Operations

By Ameet Naik
| | 9 min read


The Internet is a complex web of thousands of independently-managed networks organized into Autonomous Systems (AS). Border Gateway Protocol (BGP) is the protocol that all these Autonomous Systems use to share routing information with each other. Understanding BGP routing is the key to understanding how your network connects to the rest of the Internet, and what networks your application traffic transits before it gets to you (or to your customers). This is a key dependency governing application and network performance, especially when you’re delivering or consuming Software-as-a-Service (SaaS) applications like Office 365, Salesforce or Webex.

ThousandEyes has long included BGP route visualization as a data layer to help our users understand and troubleshoot Internet-related outages and performance issues. This data set also helps detect security issues such as route leaks and hijacks that can accidentally or maliciously send your traffic into a different network, such as we saw recently during the Amazon Route 53 DNS hijack. We recently improved this visualization to make it easier to understand the relationships between the various autonomous systems in your data path. In this blog post, we’ll take a tour of this new view and all its cool features.

Intuitive, Inference-Driven Visuals

Internet connectivity into a SaaS data center
Figure 1: Internet connectivity into a SaaS data center.

Figure 1 above shows the example of a SaaS provider hosting an application in their own data center. This BGP Route Visualization follows a left to right flow, which is more intuitive and closely mirrors our Path Visualization view. The diamonds on the left of this diagram represent BGP monitors connected to different autonomous systems all over the world. The small circle on the rightmost end represents the IP prefix covering the IP addresses of the target data center. The ovals in the middle represent autonomous systems (AS) that connect this data center to the rest of the Internet. The lines on this view represent the paths taken by different networks to reach this data center, with a thicker line indicating more networks preferring that path. So if you have multiple ISPs, as in this example, you can see and influence how much traffic comes over one vs the other.

Since we collect data from dozens of monitors, we paginate this view in order to make it easier to find the answers. Clicking on the Next Monitors button will show you the next set of monitors and you can navigate back and forth through these views, just like flipping through pages of data. We use our global inference engine to surface the most relevant information on the first page so you never have to go digging too far.

When BGP Paths Change

BGP path changes
Figure 2: BGP path changes visualized.

In Figure 2, we see an example of the same environment undergoing a path change. The solid red lines represent paths that were involved in a change, while the dashed red lines represent paths that are no longer active. In this instance, we see two changes going on. The Tier 1 provider, Limelight Networks (AS 22822), was experiencing packet loss through their network. The ISP SINAP-TIX decided to reroute traffic away from Limelight Networks, as indicated by the dashed line between them and Limelight. We also see the SaaS provider has made some routing changes of their own, in order to rebalance traffic between their two ISPs.

This is just one example of the actionable insights you can gain from BGP Route Visualization that help you troubleshoot and optimize your Internet-connected network and application delivery experiences.

Detect BGP Hijacks and Leaks

Another major benefit is detecting BGP hijacks and leaks. Earlier this year, Amazon’s Route 53 DNS service was hijacked using a malicious BGP route announcement. Let’s walk through a BGP route visualization of this incident. You can follow along live by clicking on this share link.

Healthy Internet reachability to Route 53 DNS service
Figure 3: Healthy Internet reachability to Amazon’s Route 53 DNS service.

In this example, the prefix represents the address range for Amazon’s authoritative DNS servers for the Route 53 service. This prefix is announced by Amazon to tens of Tier 1 networks across the globe since they have direct connectivity. However, we also see a more specific prefix 205.251.193/24 appear in the global Internet routing table at the time of the incident.

Malicious prefix in Internet routing table
Figure 4: Malicious prefix appears in the Internet routing table.

Selecting this prefix from the drop-down list shows us that this prefix was originated by eNET (AS 10297). We can also see that it was accepted by Hurricane Electric, a major Tier 1 ISP, and a few smaller ISPs like TDS Telecom (AS 4181), but majority of the Internet filtered out this prefix, as evidenced by most of the monitors not having a path to it (represented by dotted lines). Even this partial leak was enough to cause a significant compromise of a key DNS service affecting sites like instagram.com and cnn.com.

Detailed BGP Route View

Details of BGP AS path change
Figure 5: Details of BGP AS path with timestamps.

In addition to the cool visuals, we also make it easy for you to drill into the actual BGP route data. Right-clicking on the Atlanta monitor gives you the option to see the BGP AS Path information, along with the detailed timestamps showing when the route was withdrawn. This added granularity is immensely useful while troubleshooting BGP path changes, especially involving security incidents.

Digital Experience Monitoring is Incomplete without BGP Visibility

BGP adds an invaluable layer of context to your network optimization and troubleshooting, shaving hours, and even days off your Mean Time to Troubleshoot (MTTT). You can see not just your immediate ISP, but networks several hops away from you that are impacting your application performance.

If you’re an existing ThousandEyes user, drop into the BGP Route Visualization view on any of your tests and check out the cool new features. If you’d like to explore your own Internet connectivity in an easy, visual way, start your free ThousandEyes trial today.

Subscribe to the ThousandEyes Blog

Stay connected with blog updates and outage reports delivered while they're still fresh.

Upgrade your browser to view our website properly.

Please download the latest version of Chrome, Firefox or Microsoft Edge.

More detail