Answer: Surprisingly little.
Okay, so where's this coming from? I recently had a conversation with an unnamed analyst from an unnamed, large analyst firm whose name starts with a letter between E and J (haha gotcha!). We were discussing the move to SaaS, SD-WAN and Direct Internet Access from branch offices, and how variable behavior of Internet connectivity in various parts of the world is causing real problems with cloud adoption and WAN modernization. He was telling me that in a number of inquiries held with large enterprises, that consternation caused by unpredictable behavior is causing network teams to demand multi-month pilots with ISPs for office locations in Asia-Pacific and other regions. Now, just to make it clear that this isn’t an isolated comment, analyst firms generally are reporting issues with Internet connectivity related to cloud and SD-WAN deployments. For example, we’ve seen an analyst prediction that half of Office 365 deployments with a global scope through 2019 will suffer network performance issues.
So there's certainly good reason for a level of caution around SD-WAN and DIA transformation. At ThousandEyes, we've seen plenty of direct evidence supported by observations by analyst firms that global SaaS deployments like Office 365 tend to hit significant network-related issues. Sharepoint, in particular is a real pain point, if I may be permitted to be so punny, because it's particularly sensitive to latency issues and its delivery architecture centers on a home PoP, which for North American enterprises, tends to live in guess which continent? We've worked with enterprises where SaaS issues from Asia and other regions get bad enough that they hit the CIO's desk regularly. Ouch.
So, the impulse to test your ISP connection in farther-flung offices is totally justified. The only problem is that it's not nearly adequate. If you can't see what's going on between that office and your SaaS apps, there is a very high likelihood that you'll still have problems and the pilot period will just be a disorienting, if quaint memory.
"Past performance is no guarantee of future results."
First of all, the Internet is far too dynamic for a 3-month pilot to tell you what your connectivity future holds (on its own). It's like when you read the disclosure on stock or mutual fund statements: "Past performance is no guarantee of future results." That sentence is very applicable to the Internet. ISPs change transit and peering relationships over time. Those upstream providers also change their connectivity arrangements. Routing and paths change dynamically at the drop of a hat. Actually, much faster—at the drop of a milli-hat maybe? So, if you're trying to do DIA from global branch offices, just know that things will change.
Other Dependencies Matter
But connectivity isn't the only issue. Before you even get to Connect phase—a three-way handshake with the server for your SaaS application, you have to find the server. That means that DNS needs to work. And guess what, that's also subject to changing conditions, especially from global locations. DNS latency can cause overall latency problems with sensitive applications.
Then there's the variance in service delivery architectures of different SaaS providers. Some are delivered by a CDN. Some of them are not. Oh, and are you considering a web-based Secure Web Gateway (SWG) provider to protect your branches instead of maintaining appliances in each site? Well, then you've got to factor in the journey to and through their regional PoP before your packets go on their merry, multi ASN path to the SaaS network.
You Simply Need to See Better
If you want to manage all these externalities, there is simply no substitute for visibility. The problem is that traditional monitoring approaches don't work outside of the four walls of your business, because they rely on passive data that is collected from infrastructure you own and control. But you don't control the Internet, so you can't collect data from your ISP, SWG or SaaS provider's gear. Application Performance Management (APM) won't help you either since you can't inject code into Office 365 or Salesforce. Now don’t get me wrong—I’m not saying that APM or other monitoring is bad. It’s just not available from the Internet.
There are good answers to this visibility problem—ways to see details of DNS, Internet and SaaS performance at the app and network layer. It's the problem that Network Intelligence is meant to solve, and the problem that we’ve been working on for a while. We’re seeing awareness build on both sides of this problem domain, with 18 of the top 20 SaaS and 50+ of the Fortune 500 using Network Intelligence.
But the main point is this: you can't treat the Internet like a slow-moving, traditional utility. The Internet is more like a living organism. There are ways to monitor it, to manage it, to govern your Internet-connected providers. An ISP pilot is a good idea, but to gain real insight and start to influence your future, you need to pair that with visibility into how all your external dependencies are working so you can figure out if you need to remediate something or choose another ISP based on detailed data. You need to start practicing troubleshooting during your pilot where you’re speedily isolating the third party provider that matters, and using shared data to drive an effective escalation instead of going around, hat in hand, begging for clues. Armed with that data, you won’t need a fortune teller to enable your SD-WAN to thrive in this unpredictable, connected world.