This week on the Internet, we officially saw the end of a two-week consecutive upward trend in global outages as the observed number dropped from 271 to 205, a 32% decrease when compared to the previous week. This trend was echoed domestically, where the number of observed outages also dropped from 119 to 87; a 27% decrease when compared to the previous week. This reduction also saw U.S. outages account for only 42% of all global outages last week, which is a 2% decrease when compared to the previous week.
While outage numbers were trending downward this past week, there were two separate incidents that reminded us, once more, of the fragility of an Internet infrastructure that we all increasingly rely on for connecting, communicating and transacting online.
In some cases, this is a fragility that can be worked around or mitigated. But in other cases, we have to accept that it exists and that outages will occur unless greater efforts and resources are invested into giving all Internet-connected users an equal, diverse, and secure infrastructure footing, regardless of where they are located geographically.
The first outage I wanted to address occurred at the cryptocurrency exchange Coinbase on Sunday, February 13.
The company debuted a new commercial that aired Sunday during U.S. professional football’s championship game. The ad consisted solely of a QR code that changed color and bounced around the screen for the entire 30-second slot. Scanning the code pointed the user to a sign-up page where they could also claim $15 of Bitcoin.
By all accounts, it was wildly popular, with the influx of traffic rendering the sign-up page inaccessible for a period of time. According to Surojit Chatterjee, Chief Product Officer at Coinbase, the landing page in question received over 20M hits in one minute, which he termed “historic and unprecedented.” An error message on the site read, “Well that was more popular than we thought. We need a quick time out but don’t worry. We’ll email you when things are back to normal.”
Even with the sign-up page inaccessible, potentially negating some of the upside in running an expensive ad during the Big Game, it may still be judged an overall success, with app downloads up hundreds of percent, both week-on-week and day-on-day.
It remains unclear why the capacity of Coinbase’s compute infrastructure couldn’t keep pace with demand. The sign-up page appears to have been a temporary one, suggesting the company had done some due diligence on how to handle a potential flood of new sign-ups without impacting the performance of the trading app itself.
Still, this type of outage is rare in 2022.
It also highlights the multidisciplinary nature of lead generation activity. Commercials aired during the Big Game are a big deal for their entertainment value on a yearly basis and are often watched by people who would typically avoid them at all costs. A 30-second spot in this year’s game cost upwards of $6M USD. The planning around these commercials is generally extensive and conducted over months, not weeks or days. It’s hard to imagine the network team wasn’t in-sync with the marketing team on the running of the campaign and aware of the potential effect the commercial could have on the company’s technology infrastructure.
Still, the campaign led to a self-inflicted traffic overload incident that few would have expected.
The other incident this week that highlighted the Internet’s fragility was the reported distributed denial-of-service (DDoS) attack against Ukrainian military and financial institutions. It was described as the largest-ever malicious traffic flood unleashed against the country, and it disrupted access to key services.
As more of our lives are conducted online, across an assembly of interconnected networks with multiple dependencies, we will ultimately run into more breakages, both intentional and accidental. That is particularly the case when resiliency is unevenly distributed.
Every link that makes up the Internet has its saturation or breaking point with the amount of traffic it can handle before being overwhelmed. Building Internet infrastructures that can maintain route diversity and resiliency is key to ensuring digital equity.
We saw this recently with the volcanic eruption in Tonga, which severed the Kingdom’s only subsea cable, and the scramble to reconnect it to the Internet. Again, it shows the fragility of the Internet, and how that fragility and access to resources can have an uneven impact.
Briefly, there were some other outages of note this past week.
On February 11, Twitter experienced a 90-minute disruption, which appears to have impacted North American and UK users, and that was attributed to a “technical bug that [prevented] timelines from loading and Tweets from posting.”
And, lastly, Discord had some general API and session issues on February 15, although they were resolved within 30 minutes.
As always, we continue to watch network infrastructure and applications with a keen interest.