Let the conspiracy theories fly.
By Wolf Richter for WOLF STREET.
Anyone who has ever operated a website and server knows how frustrating it can be if something goes awry – such as an update of software or network settings – and the site goes down. In addition to website-specific outages, there have been massive internet outages over the past year that were related to updates of global network configurations. But the Facebook outage this morning is something special, not only because it’s huge and total, but because of the technical details as far as known.
I can already imagine an intern – a highly paid intern – getting blamed for screwing up some small part related to an update; and I can also imagine the silliest conspiracy theory suddenly becoming the actual reason.
“Internet traffic to Facebook services virtually disappeared at 15:39 UTC on Monday, October 4, 2021 with initial indications pointing to a DNS problem,” reported Kentik, a network monitoring company, citing aggregate NetFlow data. For users, going to the websites of Facebook, Instagram, and WhatsApp has now been impossible for hours.
“Earlier this morning, something inside Facebook caused the company to revoke key digital records that tell computers and other Internet-enabled devices how to find these destinations online,” reported internet security focused investigative journalist, Brian Krebs, on his site, Krebs on Security.
These digital records are the records in the Domain Name System (DNS), a decentralized naming system for computers and services connected to the Internet or a private network. These name records allow browsers to find the websites the user wants to see. Without DNS records, there is nothing.
Facebook’s DNS records have disappeared. And browsers that go online to find any of the Facebook sites cannot find them and display an error message.
“We don’t know how or why the outages persist at Facebook and its other properties, but the changes had to have come from inside the company, as Facebook manages those records internally,” writes Krebs.
“Whether the changes were made maliciously or by accident is anyone’s guess at this point,” he said.
For Facebook, which develops and manages a lot of technologies internally, this is a huge problem because its email system is down as well as the domain names have evaporated.
Workplace from Facebook, an online collaborative software tool developed by Facebook, is down as well, according to Ryan Mac, tech reporter at the NY Times. And other internal services and tools are down too. Employees cannot communicate with each other.
So maybe switch to Gmail or phone? Arrrrgh!
The system of electronic badges that lets employees enter into Facebook facilities is also down, according to Sheera Frenkel, tech reporter at the NY Times, based on conversations she had with someone at Facebook. Employees that showed up to help sort through the problem were locked out because the electronic entry badges no longer worked.
Citing Doug Madory, director of internet analysis at Kentik, Brian Krebs said that “someone at Facebook caused an update to be made to the company’s Border Gateway Protocol (BGP) records. BGP is a mechanism by which Internet service providers of the world share information about which providers are responsible for routing Internet traffic to which specific groups of Internet addresses.”
“In simpler terms, sometime this morning Facebook took away the map telling the world’s computers how to find its various online properties,” Krebs said.
The Facebook domain, facebook.com, name has now shown up for sale on the internet, but that seems more like a joke. It seems unlikely that this will work and that someone could actually sell it for a gazillion bucks.
Whether coincidence or not, the outages occurred following the 60 Minutes interview with Facebook whistleblower Frances Haugen.
Maybe someone inside decided to unplug Facebook to let the world know how tough it is living without it. Others responded with “good riddance.” OK, now I’m starting my own conspiracy theories.
The stock [FB] dropped 4.9% during the day, amid a broad tech sell-off that left the Nasdaq down 2.1%.
Enjoy reading WOLF STREET and want to support it? You can donate. I appreciate it immensely. Click on the beer and iced-tea mug to find out how:
Would you like to be notified via email when WOLF STREET publishes a new article? Sign up here.