The Facebook Outage — What BGP and DNS Taught Everyone This Week

On Monday, Facebook disappeared from the internet. Not "was slow," not "some users affected" — the domain stopped resolving, along with Instagram and WhatsApp, for roughly six hours. Billions of users, gone dark, because of a configuration change made by the company itself. Facebook's engineering team has now published its own account, and the outline is remarkable: during routine maintenance, a command intended to assess backbone capacity instead took down the connections between Facebook's data centers — and in the cascade, Facebook's BGP routes to its own DNS servers were withdrawn from the internet.

If you run any kind of online service, this week's outage is the most instructive incident in years — not because you operate at Facebook's scale, but precisely because you don't. The failure modes are universal; only the blast radius was exceptional.

What actually happened, briefly

Two acronyms carried the whole story:

• BGP (Border Gateway Protocol) is how networks announce to the rest of the internet "these IP addresses live here, route traffic to me." When Facebook's DNS servers detected they'd lost connection to the backbone, they responded — by design — by withdrawing their BGP announcements. The internet's routing tables simply forgot where Facebook's DNS servers were.
• DNS then finished the job. With the authoritative nameservers unreachable, facebook.com, instagram.com and whatsapp.com stopped resolving everywhere on Earth. To every resolver, every browser, every app, the domains effectively ceased to exist.

The cruelest part came next: Facebook's own engineers reportedly couldn't reach the internal tools needed to fix the problem, because those tools lived on the same infrastructure that had just vanished. There are widespread reports that even badge access to some facilities was affected, forcing physical access to data centers to reset hardware locally. The remediation took hours partly because the people fixing it were locked out — digitally and, reportedly, in some cases literally.

Lesson 1: don't put all your eggs in one infrastructure basket

Facebook runs its own DNS, its own backbone, its own everything — and one bad command took it all down together because it was all one system. Most teams have the opposite failure: everything on one cloud account, one DNS provider, one hosting box, one person's credentials.

You don't need Facebook's budget to apply the principle:

• Separate your DNS from your hosting where practical, so a hosting-level failure doesn't take out your ability to repoint traffic.
• Keep your status page on completely separate infrastructure — different host, different DNS, ideally a different provider entirely. A status page that shares fate with the thing it reports on is a decoration. Monday's masterclass: when everything is down, the only thing users and support staff have is the one page that isn't.
• Ask the fate-sharing question in reviews. For each critical component: "if X is down, can I still see, reach, and fix Y?" Monday's answer at Facebook was no, several layers deep.

Lesson 2: external monitoring tells you what users actually see

Inside Facebook, dashboards presumably showed servers running fine — the machines were healthy; the internet just couldn't reach them. This is the defining blind spot of internal monitoring: it measures your infrastructure from a vantage point your users don't have.

External monitoring — checks that originate from the public internet, resolving your DNS the way a stranger's laptop would — is the only view that matches reality. It would have said, immediately and unambiguously: the domain does not resolve; nothing else matters right now. For a normal team the equivalent scenario is mundane and common: your registrar suspends a domain over a billing hiccup, a nameserver change goes wrong, an expired domain auto-parks. Server dashboards stay green through all of it. Only an outside-in check notices — and DNS-level failures deserve their own checks, not just HTTP ones, because knowing which layer failed is half the diagnosis.

Lesson 3: DNS deserves respect and redundancy

DNS is the layer everyone forgets because it almost always works. Monday is what it looks like when it doesn't. Practical hygiene for teams of any size:

• Use more than one nameserver, on more than one network. Most managed DNS providers do this by default — verify yours actually does, rather than listing four names that resolve to the same infrastructure.
• Know your TTLs. Long TTLs (hours) mean caches shield you briefly during a DNS outage but also mean emergency changes propagate slowly. Know what yours are set to before the incident.
• Monitor your DNS records themselves. Unexpected changes to A, MX or NS records are an early-warning signal for misconfigurations and hijacks alike.

Lesson 4: out-of-band access to your own systems

The detail everyone will remember from this outage is engineers locked out of the tools — and reportedly the buildings — needed to fix it. Scale it down and it's a familiar story: the runbook is in a wiki that's down, the SSH bastion resolves via the DNS that's broken, the admin panel is behind the SSO that's offline.

The fixes are unglamorous and cheap:

• Keep copies of critical runbooks somewhere that shares nothing with production — even a printed sheet or an offline doc.
• Maintain a break-glass access path: a direct IP for at least one management host, provider console credentials stored outside your own SSO, a list of provider support phone numbers that doesn't live on the infrastructure it's for.
• Test the path. An untested emergency access route is a hope, not a plan.

Lesson 5: hours of downtime happen to everyone

The last lesson is one of humility. Facebook employs some of the best infrastructure engineers alive, and it was hard-down for six hours — by some estimates one of the costliest outages ever, with lost revenue in the tens of millions of dollars. If it can happen there, your architecture will not be the first in history to achieve immunity. The goal isn't zero incidents; it's knowing within a minute, understanding which layer broke, and having a fixing path that doesn't depend on the broken thing.

That first part is what CompleteStatus does: uptime, DNS and SSL checks run from outside your infrastructure, so when your domain stops resolving or your site stops answering, you hear about it from your monitoring — not from Twitter. Alerts go to email, Slack and webhooks the moment an outage is confirmed, and because the monitoring lives on infrastructure that isn't yours, it keeps working precisely when yours doesn't. If you've recently moved everything to HTTPS, pair it with the hygiene in our guide to fixing mixed content warnings — and if you don't have outside-in monitoring at all yet, this week is the argument. Set it up free in a few minutes.