On October 4, 2021, if you typed facebook.com into a browser, it could not even find the address. Cloudflare runs the public lookup service 1.1.1.1, and it saw facebook.com stop resolving at about 15:50 UTC and come back at about 21:20 UTC: roughly five and a half hours. Facebook, WhatsApp and Instagram were all down. That same day, Facebook's first post named a single root cause: "We want to make clear at this time we believe the root cause of this outage was a faulty configuration change."
The next day Facebook (the company now called Meta) published a longer account. It never uses the words "root cause". It reads like a chain of failures, each one linked to the next. So what caused the outage, and is "the cause" even the right thing to look for? Twenty three years earlier, a physician named Richard Cook wrote eighteen short points about how complex systems fail. He wrote them for medicine, and they fit this outage almost point by point.
This paper is hosted by its original publisher.
Read on adaptivecapacitylabs.comThe Paper in Its Year
Cook's paper is credited to the Cognitive Technologies Laboratory at the University of Chicago and carries a 1998 copyright. It is tiny: eighteen numbered points on about four pages, each one a bold sentence followed by a short paragraph. The oldest text online is Revision D (2000), which this write-up reads; Cook's 2018 Revision G says "The substance of the body text is unchanged from the original 1998 version". Page numbers below are Revision D's.
The long subtitle tells you what the paper covers:
(Being a Short Treatise on the Nature of Failure; How Failure is Evaluated; How Failure is Attributed to Proximate Cause; and the Resulting New Understanding of Patient Safety)
(p. 1)
Proximate cause is the legal term for the direct cause, the one that set the events in motion. "Patient safety" tells you where Cook came from, yet his first point names healthcare right next to transportation and power generation.
The video groups the eighteen points by that subtitle into four parts. The grouping is ours: Cook simply numbers them from one to eighteen.
