When upgrades go bad
One of the cardinal rules of running a large scale Internet business is that you want to have a production network and an R&D network that are independent. You test on the R&D network, then push it out for production. Every once in a while you have to roll back the updates due to errors, but that should not happen very often or at all. I guess Facebook didn't get this memo. Maybe I should write something on their Wall.