Hacker News new | past | comments | ask | show | jobs | submit
Example: in 2020, a 6-minute hiccup in a network switch caused asymmetric connectivity, breaking their distributed leader election algorithm, making that system read-only, breaking a different system's heartbeat mechanism causing that system to assume a failure, enact a recovery and degrade in performance, causing 25% of API requests to fail for 6 hours.

https://blog.cloudflare.com/a-byzantine-failure-in-the-real-...