The difference between a distributed system and a monolith is that in a monolith you have some idea what your single point of failure is, but in a distributed system you don't know your dozen single points of failure until they happen in production.
Example: in 2020, a 6-minute hiccup in a network switch caused asymmetric connectivity, breaking their distributed leader election algorithm, making that system read-only, breaking a different system's heartbeat mechanism causing that system to assume a failure, enact a recovery and degrade in performance, causing 25% of API requests to fail for 6 hours.
https://blog.cloudflare.com/a-byzantine-failure-in-the-real-...
I love the dependency cycles that prevent things from being started up again from scratch.
loading story #50481015