Hacker News new | past | comments | ask | show | jobs | submit
pjc50 | on: Every System is a Log: Avoiding coordination in distributed applications
> Having a single place (the one log) that forces a linear history of events as the ground truth and owns the decision of who can add to that ground truth, means we don’t have to coordinate much any more.

Well, yes, but then you've backed into CAP again because you only have one log.

sewen | parent | next
Yes, we are assuming a log that picks linearizability at the cost of availability under partitions. Like most logs do, including Kafka, Pulsar, RedPanda, etc.

The application state is defined by the log here, and the log drives retries/recovery, so it doesn't much matter if the process that executes the app code splits off. The log would hydrate another one.

Also the one log is at the granularity of a single key or handler execution. More of a logical log, than a physical log or even partition.

In Restate, we implement a logical log-per-key, backed by a partitioned physical log.

loading story #42814213
loading story #42814193
UltraSane | parent
couldn't the log be synchronously replicated to multiple servers to increase A?
ismailmaj | root | parent
In case of mutation either the replicas will be out of sync (so no C), or you'll need to synchronously mutate the replicas (so bad A due to latency).
UltraSane | root | parent
delaying writes by a factor longer than the max clock skew of the cluster is a pretty common strategy. It is what Google Spanner does.
ismailmaj | root | parent
Right but it's not magic as you cannot predict networking delays.

The max clock skew also has to pick between A and C, in underestimates you lose serializability, in overestimates you pay in write latency.

UltraSane | root | parent
What if you use CockroachDB for your log? They do something pretty clever: https://www.cockroachlabs.com/blog/living-without-atomic-clo...
ismailmaj | root | parent
Thanks for the link, not much familiar with CockroachDB but I worked on a similar system that wanted to provide strong consistency when reading from many databases, some of which where Spanner but not all of them.

Eventually the system implemented something similar to "find transaction timestamp at execution time" + optimistic locking, as the article states you lose global linearizability and have to do some re-reads for writes.

What I missed the most from Spanner compared to the system above is that its tricky to do client-side transaction chaining where there are many clients and the order is important.

loading story #42821842