Back in the 90s. I was part of a team working R&D project on flexible manufacturing. One of the central concepts was the use of buffers (also called decouplers) between manufacturing stations. These buffers were the space needed to place not finished products while they waited for the next station. Typically a conveyor belt. The main service is that both stations could run at slightly different rhythm, and they would not be causing problems on the other one. Either starving it or overwhelming it. Queues are pretty similar. You can have a consumer and a producer working and the queue in between prevents them from being blocked by each other. Thus I now see the main role of queues as decouplers.

Once you've read this, pick up Harchol–Balter's book Performance Modeling and Design of Computer Systems: Queueing Theory in Action. It's a really good introduction to the depth of this topic, and you'll come out with superpowers you didn't have before.

Two past submissions with notable discussions:

https://news.ycombinator.com/item?id=39041477 - 18 Jan 2024, 153 comments

https://news.ycombinator.com/item?id=8632043 - 19 Nov 2014, 60 comments

Reminds me of this other queuing theory blogpost classic https://medium.com/swlh/fifo-considered-harmful-793b76f98374

TLDR LIFO (stack, not queue) is often a better choice for many workloads, despite violating our sense of fairness.

[dead]

[flagged]