Something I've discovered is that if you're a small team doing something new, off the shelf products/platforms are almost certainly not optimized to your use case.

What looks like absurd scale to one team is a regular Tuesday for another, because "scale" is completely meaningless without context. We don't balk at a single machine running dozens of processes for a single web browser, we shouldn't balk at something running dozens of containers to do something that creates value somehow. And scale that up by number of devs/customers and you can see how thousands/hundreds of thousands can happen easily.

Also the cloud vendors make it easy to have these problems because it's super profitable.

You can run single-node k3s on a VM with 512MB of RAM and deploy your app with a hundred lines of JSON, and it inherits a ton of useful features that are managed in one place and can grow with your app if/as needed. These discussions always go in circles between Haters and Advocates:

* H: "kubernetes [at planetary scale] is too complex"

* A: "you can run it on a toaster and it's simpler to reason about than systemd + pile of bash scripts"

* H: "what's the point of single node kubernetes? I'll just SSH in and paste my bash script and call it a day"

* A: "but how do you scale/maintain that?"

* H: "who needs that scale?"

A very small percentage of orgs, a not-as-small percentage of developers, and at the higher end of the value scale, the percentage is not small at all.

We're almost 100 devs in a few teams - works well. There's a bunch of companies of our size even in the same city.

What's a bit different is we're creating own products, not renting people to others, so having uniform hosting platform is actual benefit.

Most of the ones that are profitable for cloud providers.

In my last job we ran centralized clusters for all teams. They got X namespaces for their applications, and we made sure they could connect to the databases (handled by another team, though there were discussion of moving them onto dedicated clusters). We had basic configuration setup for them and offered "internal consultants" to help them onboard. We handled maintenance, upgrades and if needed migrations between clusters.

We did not have a cluster just for a single application (with some exceptions because those applications were incredibly massive in pod numbers) and/or had patterns that required custom handling and pre-emptive autoscaling (which we wrote code for!).

Why are so many companies running a cluster for each application? That's madness.