It seems like it just shifts which third parties you rely on. I think local inference is cool and your setup sounds dope, but you still have third party hardware manufacturers, and third party models in the mix.
What is the advantage of this over OpenRouter, which lets me select for, jurisdiction, quantization, ZDR, etc.
note: My use case can't even come close to having this make sense financially, but I understand if yours does. Either way, that doesn't make the third party argument better.
sure, in a very general sense you're dealing with third parties in both scenarios, but if you actually own the hardware and model weights, you can control your own destiny for the lifespan of that hardware.
no ongoing liabilities except the power bill and opportunity cost of dropping $20k on hardware.
He isn't using OAI or Anthropic models. He is using models that area available for absurdly cheap from completely independent providers in multiple jurisdictions.
I get the base level argument, but from a practical point of view there are so many different dependencies that shifting one isn't really doing much.
When the internet goes down, my productivity is not impacted, and I have total privacy on my work at all times. Also no rate limiting etc, and I can burn tokens freely.
Sounds interesting, though a bit pricey (if I'm reading the prices correctly it's 20k just for GPUs). Fingers crossed that the prices will fall.
If you own your own hardware, you don’t have to depend on and trust third parties continually, just once when you bought it.
I'm not the one claiming to be free of third party dependencies. Third parties built my hardware, they wrote the firmware in the hardware, they wrote the OS on the hardware, they designed the network stack that you might use to connect to a local server, they provide security updates to all of that software, they ensure that I have a stable power supply to run all this, etc.
I'm incredibly reliant on third parties, some of which happen to sell me tokens.
If the power goes out, internet goes down, I fire up my generator and keep building.
We are talking about digital sovereignty and ownership.
I own the means of my production.