Not the OP but I would assume:
- no data retention policies to worry about in the first place
- no monthly billing outside of power bill and capex
- no jurisdiction issues
- full control over your data
- no network round trips or even worry about network outages in the first place
Exactly. I own the GPUs and bought them all used on ebay. One time cost.
When the internet goes down, my productivity is not impacted, and I have total privacy on my work at all times. Also no rate limiting etc, and I can burn tokens freely.
Maybe a naive question, but how do you even run 8x GPUs? Threadripper Pro? Anything special about OS / software stack?
Sounds interesting, though a bit pricey (if I'm reading the prices correctly it's 20k just for GPUs). Fingers crossed that the prices will fall.