I really want capable models local but it’s impossible for me to afford the hardware. If there is a magical way to run glm 5.3 without quantization at 80-100tps with a total spend of less than 10K I’d be happy but that’s impossible now.
$200 a month is only $2400 a year.
That’s 4.16 years until you spend 10k.
I don’t find it reasonable at all. Subsidized Subscriptions or api for cheaper models, is the best value.
But the future of AI is so uncertain, regarding model sizes. I don’t think making huge hardware investments is reasonable for individuals.
loading story #50448232
loading story #50450372