This is what I've wanted for the last few years. I have played around with local models and had some success, but I don't want to deal with the hassle of setting up, updating, etc.
If all of this were easy, I would happily invest in new dedicated hardware to run it. This could be a full-blown Mac or it could be a souped up HomePod-type device, loaded with RAM. Better yet, make it a revitalized Matter-enabled Airport!
MacMini with M7 Pro and 96G of RAM.
They are going to sell sure 10,000$ and you won’t need anything else for the rest of your life (if you are over 90).
Btw, llama.cpp with llama-swap ain’t that hard to configure if you have access to a potential LLM and have 5h of your life available to that task.
loading story #50450814