Invariably when local AI comes up, people try to see how it can be used for frontier-level AI tasks like asking it to one shot a web app from scratch and then they get disappointed when they discover it's not performing at the same level as Opus.
But they're ignoring that a local LLM can still be very useful for any kind of repetitive task, things like categorization or generating embeddings, translation or summarization tasks. I'm self-hosting Karakeep and use a local LLM for the AI tasks like embedding.
And once you've got that local LLM running, you can start thinking about integrations with your own custom built apps.
loading story #50563138