Hacker News new | past | comments | ask | show | jobs | submit
Even dumber local models that are heavily quantized to fit on consumer GPUs can be lovely as a dimwitted household assistant. A lot of people look to offload things they already do to LLMs, which can be great with frontier models, but you've got the highest standards and preferences for tasks you already do, so it's the most likely to fall into a "should have just done that myself" outcome. I've found great success in instead lowering my threshold of things I'd take time to do -- I wouldn't spend time on it, but I'd spend 60s prompting an LLM to take a crack. I'm fortunate to have the hardware to run Qwen 3.8 Flash Next at q8, but most of the tasks I have it do would be totally doable for a mid-quant Qwen 3.8 27b.

Some recent tasks I've thrown it:

* Go through my blood test data (from EHR XML dumps from prior years) and give me some over-time charts for consistently-sampled measurements

* Go through my Good Articles bookmarks, epubify them, and drop one down onto my ereader each morning for my commute, then archive the offline version into cold storage

* Go through my music library, identify my most-listened low-bitrate tracks, and source or assemble a shopping cart to get high res versions

* Analyze local tax/foreclosure auctions for vacant land; grab aerial screenshots, tax data, ownership data, and compile a weekly report of things maybe worth bidding on based on some specific selection criteria (fun facts: if you're willing to accept some headaches, 5+ acres can be yours for under $5k, if you're in the right area)

* Dump my grocery store loyalty data, analyze my purchasing habits, and prepare a weekly instacart for my approval that aims to sync with usage habits. Tweak and learn volume/order adjustments over time.

* Given a KVM that can mount ISOs onto the host, reimage two old laptops of unknown status with Debian and install Minecraft so my nieces and I can have a LAN party

* Organize my music (high confidence MusicBrainz Picard matches), movies/TV shows, ebooks, and video game ROM collections with metadata and good folder organization

* Archive bookmarked fanfiction for my offline library (slowly, and respectfully, and with an actual browser. The browser is anti-bot-countermeasured to the gills, but QFN regularly solves hCaptcha and Cloudflare turnstile challenges. Once I gave it the ability to drag, it even reasonably reliably solves drag-shape captchas. A little while back it made a purchase on eBay which is NOT a bot friendly site.)

Would I personally spend time on doing most of these? Generally not. Could an LLM screw them up so badly I regret it? Also generally not. I get the luxury of a psuedo-assistant that doesn't judge how menial the tasks I send it are, and I save time. With a local LLM, my privacy concerns are far less than with frontier models, and local abliterated models are less (or just not at all) twitchy around things that frontier classifiers balk at.

I'm hopeful that big projects like Muse can help to normalize menial task offloading, and that local LLMs provide the bridge for more technically minded folks to use them even more aggressively and limitlessly (as in compute limits/subscription costs) and cheaply.

Trying to get into this too. But my hardware is a bit limited atm. What are you running Q8 on? At what speed?
loading story #50564469