I do have a log reading / action / resolution product in the works.
Its sort of like having a sysadmin constantly watching. Poking through logs pcaps & monitoring for unusual activities.
Also works with local models well enough.
I love how LLMs have brought DIY software back. Need an app? Just make it. Like back when you’d write programs in BASIC
loading story #50303775
loading story #50303644
That's cute but bill for going thru even tens of log lines per second gonna be more expensive than entire rest of the infrastructure.
loading story #50312827
I’ve set up something like that with Opus. Works well enough, and points out CVEs to fix on the regular.
Running NixOS helps.
I've set up something like that with a small local model in my homelab.
How does it resolve the prompt injection?