Think of a typical loop we may ask of Claude Code today (assume we are not using TDD): run some test suite with fail fast mode, diagnose if the failure is due to recent feature changes (pass reference to backend/frontend, github issues, PRD,...). Ask CC to decide if test failed due to feature change and then update the test. Perhaps ask CC to use sub-agent to investigate and fix (if deemed so). Commit each fix, move on to next.
I know, this has so many ways to make blunder but I am talking about the agent here, not our error-prone test maintenance. What if we had an agent that had context of your codebase, deterministically ran test suite, linter, hooks, etc. The "English" prompt would become a code loop with the LLM only brought in to decide if a test has failed because of feature change. Also, we can extract git log, JIRA and what not.
Each tool here is real code. Executable code that calls others and only prompts when they meet edge cases. Edge cases are defined but we can now accelerate the maintenance of these tools using agents themselves. But the system is built on "programs that do one thing and do it well" and then reach out to an LLM for its specific edge case. The agent is how these executables work with each other.
There is this ACM blog post called "Manual Work is a Bug" [0] that was originally written to help humans automate processes using code. I find it just as applicable today as when it was written. You and the LLM look at what has to be done and then figure out the scripts/tools to make it happen. You then tie those tools into a system.
The more I use the above the more it makes sense and the worse the whole "just commit the prompt" seems like nonsense.
By trade I am a .Net software developer so as a lot of people would imagine — I was not able to accept a script that wouldn’t be reusable and flexible, basically over engineered.
I do quite some devops so I finally had to accept the fact that I can write simple script with hardcoded values that will live on a server (where I can copy paste and change values to meet other server) and most likely I will not have to look at that script for years as it will be running with cron doing its job without an issue.
Over engineered scripts designed from get go always required debugging from time to time so lots of time I was just doing stuff manually to make it quicker.
So I started winning when I accepted first script can be really simple and when needed I can move it to be parametrized but if not it will just keep doing it's job there on the server.
Gherkin style tests also come to mind
OP's idea "everything is a text file" is good and I use it too. My plans are saved as task.md files, numbered and named. Work items are checkboxes inside the file, closed work items are checked and a comment is added on the same line to provide feedback about the implementation.
I also keep a current-state-of-the-world document, it should be <20KB of text, keep the essential decisions and intents. Loading it allows resuming in <30s.
Something I never saw anyone else do - I save all user messages in a chat_log.md file which is referenced for intent alignment and state recovery. I consider the chat log on the one hand, and coded tests on the other hand as the two walls, the agent works in the mid section between them.
https://horiacristescu.github.io/claude-playbook-plugin/docs...
What I am saying is the opposite - use Claude Code or whatever else - generate actual "programs". Basically scripts. We have tons of ways for "programs" to interact with each other. Then have clearly defined edge case handlers - think "try/catch". How far do you want to go down the rabbit hole in the "catch"? Do you want to re-write a new version of the "program" itself? I do not know, but this type of a system is what Unix already is, with the addition of programs themselves reaching out to LLMs in well defined edge case handlers.
As coding agents have accelerated my work, I just build tons of tooling around existing software. Or in rare cases build new ones. If we zoom out of software engineering, we will still be in the realm of files - text or binary. That does not change.
The question is - do we let agents run the tools or the "programs" call the LLMs. The OS is the new agent, but not the same sense of "agent". I want LLMs to be lightly sprinkled in a future "agent" OS, not the other way around.