In the past few months, I've used agents to brute force and reverse engineer solutions to problems I would never have economically have figured out on my own. I did it by putting agents in loops, connected to hardware and the internet, reading technical documentation, and relentlessly trying.
The code was shit. But it's much better to start with working shit and make it correct than spend weeks frustrated that nothing works.
I get that being a domain expert and instantly knowing the output is shit is important, but even if the output looks great, the code can be shit, and it takes looking at the code and knowing something about it to figure that out.
The solution to shit output is not (always, sometimes it is) just another if statement.
Even in a very well specified OSS effort, where I have some expertise, and I carefully reviewed the AI's output every goddamn step of the way, bugs slip through that the agents confidently tell me can't happen, and when shown proof they… add just another if, instead of really questioning assumptions.
You either know what you're doing, or you don't.