If anyone wants to argue that it is inefficient writing your own AI coding harness, I wouldn’t disagree. That said it is satisfying to have long coding sessions using my own tools.

The article is a valuable resource, thanks to the author.

One practical failure mode I’ve have experienced in my agentic harness tasks similar to the “weak evaluators” point: an incomplete check suite that still reports full success. That’s was worse than a weak evaluator, because it made it look correct and decisive.

After several trial and error, what helped was fail-closed on coverage i.e if the fixed checks for each operation aren’t all there, nothing ships / nothing gets sent. Otherwise you can “pass” while never running the cases that would have failed.

install or build stuff that they can then use

it changes the environment instead of the agent/harness but in a sense how separate is the agent from its environment and why do we apply this distinction re self-improvement?

animals and humans do the same thing and are great at it, without 'self-improvement' with emphasis on the 'self'