Hacker News new | past | comments | ask | show | jobs | submit
kouteiheika | on: HarnessTax: How Much Does the Harness Matter for Coding Agents?
> It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools.

This is true. For example, the other day I was doing some testing with DeepSeek-V4-Flash-0731, and I gave it a tool called "EditFile" with "old_content" argument. It ignored the tool definition and called it with "old_string" instead, presumably because it was trained on a bunch of Claude Code traces.

So either make sure your tools look nothing like the tools in the popular harnesses, or make them similar.

raffael_de | parent
> or make them similar.

identical. similar is what you tried and it didn't work.

kouteiheika | root | parent
No. You don't need identical. For example, I have extra arguments in my edit file tool that the model can specify, and that works just fine.
dns_snek | root | parent
Backwards compatible then.