This is a surprising result. With structured inputs like source code, I’d expect grep to outperform semantic search, but natural language’s errors and inconsistencies seem to leave so many cracks for information to fall through.
This paper is based on quality so I don't think it should be that surprising if you take loops into consideration. What the agent finds in the first pass, can help if formulate the next grep if needed.