I asked Gemini to review the repo. Apparent issues include:

  - Corrupted/dropped Gemini thought_signature (reasoning_details) in tool loops
  - Blinded model.api.id = "contestant" bypasses gemini.txt and serves Claude's prompt (default.txt)
  - Anthropic-only prompt caching

The full report is here, if you're interested: https://docs.google.com/document/d/1-HqujXuDRJSWJIqEvmctcINr...

Prompt was:

> I read on hacker news the following, which does not match my expectations. Check if something in the harness or bench is hurting gemini perf.

followed by your post.

That doesn't seem to match the arena.ai rankings, who I'm assuming has a far larger dataset of rankings than you currently hold.

https://arena.ai/leaderboard

I've used argon via `agy` cli tool and was not impressed at all. However, I decided to burn up the remainder of my ultra credits on a visual reasoning pet project.

Argon in low and medium mode had higher quality results than it did when running in high mode, almost every time. My tasks involved the agent finding some details in a raster image and then mark it up with predictions about what it found.

Argon on high would do the work, and then a whole bunch of other stuff i did not ask for. All that extra broke everything else downstream from the analysis.

However, doing the same work with Argon in low effort was on par with results from Haiku 5.5 (xhigh) and GPT-6.1-Terra (medium). So, Argon low works better than Argon high, but for my tasks -- a very weak model relative to how google positions it.

I'm ready for Geimini Unobtanium!

Well, my perspective is somewhat narrow (I use it primarily for software engineering and pretty technical data engineering work), so it's possible that Argon is well-suited for the work I do but not for other tasks. I suppose it's just more data points to consider.

This does not mesh with other battles like Arena where Argon is nearer to the top.

> local coding agents

what does "local" coding agents mean? Surely you're not hosting it..

Have you considered that the harness might be the issue?

my anecdotal experience is that argon and muse wonder full are both a full tier below opus/sol.