We allow early voting. This is against what most arenas do, but is on purpose. If the model is faster to do simple stuff, it should be chosen. In this sense, DS4.1-Flash is better on many tasks.
You can remove early voting from leaderboard and they are no longer the same.
At the point where I choose an expensive model like Kimi 3.1, speed doesn't matter, at all.
Usually, it's because all else failed, and I have to resort to burning Tokens with more expensive/better models, so a few minutes here and there really should not matter.
loading story #50375162
But how do you verify a fair or representative distribution of complexity?
loading story #50375155