Hacker News new | past | comments | ask | show | jobs | submit
topspin | on: Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard
So far, the comments have focused on hopes for the effectiveness of open-weight models versus proprietary frontier models. That's fine, but I'm thinking about the approach and the results, as opposed to the model.

Taking the results at face value, what is it about the "team" that makes the system so effective? That's the key here: presumably, the approach should work regardless of the model, open-weight or otherwise.

My guess would be that the division of labor makes the separate contexts more effective. This is an optimization around the limits of context. Some of that is simply the net amount of context brought to bear: each team member has a context, so the net amount of context is multiplied by the number of team members. Also, since the division limits the scope of each team member's focus, the context a team member has is used more effectively.

edg5000 | parent
I would not think in terms of "how much" context, but really just how the harness manages it. I've been fiddling around with harnesses earlier this year and the rabbit hole goes very deep. I eventually had to drop it because it consumed me, and I can only do one thing well. But what I learned is that there is a lot on the table at the harness level.

The elephant in the room and the constant tension is that the models do get "used" to a certain harness (trained for it by the model labs), but they also generalize well. So one really advanced mechanism could fall short because the model could fail to generalize, whereas in other cases the model might handle large deviations from how a conventional harness works better than expected. This is where the gains are.