UK AISI cyber evals seem to under-elicit capabilities from quirky models [1]. Kimi K3 is a token-hungry model, and I suspect it hit the eval's 100M token limit well before saturating scores [2].

This gap was true for GLM 5.2 as well; they ranked it at Opus 4.5 level [3]. Both anecdotally and with a held-out eval, I've found GLM 5.2 to be better at security research than Opus 4.6 [4]. But it's a quirky model that degrades quickly at long context lengths.

Personally, I'd rank Kimi K3 above Opus 4.8 and lower than GPT 5.6 Sol in its ability to find vulnerabilities and exploit them. But it's not far from the frontier.

[1]: From the UK AISI: "Our setup likely slightly underestimates open weight models’ maximum capability: we didn’t pursue specific elicitation or optimisations which could have improved performance" (https://www.aisi.gov.uk/blog/how-far-behind-the-frontier-are...).

[2]: Their eval also counts cache hits towards the token budget; the 100M token budget is comparable to a ~5M token budget in other evals.

[3]: See GLM 5.2 eval scores in https://www.aisi.gov.uk/blog/how-far-behind-the-frontier-are...

[4]: https://dualuse.dev/posts/chinese-models-are-sometimes-bette...