Hacker News new | past | comments | ask | show | jobs | submit
reissbaker | on: Kimi K3.1 Pro model weights are out
If you use it day-to-day you can definitely feel that K3 (and K3.1) are much stronger than DS4.1-Flash-1017. There's something the benchmarks don't capture. DS4.1-Flash-1017 is an amazing model for the price, but it's not a very large model in comparison.

I wouldn't over-weight internal benchmarks from a competitor that claim benchmaxxing TBH. Kimi K3 has been at the top of Design Arena for open-weight models for a long long time, and I suspect K3.1 will be there for a long time too. It's pretty hard to fake that one since it's actual human preferences for the artifacts produced.

seizethecheese | parent | next
Speaking of arena comparisons of K3.1 and DS4.1-Flash, we're spinning one up that's more relevant than design, I think: https://agentduel.com/leaderboard

Here, DS4.1-Flash and K3.1 are basically tied, with K3.1 being slower and more expensive. Big error bars, though.

Even with error bars, DS4.1-flash is clearly an impressive model.

reissbaker | root | parent | next
If your benchmark puts DS4.1-Flash at the same tier as K3 or K3.1, I'm sorry to say your benchmark is broken. It really isn't, as much as I wish it was.
seizethecheese | root | parent | next
We allow early voting. This is against what most arenas do, but is on purpose. If the model is faster to do simple stuff, it should be chosen. In this sense, DS4.1-Flash is better on many tasks.

You can remove early voting from leaderboard and they are no longer the same.

purerandomness | root | parent | next
At the point where I choose an expensive model like Kimi 3.1, speed doesn't matter, at all.

Usually, it's because all else failed, and I have to resort to burning Tokens with more expensive/better models, so a few minutes here and there really should not matter.

seizethecheese | root | parent
I think you’re alone to think time for task completion doesn’t matter
rocho | root | parent
But his point is that the faster models could _not_ complete the task satisfactorily.

I've seen this myself too: DeepSeek 4.1 Flash is really good and incredibly cheap at constrained implementations, searches, etc. However, frequently the implementations don't take into account all possible cases they should cover. Then I'm either forced to step in and steer it very precisely, or have a heavier model review and provide detailed feedback. I've had good success with this mixed workflow using GPT 6.1 Sol as reviewer. Then my work focuses mostly at the beginning (architecture, goals, constraints), and at the end (final review or very quick intermediate reviews). I'm not aware of benchmarks that simulate this kind of mixed workflow with different models.

nostrebored | root | parent
But how do you verify a fair or representative distribution of complexity?
seizethecheese | root | parent
We validate people are doing real work using LLM judges.
haizhung | root | parent
I tried both of them for agentic coding and didn’t feel like K3 was adding so much benefit to justify its almost 10x price (on the modem provider I’m on).

In fact, I barely see a difference between the results at all. The models being tied smells correct to me.

I didn’t try 3.1 yet, though.

eunos | root | parent
It's quite hilarious that kimi K3.1 was competitive against Opus 5.5 and a bit lagging against DS V4.1F. While DS struggled against Opus
carterschonwald | parent | next
every time i accidentally use flash versions of deep seek i get really frustrated with the model till i realize that it was the wrong model. im also a little confused because it seems like at least comparisons make it sound like ds is no longer focusing on their high reasoning model family …
CamperBob2 | root | parent
1017 is nothing like previous DeepSeek Flash models. It is vastly superior in all respects.
verdverm | root | parent
I'm still unimpressed with the latest DS4 flash, it makes more mistakes or misses things. Very much like Ember-1 (Kimi K3 fine-tune to reduce thinking), it is not a reliable or trustworthy assistant (ime)
loading story #50378544
potamic | parent | next
And how do you reckon GLM and Qwen stack up?
ricardobeat | parent
How do you use them day-to-day? The subscription lasts for minutes of usage at at time, and API costs are crazy high. There is zero reason to use it over an Anthropic subscription, it is not faster or cheaper - unlike DeepSeek 4.1, MiMo 2.6 or other flash models.