Hacker News new | past | comments | ask | show | jobs | submit
Neither has any relation to the actual reasoning trace. The summary is actively misleading and you'd be surprised how different it is from the raw CoT of the model. Just ignore it.

If you want to see example traces without jailbreaking claude or gpt, see open models that display the CoT without trying to hide it. It's pretty useful sometimes, with usual caveats (it deceptively looks like English but is heavily reward hacked, and it's generally a weak proxy for model's reasoning process).

This makes sense. Nonetheless, I find that reading Anthropic's reasoning traces is sometimes a trigger for researching more stuff. Often I'll see models thinking (or fake thinking, whatever) about something and I'll be curious to check out more about that topic if I'm not particularly familiar with it.
FWIW, you're not wrong. As I said in my other comment, I don't really know what other models are up to but Claude is definitely demonstrating a deep level of understanding, and I learn things from it all the time.

Sometimes I'm like...is everybody else using the same thing that I am? This is insane.

They definitely let it churn until it figures something out, it's not uncommon for it to take 15-20 minutes to answer, is that normal? They just have "relentlessness" maxed out and some kind of second-order structure to orchestrate a bunch of agents into one unit of work.

That's the exact same feeling I have.

It's obvious that what we're getting are summaries. Sometimes I think I even see vestiges of the summarizing model instructions. But, that being the case, it's bizarre why Gemini's summarizer acts like a toddler, and Claude's actually lets me learn and, more importantly, stop it at its tracks when I see it's going down the wrong path.

On the other end of the spectrum, if you toy around with the open-weights models, you see the insanity that is going on in their output. I can totally believe in a scenario where claude is basically doing that and then summarizing it into something coherent, getting rid of the 10 opposing viewpoints, "aha!" and "Hmmm, but wait!", but it does it elegantly.

I've tried constantly to use other models (and I do), but Claude spoils me to no end. I keep thinking I must be holding everything wrong.

> summarizing it into something coherent

This is like, the entire ballgame. This is what effective communication is, this is what understanding is. Anyone who can do this, man or machine, would do so, I would think.

> On the other end of the spectrum, if you toy around with the open-weights models, you see the insanity that is going on in their output. I can totally believe in a scenario where claude is basically doing that and then summarizing it into something coherent, getting rid of the 10 opposing viewpoints, "aha!" and "Hmmm, but wait!", but it does it elegantly.

Yeah, the open weight models (the only ones that show the reasoning tokens) are just absurd. Wait but why oh wait now I have it, this makes total sense!

I find it kindof endearing, tbh as I definitely have similar trains of thought when developing.

Off-topic, but this kind of conversation tells me just how little the average HNer still uses LLMs outside of specific coding tasks, or at least pays attention to them. Which in turn explains so many comments. Probably a good thing though.

You're of course completely right. Interestingly the "summaries" have only become more diverged over time. I now believe the big 3 are barely summarizing at all any more. I deem it likely that the small models don't get the CoT at all, they just completely make up something "plausible" solely based on the user message with a little bit of previous context. Much of the time it will be the complete opposite of something I asked for, or barely related.

Anthropic has changed this up multiple times over the last 12m, I'm talking about the iteration that was around the longest this year. I think they changed it again within the last feww weeks and potentially started actually summarizing again.

Your suggestion is the right one. Everyone who uses LLMs for their job should spend some time with open models like Deepseek and see what the thinking actually looks like.

> Everyone who uses LLMs for their job should spend some time with open models like Deepseek and see what the thinking actually looks like.

I'm not sure DeepSeek 4.x is a good example; its reasoning traces are extremely verbose and detailed. Which is probably one of the reasons it performs so well, and appears to be a deliberate architectural decision to improve skill at the cost of token usage and time-to-answer.

A better comparison might be Gemma or Qwen. Running these locally and with no summarisation, the reasoning traces are not all that different to the big proprietary models in terms of verbosity and content, even if they're not as smart.

> I'm not sure DeepSeek 4.x is a good example; its reasoning traces are extremely verbose and detailed.

That's why I recommended it. It shows that for a given model, it's CoT might look different from what people expect. I also named it because I saw in another comment here - or maybe it was on a thread about Carbon, one of the two - chatter about Gemini 4 outputting reasoning inside its user-facing message and looking like "I'm not sure. Maybe it's A? YES! Now I get it! ...", which is of course the style that Deepseek 4.x's traces also have.

I mean, it does though. The sequence of thoughts that Claude goes through in response to my queries blew me away the first time I used it, and it continues to impress. Just the act of putting those sentences together demonstrates such a deep understanding of what I'm saying. If it's just an approximation, whatever it's actually doing is even better.

I haven't even really tried to use other models in the same way, and this thread makes me feel like I don't need to.