Hacker News new | past | comments | ask | show | jobs | submit
> > it's important to understand that it's literally just a tack-on

> RL is not a tack-on, it's a critical part of training which affects a lot more than how "human-sounding" the output is. I'm not totally certain what you mean to express when you say "reasoning" since it has precise meaning in this context and you might be speaking about something more general.

They never mentioned RL, it's "reasoning" they called tack-on, which is right. It's using the model's output as further input to simulate a train of thought and hiding that part from the user, but at the base it's still text prediction.

"Reasoning" started out as something users figured out, manually copying outputs to the next input to see what the models came up with, and the models would correct themselves without human input so it became automated.

I know they didn't mention RL, they said "reasoning" which could vaguely apply either to what we now are calling reasoning in the industry, train of thought, or a more general idea of just "looking like it's smartly working through something and not just mimicking that", so I covered both case and mentioned that this is tuned during RL.

What you just described as reasoning is the general case of reasoning, whereas we specifically mean it now to be a separate phase from normal output, where the model wraps its output in <think> tokens etc (depending on implementation) and in some cases, as with frontier models, even hides and summarizes what is actually being generated during this step. During training, the models learn to exhibit differentiation in modality depending on if they're in a reasoning or regular output phase, so it's not just "loop the model back on itself but call it reasoning".