These things don’t think. We’re going to have to reiterate this for a long time, I fear.
There is now a trillion-dollar industry bent to the task of convincing people these things can think. It’s gonna cause some damage.
loading story #48259115
…but they reason well enough given enough context (using their matmuls).
To this day frontier models think that A and not B means A and B when the sentence gets pushed far enough back in their context window. The context length that model can reason over without obvious errors is much smaller than the advertised context. Between a 1/4th to a 1/20th what is advertised on the tin.
Do you also happen to remember what you ate last thrusday?
loading story #48258993
loading story #48260162
Critiques like this tend to focus very hard on what models can't do. It's true, they have limitations.
But they're also superhuman in so many other ways. It's valid to point out limitations, but that doesn't support the conclusion that models are not incredibly powerful and capable of the functional equivalent of reasoning at human or superhuman levels in many scenarios.
loading story #48261735
loading story #48259793