The main criticism is, imo, that these models cannot think and have no logic. This is proven time and time again. Every day something new comes out, like counting Rs or the ”drive or walk“ test etc. These are the ”fixed“ or rather band-aided on a case by case basis, but nobody seems to be able to fox the underlying problem.

Having done my master‘s thesis on Transformer LMs, I think this is a fundamental limitation of the architecture.

These models can solve math problems and then fail to count the number of problems they solved. Not being able to count to 68 is not intelligence.

Saying it is a 'next token predictor' should not diminish what LLMs are capable of. It is incredible what can be achieved by a simple rule like this.

Next token predictor is not a very good argument. A submarine does not need to be a fish to swim.

It's a statistical machine, so you only predict a statistical Einstein, ie. something that might appear quite like Einstein and even can be fairly accurate in predicting the next word he types, but which is not him, and will be unable to predict the words that are produced not just statistically but with certain intelligence, direction and intuition.

“It” is not “thinking” for christ’s sake, it’s a massive statistical regression curve fit to a massive, frozen library of text. Unless you’re thinking of “thinking mode” which still is just a template that tells the model to ask questions, which are asked one token at a time, turtles all the way down.

LLMs cannot, by their core design, predict next word correctly every time. That is the main thing you are missing. Another thing is that I know that I am not predicting next word based on othr words in my "trainin data", there is much deeper understanding of most subjects that I am using to "predict next word". This may not be true for everyone, I cannot know for sure, but it might be possible that there are people out there who are just biological LLMs, and in that case I am certain those people will not be able to predict next word Einstein would say either.

> If you can predict the exact next word Einstein is about to write, every time, you’re as intelligent as Einstein!

Einstein is not hold in such esteem because he was really could at spewing out text or formal proofs or theorems, but because he was able to invent something no-one else could. Some people say he advanced physics 100 years wuth his theories in relativity, whether it is true or not.

So even if you can predict what Albert is going to write next, you are still merely copying old Albert. They are amazing at processing large amounts of data, naturally, but creating something new and surprising we have yet to see evidence of (the solutions of the millennium problems seem to be brute-forcing a solution based on fuckton of data to me, but I am not a mathematician so I guess it is up in the air still).

> I never understand the “only a next token predictor” argument

Yeah, that's typically the case for people who have no clue about LLMs or AI.