The main criticism is, imo, that these models cannot think and have no logic. This is proven time and time again. Every day something new comes out, like counting Rs or the ”drive or walk“ test etc. These are the ”fixed“ or rather band-aided on a case by case basis, but nobody seems to be able to fox the underlying problem.

Having done my master‘s thesis on Transformer LMs, I think this is a fundamental limitation of the architecture.

These models can solve math problems and then fail to count the number of problems they solved. Not being able to count to 68 is not intelligence.