> Recent debates have been clouded by a misleading inference pattern, which we term the “Redescription Fallacy.” This fallacy arises when critics argue that a system cannot model a particular cognitive capacity, simply because its operations can be explained in less abstract and more deflationary terms. In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions. Such arguments are only valid if accompanied by evidence demonstrating that a system, defined in these terms, is inherently incapable of implementing . To illustrate, consider the flawed logic in asserting that a piano could not possibly produce harmony because it can be described as a collection of hammers striking strings, or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. The critical question is not whether the operations of an LLM can be simplistically described in non-mental terms, but whether these operations, when appropriately organized, can implement the same processes or algorithms as the mind, when described at an appropriate level of computational abstraction.
This sounds like a dismissal of the argument through a characterized straw man.
That is, it seems that reducing the complexity of the brain to "collection of neural firings" is not being honest about everything involved to a much greater degree than saying neural networks are a "collection of statistical calculations".
I too believe LLM's will grow in complexity, but presently I can not even fathom how they can be compared to the complexity of a system such as the human brain.
Like driving a car — it's transportation, and it will get you where you're going, but it doesn't use bones or muscles. It has many characteristics in common with builogical locomotion, such as energy requirements, intertia, and the need to navigate, but it doesn't involve proteins or sugars really.
GenAI thinks like the human mind in the same way that cars run like the human body.
Similar utility in drastically different ways.
Totally understandable; I don't think we can fully understand the human brain, using the human brain. We can understand its principles (firings and chemistry, structure and specialized areas, etc) but otherwise it's a capacity problem.
And while I can't fully understand myself, let alone another person, I definitely enjoy talking with people and sharing thoughts that I realize I wouldn't have had on my own.
Nobody actually makes this argument though.
https://www.goodreads.com/en/book/show/217432753-the-ai-con
which describes LLMs as "souped-up autocomplete", complex statistics that cannot truly understand anything. A more recent example is this paper:
https://zenodo.org/records/20071869
which says,
> [LLMs], as turbo-charged statistical models (recall their formal relation to logistic regression) can only but provide correlations.
And, of course, the Stochastic Parrot paper is the classic example in this area. It is from 5 years ago, but "LLMs only do statistics / can't understand" is very much alive and active among academics, even if it is a minority position.
That term is used to describe mental aptitude or skills, like the ability to learn new languages or do math.
By the way, I know it's a parody of another story that makes this exact refutation. But I think this only serves to highlight the point.
How do you connect that description to "LLMs could not possibly be good models of some cognitive capacity"?
It reminds me, oddly, of the debate over whether video games can be "art". A turning point was when they actually did something that art does: [evoke profound emotion and thoughtfulness](https://en.wikipedia.org/wiki/Shadow_of_the_Colossus#Legacy) for the player.
(And before that, "[Can photography be art](https://daily.jstor.org/when-photography-was-not-art/)?")
We may not come to something as simple as "machines can be conscious", but we will certainly have to understand consciousness better if we want to refine our questions.
---
Edit: My point is that we don't need to be angry, but we may have to tolerate people expressing their exploration through overly-confident language, and be patient with that.
And Ted here is obviously exploring. His examination of Claude's constitution clearly shows some nuance. He asks:
> So, given that Claude is not conscious, what are we to make of Claude’s constitution?
And his conclusions are split, between this is useful and this is dishonest. It's a great tension IMO.
> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter. This might seem like a reasonable goal to work toward; I think we’d all prefer it if chatbots never emitted sentences such as “You should kill yourself.” However, for all the times that “honesty” is mentioned in Claude’s constitution, I would argue that it is fundamentally dishonest to have a machine emit many categories of sentences, including any sentences using first-person pronouns.