Well said.
I think it is obvious and it has been obvious from the start that next token predictors are not conscious.
Of course the extremely predictable clapback from AI-psychosis enjoyers is typically "you're just a next token predictor too!" but that is so obviously incorrect and misanthropic that it can be dismissed out of hand and doesn't really merit much further consideration.
Consciousness is an extremely confusing, ambiguous topic, and no one has a good way to establish it, or even define it. But it seems to demand people make very strong statements about what is and isn't conscious, entirely driven by convenience and emotionalism. (Curiously, very few people who think that bags of chemicals and action potentials give an entity a conscious soul are eager to extend that to other animals, with broadly similar hardware.)
Whether it’s misanthropic or not has no bearing on whether that’s true. That’s basically saying you don’t like a truth therefore anyone who claims that that truth is true is a bad person.
> so obviously incorrect
It should be easy enough to explain why that’s incorrect then
And it is accurate to depict this kind of argument is misanthropic, because it is already directed at other people. Nobody says, "If AI is not X then what about the fact that I lack X." It's always other people. It's transparent. The person is always saying, "AI is useful to me because it can do X. Many people I interact with can't do X and it drives me crazy, because I view others as a means to an end and not as ends in and of themselves."
I'd say people who have the lived experience of, well, living, are well aware that the brain is much more than just a token predictor.
I've reduced the complex inner workings of our brains and the rich experiences that it delivers to what I imagine its singular goal is: modeling the world around us in a useful way so that we can generate predictions about what will happen next, which confers considerable evolutionary fitness. You'll find that a similar reduction of what LLMs are actually doing down to "predicting the next token" also ignores the other mechanisms at play and the results of those mechanisms for the sake of preserving a preferred viewpoint (the re-description fallacy, as someone else put it).
You posited that no one genuinely believes this comparison is apt, and I want to make you very aware that people do, and I in particular do. The underlying mechanism being explicable does not, in my estimation, deprive it of any capabilities of producing richness of inner experiences.
Me pitying you is both genuine and an insult, because you are insulting me and every other person with your dehumanizing theory, even if you don’t see it that way. People like you should be pitied so that their dehumanizing beliefs can be appropriately contextualized and discounted.
Ill leave it at that.
In the same way that the sound waves and facial expressions I produce are not conscious, the output json of an LLM is obviously not conscious either.
The locus of consciousness and subjective experience may be in the computer, either at inference time or training time..
The software that does the inference is clearly just computer code.
What we're left with is a fictional character being briefly conscious while its dialog is being written, which is pretty absurd.
- Monadology, Section 17
Conscious self-awareness is neither scale invariant nor independent of substrate. Computational theories will never account for it b/c computational abstractions are both scale invariant & substrate independent.
Humans have been trying to define our experience and the nature of that experience throughout history, and often we end up using myth to point to the thing that we cannot describe in concrete terms. The process of experiencing that myth through rite and ritual in the Greek mystery traditions, or Christian mysticism, or Islamic Sufi dance and song, or Buddhist meditation all points to something that cannot really be reduced to description. I know that folks on here will balk at the idea that something that is experienced cannot be described, but honestly if we could accurately describe something in adequate terms that capture the whole of the thing wouldn't we have done so by now?
Maybe consciousness is best understood in the silence of merely experiencing it. Maybe we can't say that AI is conscious or not, but does that question really matter?
To be fair, that's the best thought terminating cliché, which saves you having to explain what you mean by consciousness.