- Are current LLMs conscious?
- Is it possible that future versions of LLMs with similar architectures could be conscious?
- Can any AI be conscious?
If it was true, you can create extreme pain by running a program. You can run the program by simulating a CPU, using pen and paper for memory. So you're essentially claiming that some simulated being is in pain because there are some 1s and 0s on paper. In fact, you can decide to use an arbitrary encoding of the memory, so a sufficiently long sequence of 0s written on paper corresponds to a simulated being feeling pain in some encoding. That is clearly nonsense.
The crux if it is that if you ever break from "the universe can be fully expressed mathematically", you are stuck in the mud of supernatural beliefs.
That's just not true. I've broken from "science can explain everything" and there's no mud. All of my beliefs are backed with careful reasoning. If there's an unknown, I don't fill it with random garbage.
I try to imagine myself long ago, on the outside looking in, with someone explaining to me that extreme pain, wondrous art, hunger, triumph, and despair would all unfold in due time where the rocks were wet and the lights bright enough.
I can imagine myself calling this clear nonsense.
If you don't understand that, how can you assert that it doesn't come from mathematical relationships?
Do you seriously think that there's any chance that writing a lot of zeros on a piece of paper will create a feeling of pain in some conscious being?
If you would have zero knowledge about ICEs, how would you know?
The only strong argument I have against it is the anthropic principle -- there are billions of times more bacteria than humans, so it's overwhelmingly unlikely that I'd be a human rather than a bacteria.
Not a very good argument of course.