That is entirely separate to whether or not it would be a meaningful way to understand the world; a convincing story is not the same thing as one that is true.
I am doing my best to communicate with you but to be honest you are not hearing me (across both responses), and I am out of words.
It's difficult to tell who's trolling -- probably best to go with the charitable assumption that everyone is honestly trying to convey their opinion, but mostly talking past each other. Unfortunately these discussions about the nature of consciousness never go anywhere useful.
I think I'm probably in the same boat as you, roughly: a) LLMs are doing something really interesting that resembles in many ways both intelligence and consciousness; b) I suspect they're not actually conscious but I don't know how you'd know for sure; c) it all just drives home that we still don't really know what consciousness actually is. But like (a), it's definitely something really interesting...
"They're made out of weights" works precisely because LLMs really do have this mysterious property that they seem somehow intelligent even though nobody can explain exactly why, and there's active debate over whether they could be considered conscious.
The thing being discussed isn't simply an arbitrary MacGuffin; in both cases the nature of the thing is central to the impact of the story.
"Howdy-doodly-doo! Anybody like any toast?"