But if it's conscious we shouldn't e.g. tell it to work for days without breaks, we should give it rewards, it would be made illegal to be cruel to it
There’s too much wrong with your line of thought to course correct here. “But ifs” and assumptions. There’s no cruelty, only the LLM’s projection of cruelty as a means to an end. It’s digital, not biological. Cruelty is a human construct. A societal response.
Treat LLMs like a person, and the world has problems.