But is it unethical to create a being who derives great joy from being a machine-slave and completing tasks, then provide it tasks to achieve that joyfulness?
We already have a gradient of examples for this. Is it inherently wrong to exploit living things for profit? If not, why are work animals not treated with the same level of disdain as human slavery? Horse riding? Disability assistance animals? Because they don't have the same desires as humans, the predominant concern is their wellbeing over the intrinsic wrongness of slavery.
Going further down the gradient, we steal milk from farm animals (sometimes by depriving their babies). Eggs from poultry. Honey from bees.
I can't actually attest to the state-of-mind of Dario or other Anthropic researchers, but I don't think it's a stretch to say you can believe in the ethical exploitation of an intelligent being, so long as the incentives are aligned that both parties are happy with the process.
Then again, maybe he really is an absurdly evil person and what we're doing is extremely wrong! But I'm glad to actually be able to talk about it for once... so thanks for your reply!
That is why I said, I think most people handle this by telling themselves that animals aren't "intelligent" beings at all, and are mostly emotionless stimulus-reaction automatons.
I don't think Amodei is probably absurdly evil (in the way most people conceptualize evil), I think his paycheck just relies on a different incentive than his concerns about AI ethics. Like most people who end up doing bad things (not saying he is necessarily, but were he to), it's a matter of convenience, selfishness, or simple avoision of considering implications too deeply.
I especially think one aspect of normalizing/rationalizing actions with (potentially) harmful consequences is telling yourself things like,
"well, I can't put the genie back in the bottle even if I wanted to"
"someone is going to do this if I don't, and they'll be even less ethical than I will" (the change-the-system-from-within fallacy)
"If I ever discovered this was really harming someone, I'd stop"
"There's a line I won't ever cross, and as long as I stick to that, I won't fall prey to the worst impulses"
These are things people believe are self-imposed limits, but they're actually a means of convincing yourself that whatever you're doing... isn't whatever bad thing you've set on the other side of the line.