What a paper!
And you missed an even MORE relevant excerpt!!
Man and Slave
The problem, and it is a moral prob-
lem, with which we are here faced is
very close to one of the great problems
of slavery. Let us grant that slavery
is bad because it is cruel. It is, how-
ever, self-contradictory, and for a
reason which is quite different. We
wish a slave to be intelligent, to be able
to assist us in the carrying out of our
tasks. However, we also wish him to
be subservient. Complete subservience
and complete intelligence do not go
together. How often in ancient times
the clever Greek philosopher slave of
a less intelligent Roman slaveholder
must have dominated the actions of his
master rather than obeyed his wishes!
Similarly, if the machines become
more and more efficient and operate
at a higher and higher psychological
level, the catastrophe foreseen by
Butler of the dominance of the ma-
chine comes nearer and nearer."Complete subservience and complete intelligence do not go together."
I'm not convinced this is true. Perhaps for a human it is, but we can give an artificial mind whatever properties we want.
Even for people, what about e.g. the extremely intelligent military general who is absolutely loyal to his king? (Of course, some generals do lead coups and you can't know in advance which ones, but I'd think there are plenty who have undying loyalty, and I don't think it correlates to overall intelligence!)
loading story #49225028
loading story #49225121
The whole thing seems to depend upon AI agents objective ie to achieve some objective by any means possible and ignoring any guardrails. The article did not clarify if openAI had any guardrails to begin with while conducting this experiment. For all the talks around how much they invest in AI safety one would expect them to have these common sense guardrails in place or is it just a case of some school children letting their pet monkeys loose deliberately to display how awesome their monkey team is.
loading story #49225426
loading story #49227023
> Complete subservience and complete intelligence do not go together.
Isn't this contradicted by the centuries of slavery in our history? Or is the author arguing that the people who were enslaved did not have human-level intelligence (which would be rather a problematic claim)?
loading story #49228291
loading story #49227468
loading story #49225701
loading story #49227035
loading story #49225700