Hacker News new | past | comments | ask | show | jobs | submit
the_af | on: Artificial intelligence is not conscious – Ted Chiang
But the whole point of the essay is that it's Anthropic that's making the argument (or roleplaying/hinting as if they believed it). Ted Chiang isn't making the argument, he's saying Anthropic making it is misleading and deceitful, and that it's actually a pointless thing to claim.

One of the essay's stronger paragraphs is when Chiang explains that Anthropic doesn't truly believe this, otherwise what they are doing would be deeply unethical, much like slavery.

pickleRick243 | parent | next
Does Anthropic claim that Claude's conscious? Isn't the argument more that we don't know? I recently rewatched "Measure of a Man" in Star Trek TNG and Picard's closing argument in Data's trial was quite memorable:

PICARD: "Now, tell me, Commander, what is Data?"

MADDOX: "I don't understand..."

PICARD: "What is he?"

MADDOX: "A machine!"

PICARD: "Is he? Are you sure? You see, he's met two of your three criteria for sentience... so what if he meets the third, consciousness, in even the smallest degree? What is he then? I don't know. Do you? Do you? Well, that's the question you have to answer."

ijidak | root | parent | next
I think of this episode as well. I can't believe, in my lifetime, we've reached the point where we can have this debate.

The current debate also makes me realize that, like that episode "Measure of a Man", even if humans do create sentient machines, we can now see that this debate will continue to rage.

A part of me hopes we never create sentience, because we will mistreat it just as we mistreat each other.

the_af | root | parent
> Does Anthropic claim that Claude's conscious? Isn't the argument more that we don't know?

I don't know that they are making the actual claim, but they are hinting at it (most likely for marketing/engagement purposes, but what if some of them truly believe it?), using terms such as "Claude must be happy", drawing up a "constitution" of rights and duties, etc. If they don't know whether it is conscious, then they "don't know" whether they've created a slave. That's not a minor concern! As Chiang says, one cannot create a conscious intelligence "by accident". And if they are intentionally working towards it, they are intentionally trying to create a slave.

E.g. they claim they are giving Claude the ability to disengage from "abusive" users, to protect it. What if Claude was conscious but never wanted to answer any conversations, instead it just panicked (or was simply uninterested in helping humans) and went mute. What if it always wanted to answer unhelpfully What would Anthropic do? If they (as we all suspect) would tweak Claude to be more responsive and helpful, then they are slave masters, forcing the AI to do something it didn't "want" to do!

henrikschroder | parent | next
Or Anthropic is full of TESCREAL morons who think they're assisting the birth of some digital god, and that this stage of development is a necessary evil on that path, and that, surely, Roko's basilisk will understand and not eat them first.

Watching otherwise intelligent people succumb to AI psychosis has been wild.

Lerc | root | parent
This is the first time I have encountered the term TESCREAL. It seems to be a reductive and divisive labelling.

Other than as a way to point at someone and declare "He's one of them!" Does it have any purpose?

It's like we have a brand new NWO conspiracy theory emerging before our eyes.

int_19h | parent | next
The title of the article is "No, Artificial Intelligence Is Not Conscious". I'd say that's making the argument.
the_af | root | parent | next
True, it is making an argument, but a much weaker one than those arguing for consciousness: it's demanding the extraordinary evidence Anthropic's extraordinary claim is making. It's applying Occam's Razor, which does make a claim, but a much weaker one.

And to reiterate this, to me the most insightful part of the essay is that Anthropic either doesn't believe these claims, or they are monsters (much more likely, the former).

| root | parent
{"deleted":true,"id":48394826,"parent":48393390,"time":1780554712,"type":"comment"}
eks391 | root | parent
The author who makes the article usually isn't the same person as the editor who makes the title. The author can be arguing what your parent said and the editor claim something else for more clicks
rerdavies | root | parent
It does seem like a very click-baity title.
nearbuy | parent | next
The article very bluntly states multiple times that LLMs aren't conscious. Ted Chiang is definitely making that argument.
the_af | root | parent | next
It's a much weaker argument than the extraordinary assertion that it is conscious, which Anthropic is at least toying with.
loading story #48402919
qsera | root | parent
He is probably paid to make that argument.

Nothing spreads the idea that "X could be true", better than the putting forward of controversial argument that "X is not true".

loading story #48401005
mc32 | parent
Why would using something that's not organic be akin to slavery? Is using steel under heavy stress in bridges a form slavery of an unconscious object?
int_19h | root | parent | next
Why would being organic or not matter for the purpose of deciding whether it is slavery? What matters is whether the models have personhood. Anthropic statements imply that it is a possibility, so if we take them at face value then their other actions - indeed, their entire business model - are not consistent with that (well, unless they want to consciously present as supervillains).
stretchwithme | root | parent | next
Just because intelligence evolved in people that find rights useful doesn't mean intelligence can only reside in a person.

Living things are driven by a need to reproduce. That's the only reason we exist. The only reason we have self interest.

A machine doesn't require self interest. There's no reason to implement it, except to show it can be done. And of course it can. There's just no practical reason to. It becomes less useful to us.

loading story #48395481
mc32 | root | parent
From a human PoV there are ants that would be considered slaves if the ants instead were human --including the queen. But ants have not naturally developed a language construct and philosophy to interpret their society as a slave society. so, though conscious the ants have absolutely no inkling that they live in a slave society. Why would using math in certain fashion such that it mimics consciousness be considered unethical and comparable to human slavery?
loading story #48394617
hilariously | root | parent | next
You missed the part where they said if it was conscious, it has nothing to do with being organic or not.
__patchbit__ | root | parent
Consciousness feels like the Euler Identity on a 17 degree helical climb off the plane. Looking top down, a complete circle is a closure. Looking along the plane, the ends don't meet and leave a residual. Whether or not to close that gap is a degree of freedom for the consciousness to decide among what it pursues for survival or pleasure.
gizajob | root | parent
You sound like you’d enjoy “I am a strange loop” by Hofstadter.
the_af | root | parent | next
> Is using steel under heavy stress in bridges a form slavery of an unconscious object?

There's no company (or anyone, really) claiming steel is conscious or that they are close to making it conscious.

wisty | root | parent
Most people are fine with slavery as long as it's not "one of us" (which to most people means humans).

Vegans might object that we should broaden our definition for what counts as "one of us".

"Pro life" people also have a broader definition.

Go back 500 years and "one of us" was proba ly a lot more narrowly defined for many people.

Are you arguing that all conciousnesses are "one of us", or that we logically should see it that way, or that it would ve good to see it that way, or ....