Ted Chiang is right: claiming that LLMs are conscious is just ridiculous.
One simple example. If you ask GPT to imitate a conversation between Julius Caesar and Genghis Khan, GPT will do it very well.
It will talk about wars, betrayal, and power. Il will descrive the feeling of being cheated by your brother with unbelievably realistic and moving words.
Does this mean that GPT contains a self-conscious copy of Julius Caesar or Genghis Khan? Of course not.
Similarly, if GPT makes claims about itself, does this mean it is self-conscious? Of course not.
An LLM is just simulating language, feeling, and consciousness.
True, we don’t have an accepted definition of consciousness. But, at a minimum, to be conscious, an entity must have something at stake.
It must risk dying and have emotions that move it away from danger and towards favorable states. It must have a driver.
This is also why I share Chiang’s worry about moral atrophy.
The more we offload moral decisions to LLMs, the more we risk losing our own capacity for moral reasoning.
Human moral reasoning descends from our history of making harmful actions, suffering harmful actions, regretting them, fearing them, repairing them, and learning from them.
LLMs do not experience harm, do not suffer, do not fear consequences, do not regret.
So they cannot do moral reasoning.
We are offloading moral reasoning to systems that cannot do moral reasoning.
What can go wrong?
*
Full piece in the first reply
@teachrobotslove This was really beautiful. Thank you for posting. Surrendering to love is a core part of what the whole thing is about and I think you expressed that very well.
Also: congratulations to you and your husband. Mazel tov!
@seanjagermann@eatboldbar I know, but my point is that cleanliness of label alone is probably not as big of a differentiator as you might be thinking. Maybe! But " the healthy one that also has caffeine" is an easier hook than "one of the ones that is very healthy, also no tallow"
@soncharm Math Understanders, I think, on the contrary would prefer no memorization at all. The memorization helps get right answers faster, but in a way that's just shortcuts - in principle, every part implies all other parts.