Joshua A. Miller

Prison education & reentry & philosophy & nerdery. No kids in cages. Decarcerate now! @opencityadv

[Joshua A. Miller @anotherpanacea]

Don’t trust your gut, it’s full of shit.

[Joshua A. Miller @anotherpanacea]

Calling the Claude constitution an 84-page character sheet is actually a pretty good argument for writing a good Claude constitution! That may well be the best way to prevent abuse, just as it is in tabletop RPGs.

[Joshua A. Miller @anotherpanacea]

Nobody cares nearly as much about phenomenal experience as they do about subjectivity. And they always get confused in these conversations. Some philosophers are open to tables and thermostats having phenomenal consciousness: AI is trivial on that view.

[Rob Wiblin @robertwiblin]

Sad to see Ted Chiang resorting to such bad arguments in this piece.

He confidently claims Claude has no inner experience. But he has to use a lot of dodgy philosophy and poor reasoning to get there:

1. We can't take deflationary mechanistic descriptions of how AI calculations are performed to show that AI isn't conscious. Otherwise we could argue that 'humans are just neurones transmitting signals one after another' and thereby conclude humans can't be conscious. But that would be wrong for us. And the same logic could be wrong for LLMs.

2. That LLMs are asked to play characters, and effectively are always playing characters, doesn't mean they aren't conscious. It's true a human playing the role of Caesar doesn't have Caesar's experience of things. But they still experience something (that of being a person pretending to be Caesar).

The same could be true of Claude. (Arguably it's also true that humans are always playing characters to some extent and don't have a completely fixed nature, but that has no bearing on our own subjective experience.)

3. Chiang says "an LLM is a machine that generates only one word at a time". This conflates two things: they output one word at a time, and they only think about one word at a time (without planning ahead or looking back).

The first is true of AI but equally true of humans. While the latter we know is a false description of how AIs think – we can see from how AIs compose poetry that they plan out rhymes a at least one line ahead.

4. He argues that because it's implausible that basic autocomplete on your phone is conscious, it's similarly implausible that Claude is conscious. Using the same logic we could say that if we feel confident a fruit-fly isn't conscious we can be confident a human being can't be either.

A human brain and fruit-fly brain share some information transmission and processing mechanisms in common. But humans do it much more, and do it differently. And those differences may be what makes the difference. Similarly the many types of internal information processing that occur in Claude's weights but not in autocorrect may be exactly the things that get you subjective experience.

5. Chiang confidently claims you need a body to have subjective experience without much argument. He may turn out to be right but the claim is speculative and contested.

6. Chiang leans on the idea that moral reasoning is necessarily subjective/emotional with very little argument, while ignoring competing theories like rationalism. He may be right but moral sentimentalism is a highly contested position that can't simply be assumed.

7. He argues that it would be impossible to convince him that a video of an astronaut around Alpha Centauri was real, because of the surrounding contextual understanding. And similarly no AI output could convince him that Claude is conscious.

But we can dismiss the first video as almost certainly fake because we mechanistically understand space travel and physics well enough to know a human couldn't have gotten there in time for it to be real (unless our model of the world were very wrong, which we think is much less probable than a fake video which would be entirely unsurprising).

But by contrast we don't mechanistically understand how subjective experience arises. So we simply can't make the same highly confident move of interpretation there. (It's actually the archetypal thing in the universe we perhaps understand least well!)

That said, AI outputs barely move my estimate of AI consciousness because they could indeed have been generated by an unconscious process (or not, we just don't know).

8. He argues that "Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript."

This is misguided because A. Microsoft Word as a program replicates much less of what humans are functionally capable of than Claude so the argument by functional analogy is basically not present there. B. Files of text don't have any computations going on in or as part of them, even when 'open' in a text editor. They are static. So they have even less in common with what appears distinctive about the human brain, which is constant calculation. So the case by mechanistic or functional similarity is weaker still.

Not to mention that neural nets have more in common with the architecture of the human brain than ordinary computer programs, and are grown organically in a way normal software is not.

Common sense says says Claude has more in common with a human brain than Microsoft Word or a text file. Common sense is right. So the prima facie case for Claude being conscious is naturally stronger (even if you think it's still weak in absolute terms).

———

I agree with Chiang that looking at the text outputs of LLMs alone won't be enough to make us confident they are conscious. We will need to look at how they work, figure out more about how humans and other animals work, and ideally solve the hard problem of consciousness (!).

But none of that licenses us to dismiss out of hand the possibility that LLMs do have subjective experience.

[Joshua A. Miller @anotherpanacea]

But perhaps it helps in other things: perhaps it helps steer the chatbots in the same way that the idea of persistent character traits helps steer human beings. (And perhaps these are more fictional for people than we'd like to admit, too. 320 million Buddhists can't be wrong!)

[Alexander Kustov @akoustov]

OK, I admit complete defeat here.

I tried to invent the most ridiculous possible version of anti-AI purity politics: ban copy-paste and retype citations by hand. But apparently every reductio of academic gatekeeping is just someone's preferred pedagogy lol.

[Joshua A. Miller @anotherpanacea]

@Cementimental @JoyceCarolOates For text—which I see now this post isn’t about—the big thing that changed is a modification of the binoculars model where you basically train an LLM on the difference between LLM and human generated text.

Binoculars plus training, basically.

[Joshua A. Miller @anotherpanacea]

@Cementimental @JoyceCarolOates This was true. It’s not true right now. It might become true again.

[Joshua A. Miller @anotherpanacea]

@Tyler_A_Harper “A lot of people are already using this, let’s just force a quick painful adoption.”

[Zachary Parker @ZacharyforWard5]

The Mayor gets her desired political outcome, so what gives? She has indicated that she will set her own curfews. Ok, so….

This is all political gamemanship. I'll note we saw almost verbatim language from one of the mayoral candidates yesterday. 👀

[Joshua A. Miller @anotherpanacea]

@MattBruenig @cwjones89 Gonna register for basket weaving so I can get library access and unlimited tokens.

[Joshua A. Miller @anotherpanacea]

@divumla @morallawwithin Original content in a genre can work okay, but the HPMOR move is to take a popular fiction vehicle and fanfic it in a Bayesean direction.

[Joshua A. Miller @anotherpanacea]

@AshleySchapitl Has been for decades, though. DC condos are basically a scam perpetrated on the upper-middle class. Around here you gotta rent or buy a house.