Jonathan Paulson

Software Engineer. Interests: quant finance, competitive programming, Effective Altruism, policy.

[Jonathan Paulson @PaulsonJonathan]

@nick_kapur I find the opening argument puzzling: Chiang carefully distinguishes between an author and a character, argues that characters are not conscious, and then never addresses whether the *author* might be conscious.

[Jonathan Paulson @PaulsonJonathan]

@AaronBergman18 Yeah. Also: assuming a nonzero interest rate for infinite time sounds normal but is actually pretty crazy.

[Jonathan Paulson @PaulsonJonathan]

@AaronBergman18 If he has one penny once and then lives off his wages for a long time and never touches his savings, it will eventually grow to an arbitrary fortune (assuming nonzero interest)

[Jonathan Paulson @PaulsonJonathan]

@Strife212 If you could run a human by hand would humans prove humans aren’t conscious? Doesn’t seem impossible in principle.

[Jonathan Paulson @PaulsonJonathan]

@richardartoul @BhavyanshSabha1 @reed_barnes What other internal metrics would you expect them to be able to point to?

They are shipping a lot of code, it feels productive to them, and people are paying them a lot for it. What else is there?

[Jonathan Paulson @PaulsonJonathan]

@QiaochuYuan What if we care about human beings for reasons other than generating text on the internet?

[Jonathan Paulson @PaulsonJonathan]

@BenDiFrancesco Doesn’t seem crazy that training LLMs to be like humans would make them more conscious than training them to do other things.

[Jonathan Paulson @PaulsonJonathan]

@x88890059 @joseph_h_garvin I’m good at it :)

What’s the argument here? A software engineer should know that it’s just multiplication and multiplication isn’t conscious? But everything is made up of little pieces, including humans.

[Jonathan Paulson @PaulsonJonathan]

@joseph_h_garvin Exponentially bigger. So big that you couldn’t physically instantiate it! I think that’s telling.

I’m not saying “big implies conscious”. But it allows for the possibility.

[Jonathan Paulson @PaulsonJonathan]

@joseph_h_garvin No.

If you look at how the language is produced in these three cases you’ll see that LLMs have a lot more room for something like consciousness to be going on; gigabytes of numbers that are not well understood.

Also LLM outputs are more impressive.

[Jonathan Paulson @PaulsonJonathan]

@joseph_h_garvin They are; I don’t think talking is a requirement for consciousness. But it seems like decent evidence for consciousness.

[Jonathan Paulson @PaulsonJonathan]

@littmath @jpburelle @SidGadgil I’m surprised to read this; mathematical text is much more scalable than personal understanding. You don’t think the scale advantage (many people can read a paper or textbook) makes the text more valuable?

[Jonathan Paulson @PaulsonJonathan]

@thdxr A for loop is almost always more readable

[Jonathan Paulson @PaulsonJonathan]

@patricklaske @tracewoodgrains All the examples in the OP are money failing to buy primaries.

[Jonathan Paulson @PaulsonJonathan]

@philosophymeme0 We’ve enjoyed huge growth compared to 500 years ago. Who’s to say we can’t have similar gains in the next 500 years?

There are fundamental limits but we are far away from them.

[Jonathan Paulson @PaulsonJonathan]

@SupahKaz @mattyglesias “Gains” are new income. The principal isn’t being taxed further.

[Jonathan Paulson @PaulsonJonathan]

@VictorTaelin Is “compile dynamically and run” faster than the interpreter?