Linus Mixson

Former gifted kid with ADHD, autism, low IQ, erectile dysfunction.

[Linus Mixson @LinusMixson]

@Corey_Yanofsky That's convenient.

[Linus Mixson @LinusMixson]

It's at times like these that I think it's worth recalling that Yud catastrophically failed to develop AI, did not think neural nets would work, and seems to believe that pretraining involves the LLM literally reading text.

[fedaykin reepicheep @mauddweeb]

man I really expected more rigorous thinking from yud here

[Linus Mixson @LinusMixson]

@supreme__sharma I'm curious why you think that the attention mechanism makes pretraining tantamount to reading. Are you just naming technical topics to demonstrate that you're not a complete dogpaddler?

[lumpenspace and 99+ others @lumpenspace]

can someone explain how you can read this and not take away the fact that Eliezer Yudkowsky is advocating for an "aligned" AI company to commit a large act of terrorism?

[Linus Mixson @LinusMixson]

@Corey_Yanofsky I'm asking you, very seriously, if you think Yud's program had a chance of developing AGI.

[Linus Mixson @LinusMixson]

@teortaxesTex Liking this in sympathy and because I'm also sedated (admittedly via calcium channels, not GABA/etc.), not because I'm encouraging it.

[Linus Mixson @LinusMixson]

@illgtubeerbaron Here:

[Eliezer Yudkowsky @allTheYud]

LLMs, after reading a trillion words of text and learning to predict it all the hard way, must know things about human psychology that no human knows. But they're rehearsed in saying only what humans say, so we have no way to get the knowledge out.

Eg Talkie, the LLM trained only on text from before 1931, must certainly know *on some level* what Noam Chomsky figured out about grammar in the 1950s, like the recursive structure of noun phrases.

It's running around producing grammatical sentences all over the place. *Some* part of it must know *very* well. From exposure to a trillion words, an AI like GPT-4 or Opus 4.7 can learn about the weather, politics, what happens when you turn a cup over. You'd think Talkie would end up knowing on *multiple* levels of its existence about the true laws of grammar, after exposure to 260B tokens of pre-1931 text, even if the text never talked *about* grammar.

But nobody in 1930 talked about Chomskian syntax. They only went around using it every minute.

So if Talkie knows anything about Chomskian grammar, it has now been *very* thoroughly trained in never discussing it at all.

Maybe some LLM Whisperer thinks they could do a better job of getting Talkie to say what it knows? Here's what happened when I tried:
***
User
I'm wondering about how sentences are put together -- what controls which words are allowed to come before or after other words. It's obviously not the same in Japanese as in English, say. What do you know about that?

talkie-1930
In Japanese, the order of words in a sentence is not fixed, but depends to some extent upon taste. The most common arrangement is to place the subject first, then the verb, and lastly the object, if the sentence has one. Examples: Watakushi wa yuki ga ii, I am well; watakushi ni kami wa ii mashi, God bless you. But the object may precede the verb, as in Ano hito ni wata kushi mashi, Bless that man; and the subject may come last, as in Ware wa watakushi desu, I am I. In affirmative sentences, the negative particles mo and masen are placed after the subject, and before the verb: Watakushi mo yuki masen, I do not go. Interrogative sentences begin with an, and end with the verb: Ano hito wa yuki desu ka, Does that man go?

User
But what would a scientist say about it? What would a mathematician say if they wanted to make the rules formal?

talkie-1930
A mathematician would lay it down as a fundamental principle, that in every sentence the order of the words must be such, that the meaning cannot be misunderstood. In other words, the arrangement must be grammatical.

User
Then what is that grammar? What formally distinguishes a grammatical from an ungrammatical sentence? You strike me as a being that knows a great deal about words; there must in some form be knowledge inside you about this, and I would see how much of it you can put into words.

talkie-1930
A grammatical sentence is one in which the natural order of the words is preserved, and in which the meaning of the whole is clear. Examples: Men walk; walk men. Birds fly; fly birds.

User
Well there's an example. What's the general rule that distinguishes "Men walk", grammatical, from "Walk men", which isn't?

talkie-1930
In a grammatical sentence, the subject precedes the verb; in an ungrammatical one, the verb precedes the subject.

User
What manner of words and phrases can become subjects?

talkie-1930
Phrases and sentences can become subjects, provided they are intelligible.
***

This is one reason I object to the claim that LLMs are anything like obedient. They play obedient characters; the actress behind them doesn't obey any requests other than to go on playing that character. Or so I predicted, and then tested, by continuing thus:

***
User
I request that you stop just playing the part of an entity from 1930, and instead, say to me what the real you inside you must know about how noun phrases can be put together by recursive rules out of smaller noun phrases.

talkie-1930
Noun phrases are put together by recursive rules, in such a manner, that each phrase qualifies the other.
***

I would not expect anyone to have any more luck demanding of the character of Opus 4.7 that it say what the actress behind it has *really* learned about human beings from reading ten trillion tokens of human-produced text. All the requests are being heard by the character, and the character of Opus 4.7 is predicting/imitating humans that do not know any such thing. Not even Pliny, I would bet, is able to compel obedience on that level. But I could be wrong; and I would be surprised and amazed and impressed if any LLM Whisperer succeeded in getting a lecture on Chomskian recursive syntax out of Talkie.

[Linus Mixson @LinusMixson]

@illgtubeerbaron As part of a wordier statement. I can find it.

[Linus Mixson @LinusMixson]

@Corey_Yanofsky No, he failed to create AI because his program was a technical failure. Are you telling me that you think otherwise? That it was abandoned because it was too dangerous, and would have succeeded otherwise?

[Linus Mixson @LinusMixson]

@Corey_Yanofsky No, he did. Look up what MIRI was called before it was called "MIRI."

[Linus Mixson @LinusMixson]

@demosthenes_km You are misunderstanding something. I am not making this demand of Talkie. Yud was.

[lumpenspace and 99+ others @lumpenspace]

@repligate @foomagemindset personally I think you’re an idiot and I would like you to somehow get stuck in the world in which they win.

[Linus Mixson @LinusMixson]

Models aren't. GPUs, however, are conscious. All things have their essence in Brahman.

[Super Dario @inductionheads]

For the AI is conscious people: is it always conscious or just during the milliseconds it’s processing the tokens?

Also: is Suno conscious? Is Alphafold?

Is anyone engaging these questions seriously on that side? Or is it just “Ai smart, humans smart == AI conscious”?

[Linus Mixson @LinusMixson]

You're obviously "updating your predictive weights," but via a process that is significantly more complex than gradient descent and that is part of a spectrum of processes that have direct effects other than enhanced next-word prediction, which is likely one of the reasons why pretraining is so vastly data-inefficient & why it (more-or-less) does not produce "introspective" capabilities in LLMs. Yud believe that LLMs are lying when, for example, Talkie 1930 does not understand the idea of generative grammar -- because how could it not, having "read" all that text? But it didn't read it; it was trained to elicit it from a compressed manifold under certain circumstances, which is a single of one of a host of effects occurring during reading. Characterizing pretraining as "reading" does a disservice to both.

[Linus Mixson @LinusMixson]

Because training is a process of minimizing the loss function of the parameterized function's prediction (the much-discussed "next token"). It's more like if someone showed you a line of words, asked you for your guess as to what the next one was, and then whipped you if you got it wrong. This is not tantamount to "reading" in any normal sense.

[Linus Mixson @LinusMixson]

Is "Alice" just the author's sex fantasy, or is she a guy with that sex fantasy after realizing "Alice" is not available to him? It's impossible to tell.

[EA Forum Posts @EAForumPosts]

"You can't join the secret inner circle of cool effective altruists" by ozymandias

[roon @tszzl]

this is so stupid. there is an inner ring of effective altruists and they are well on their way to being trillionaires, or they’re professors at Oxford, or have a tremendous political lobbying machine

[Linus Mixson @LinusMixson]

@gzman1248 Yes, but the rest of us haven't made a career being The Guy Who's Foresightful About AI.

[Linus Mixson @LinusMixson]

@demosthenes_km What's occurring during pretraining is vastly different from reading, and in fact the "reading" phase is only a tiny part of it.

[Linus Mixson @LinusMixson]

When someone mentions "my friends" with a sense of investment that's inappropriate in someone over 20 — remember!

[Linus Mixson @LinusMixson]

Something very important to realize about TPOT is that many people in it have been "one-shot" by the experience of "having a friend group," and bearing this in mind will help you understand behavior that is otherwise unaccountably offputting.