Fatih⏩⤴️

Lover of all novel and hard concepts: especially machine learning and systems theory

[Fatih⏩⤴️ @taskinfatih]

@aimalysheva @BetaTomorrow

[Fatih⏩⤴️ @taskinfatih]

@che_shr_cat I mean combination of Text to Lora from @SakanaAILabs and Tiny Lora from @jxmnop ?

[Fatih⏩⤴️ @taskinfatih]

@che_shr_cat Isn’t this Tiny Lora, even QR code?

[Fatih⏩⤴️ @taskinfatih]

@oussamazekri_ @jdeschena Is it bidirectional?

[Fatih⏩⤴️ @taskinfatih]

@HunterBiden Totally agreed Hunter. Here is a video guilt, shame and more importantly grace.

[Fatih⏩⤴️ @taskinfatih]

@cutesuscat Reminded me this

[Elan Barenholtz @ebarenholtz]

New Preprint! https://t.co/qbiPEwZAIh

Question: Are humans doing next-token generation like LLMs?

One important source of behavioral evidence comes from surprisal: when LLMs are used to estimate how unexpected each word is in context, those surprisal values turn out to predict human reading times remarkably well. This convergence is consistent with the idea that humans, like LLMs, are running something like next-token prediction.

But surprisal only captures the final probability distribution the model outputs. It tells us what the model concludes about the next word, not the rich sequential computation that produced that conclusion. If we want to test whether humans and LLMs share something deeper than convergent output statistics, we need a measure of what the model is doing inside.

In this paper, I introduce trajectory extrapolation error: a measure that captures a model's internal representational geometry as it processes each word. Rather than asking what the model predicts, it asks how the model's internal state is moving, and how much each new word disrupts the trajectory it had established.

I found that this measure independently predicts human reading times beyond surprisal, across multiple datasets and model architectures.

Why this matters: trajectory extrapolation error gives us a window into the model's actual sequential processing, not just its output. The fact that this internal geometry tracks human reading behavior is much stronger evidence for human/LLM correspondence than surprisal alone could provide.

More work is upcoming that shows trajectory extrapolation is a better predictor of brain activity during langauge processing too.

[Fatih⏩⤴️ @taskinfatih]

@jxnlco Github?

[Fatih⏩⤴️ @taskinfatih]

@Yomatomoko Reminded me this

[Fatih⏩⤴️ @taskinfatih]

@vividvoid Muslims believe we have God’s breath in us.

[Fatih⏩⤴️ @taskinfatih]

@hthieblot Stumbleupon

[Fatih⏩⤴️ @taskinfatih]

@NousResearch Thank God. I love terminals but it is a torture to run a long conversation in them.

[Fatih⏩⤴️ @taskinfatih]

@aaaronson 10=On (Turkish)

[Fatih⏩⤴️ @taskinfatih]

@creatine_cycle Input token or output token?😅

[Fatih⏩⤴️ @taskinfatih]

@CSProfKGD @ylecun here is the link of the presentation:
https://t.co/Sy3cBS7lfW

[Fatih⏩⤴️ @taskinfatih]

@CSProfKGD @ylecun Is there a recording of this?

[Pope Leo XIV @Pontifex]

Artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships, and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate or even simulate, but they do not understand what they produce, for they lack the affective, relational, and spiritual perspective through which human beings grow in wisdom. #MagnificaHumanitas

[Fatih⏩⤴️ @taskinfatih]

@toddsaunders It shares more of its chain of thoughts and initial findings, plans etc.

[Fatih⏩⤴️ @taskinfatih]

@felixleezd Just you

[Wholesome Side of 𝕏 @itsme_urstruly]

Never forget that YOU are a data center. Drink some water!!