Francisco Portillo

Linguistics @unav || 🧠 & 🤖 || Researching at @bcbl_ • @Harvard • @MPI_NL

[Francisco Portillo @fportillolo]

Keep working!!! 💪🚀

[SpeechLabUNAV @speechlabunav]

Meet the SpeechLab Team!! 🚀 (26-27)

🧵

[Michael C. Frank @mcxfrank]

Wordbank, my lab's open database of children's vocabulary development, has a new home. There's a new website, a new data release, and wordbankr is back on CRAN. A few details 🧵 https://t.co/alwjO7sR8M

[Francisco Portillo @fportillolo]

I find it funny how we talk about AI sometimes.

We call AI "intelligent" when it performs well, but suddenly claim it doesn't really understand anything when it makes a mistake.

Are we using a consistent definition of intelligence?

Or are we moving the goalposts?

[Francisco Portillo @fportillolo]

@devarda_a @statnews @ev_fedorenko @roger_p_levy Congrats Andrea!! 💪💪

[Francisco Portillo @fportillolo]

@nrol_ling Very cool Nathan! Thanks for sharing, I will play with it ;)

[Francisco Portillo @fportillolo]

Moral: "is this LLM cognitively plausible" is the wrong question. Plausible for what? Mine predicts adult reading great, tells you nothing about how kids get there. Cognitive plausibility isn't one number.

[Francisco Portillo @fportillolo]

Turns out a kid learning "dog" and an adult reading a sentence with "dog" in it care about completely different things. New preprint on why LLM surprisal is a great model of one and useless for the other
🧵

https://t.co/gfrPTvL8Kv

[Francisco Portillo @fportillolo]

Ran the numbers to make sure this wasn't just "one test was significant, one wasn't". The two effects are formally, statistically different (z = 3.63, p < .001). Not a fluke, not underpowered. A real split.

[Francisco Portillo @fportillolo]

The main pattern in one figure: frequency was strongly related to when words were acquired, while surprisal was much more clearly related to how adults processed words in context.

[Francisco Portillo @fportillolo]

A word can be surprising without being important for learning.

At least, that’s what I found when I compared how surprisal relates to learning words vs. processing words we already know.

I didn’t quite expect the two to look this different.
🧵

[Francisco Portillo @fportillolo]

Are we using the same statistical information to learn a word and to process it once we know it?

Maybe cumulative exposure matters more for building the lexicon, while contextual prediction becomes more visible once that lexicon is already established.

What do you think?

[Francisco Portillo @fportillolo]

@KarolinaADrozdz Congrats!

[Francisco Portillo @fportillolo]

@RTomMcCoy Really liked the “former you” idea. It’s easy to forget how much you’ve learned along the way.

[Francisco Portillo @fportillolo]

The more I read about surprisal and entropy, the less obvious the answer seems.

[Francisco Portillo @fportillolo]

If two language models assign the same probability to the word that actually appears, but have very different probability distributions over the words that could have appeared, are they making the same prediction?

Also: would humans process those two situations in the same way?