Andy J Yang

[Satwik Bhattamishra @satwik1729]

Given black-box access to a Transformer's output, can we efficiently recover its parameters?

We analyse the learnability of attention-based models with query access in our new work. Accepted at #ICML2026 🎉

Work done with @shahkulin98, @mhahn29 and Varun Kanade.

🧵

[Gašper Beguš @begusgasper]

Can syntax begin to emerge from speech alone?

In our new paper, we show that unsupervised neural networks trained only on individual spoken words can spontaneously generate two- and even three-word sequences — without ever seeing multi-word examples during training.

We call this phenomenon spontaneous concatenation. It can help model an early step in both language acquisition and the evolution of syntax.

We also propose a possible neural mechanism behind this behavior: disinhibition, which offers a pathway from raw speech representations toward compositional structure.

Using AI interpretability techniques, we can begin to identify neural mechanisms behind operations that resemble basic symbolic processes such as concatenation — a precursor to operations like Merge.

[Tom McCoy @RTomMcCoy]

Now out in TiCS: "Whither symbols?"

Neural networks can now do many things long thought to require symbols. What does this mean for the role of symbols in CogSci? Read the paper for our answer!

[Aleksandra Bakalova @abakalova13175]

Will be presenting this at the Latent & Implicit Thinking Workshop (https://t.co/8cytLakurb) at #ICLR2026 Come by our poster! Always happy to chat :)

[breandan @breandan]

New work! Introduces a parallel RASP variant highly suited for SIMD architectures. I implement a VM bytecode and lower a heapless array language onto it, demonstrating significant speedups over serial evaluation on a massive multitenancy benchmark with millions of concurrent VMs.

[Pete Shaw @ptshaw2]

I will be presenting this paper at ICLR next week! 🇧🇷

Come chat about Kolmogorov complexity, the MDL principle, and what this all means for training better models! 🧵

[Michael Hahn @mhahn29]

We have 1-2 more extra spots due to new funding -- apply by end of April!

[Andy J Yang @pentagonalize]

Last day to apply for financial support!

Find the registration and financial support forms here: https://t.co/FRJlkLr4oH

[Andy J Yang @pentagonalize]

📣 FLaNN 2026 at Yale 🍮

Invited talks+posters (non-archival): expressivity, computation, and learning in neural nets/LLMs

Speakers: Pablo Barceló, David Chiang, Will Merrill, Naomi Saphra, Gail Weiss

Abstracts due Feb 12, 2026
Details: https://t.co/AzgF1TMyOS

[Satwik Bhattamishra @satwik1729]

Given access to a language model, can we extract an interpretable object like a DFA that captures which strings a language model is likely to generate?

Our new work on automata learning theory studies this question. To be presented at ##ICLR2026 🎉

[Yash Sarrof @yashYRS]

Most work on Transformer length generalization assumes a fixed vocabulary. But in real tasks, longer inputs may have new symbols (e.g. more objects in planning). Our new paper introduces C-RASP* to study this and explains the inconsistent performance of Transformers in planning.

[Andy J Yang @pentagonalize]

📍 New Haven, CT, USA
📅 May 11-13, 2026
🌐 https://t.co/AzgF1TMyOS
📩 [email protected]

[Andy J Yang @pentagonalize]

A reminder to register for the FLaNN workshop!

The financial support application is now open to all attendees, not limited to graduate students.

Find the registration and financial support forms here: https://t.co/FRJlkLr4oH

See you there!

[Andy J Yang @pentagonalize]

📣 FLaNN 2026 at Yale 🍮

Invited talks+posters (non-archival): expressivity, computation, and learning in neural nets/LLMs

Speakers: Pablo Barceló, David Chiang, Will Merrill, Naomi Saphra, Gail Weiss

Abstracts due Feb 12, 2026
Details: https://t.co/AzgF1TMyOS

[Aleksandra Bakalova @abakalova13175]

Can we rewrite Transformers as a human-readable code?

In this paper, we decompile Transformers trained on algorithmic and formal language tasks into D-RASP – a programming language that mirrors Transformer architecture. 🧵

[Andy J Yang @pentagonalize]

We also have (limited) financial support available on a need basis for graduate students who are not able to attend otherwise.

[Andy J Yang @pentagonalize]

The FLaNN Workshop submission deadline has been extended to Feb 19!

Invited talks + posters (non-archival): expressivity, computation, and learning in neural nets/LLMs. Previous work welcome. Graduate students encouraged to submit!

📍 Yale University
🗓️ May 11-13, 2026

[Andy J Yang @pentagonalize]

📣 FLaNN 2026 at Yale 🍮

Invited talks+posters (non-archival): expressivity, computation, and learning in neural nets/LLMs

Speakers: Pablo Barceló, David Chiang, Will Merrill, Naomi Saphra, Gail Weiss

Abstracts due Feb 12, 2026
Details: https://t.co/AzgF1TMyOS

[Andy J Yang @pentagonalize]

Call for Submissions: https://t.co/tSof6Ajvbp
Registration: https://t.co/FRJlkLrCef
Contact: [email protected]

[Andy J Yang @pentagonalize]

We welcome posters on the formal expressivity, computational properties, and learning behavior of neural nets (incl. LLMs). Graduate students are especially encouraged to submit!

Contact: [email protected]

[Andy J Yang @pentagonalize]

📣 FLaNN 2026 at Yale 🍮

Invited talks+posters (non-archival): expressivity, computation, and learning in neural nets/LLMs

Speakers: Pablo Barceló, David Chiang, Will Merrill, Naomi Saphra, Gail Weiss

Abstracts due Feb 12, 2026
Details: https://t.co/AzgF1TMyOS

[Andy J Yang @pentagonalize]

Deadline in just under two weeks!

[Andy J Yang @pentagonalize]

Inviting submissions to the first Workshop on Formal Languages and Neural Networks!

We welcome posters dicussing the formal expressivity, computational properties, and learning behavior of neural networks!

Call for posters: https://t.co/tSof6AiXlR
Deadline: February 12, 2026

[Andy J Yang @pentagonalize]

Announcing the first Workshop on Formal Languages and Neural Networks (FLaNN) 🍮!

We invite the submission of abstracts for posters that discuss the formal expressivity, computational properties, and learning behavior of neural network models, including large language models.