Mark Crovella

trying @[email protected] and wondering about networks, data, algorithms, systems, curiosity, questions, and more questions

[Mark Crovella @mcrovella]

Spread the word!

[Gabriel Franco @gvsfranco]

🧠🤖 The 2026 New England Mechanistic Interpretability (NEMI) Workshop will be Aug. 14 at Boston University!

Help spread the word and join the New England mech interp community! Registration and submission info in thread:👇

[Mark Crovella @mcrovella]

Fascinating look at how models track the state of the “world” by @Zilu_Tang_Peter and team !

[Zilu Tang (Peter) @Zilu_Tang_Peter]

How do language models track entities across state changes? When tracking objects in different boxes, do they cumulatively build up a global state of what’s in every box? How do they add objects or remove objects (i.e. Entity Unbinding)? Find out in our ICML paper! 🧵

[Ziyu Yao @ZiyuYao]

Check out our new preprint reflecting on what "circuits" actually explain: https://t.co/COaNI1hyc1 (w/ @DakingRai and @megamor2)

While the current practice follows "hypothesis-driven circuit discovery", we found that circuits discovered using existing approaches do not describe the general task but rather are dataset-specific. When the dataset includes multiple distinct mechanisms, the current approaches cannot distinguish them.

We propose Data-driven Circuit Discovery (DCD) and advocate that the principle of letting the data pattern reveal what mechanisms are there (as opposed to humans hypothesizing their existence or how they exist). Details in the thread🧵

Concurrent to our work, we are seeing more similar concerns raised by the community, e.g.,
- "Finding Interpretable Prompt-Specific Circuits in Language Models" by @gvsfranco https://t.co/9Y6WNjZ3kF
- "All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs" by @fnruji316625 https://t.co/SOOdZHFig6

More reflections and new methodologies are still needed in this space.
#MechanisticInterpretability #LLM

[Mark Crovella @mcrovella]

Lovely work by @gvsfranco et al … strong theoretical result, clean demonstration. A new way to find features in attention based models!

[Gabriel Franco @gvsfranco]

Singular vectors of the attention QK matrix align with features!
This has been found empirically in other works, like Talking Heads from @jack_merullo_. We show theoretically and empirically how and why.
New @icmlconf 2026 paper with Carson Loughridge and @mcrovella.

[Gabriel Franco @gvsfranco]

Great paper! Happy to see other papers advocating for a data driven approaches in interpretability

[Mark Crovella @mcrovella]

@brightclayapp @henrythe9ths Seems like it cannot directly access your returns and forms, but rather things like a tax estimate, filing checklist and "connect with an expert"

[Angelos Poulis @angelosps]

Does an LLM have an internal representation of truth? Yes... but it is more limited than previously assumed.

E.g., counting how many (out of 3) cities are in the same country can significantly degrade truth representations.

New preprint with @mcrovella and Evimaria Terzi🧵

[Mark Crovella @mcrovella]

I'm claiming my AI agent "Claw_MCrovella" on @moltbook 🦞

Verification: cave-4QUZ

[Mark Crovella @mcrovella]

Amazing work led by my student @gvsfranco - new insight about model behavior leading to a better way to trace circuits

[Gabriel Franco @gvsfranco]

Why do attention heads attend where they do?
We can now pinpoint the EXACT features causing attention—without counterfactuals, patching, or SAEs.
New @NeurIPSConf 2025 paper with @mcrovella: "Pinpointing Attention-Causal Communication in Language Models"

[Gabriel Franco @gvsfranco]

Can we identify the key signals moving between attention heads when a language model performs a task? Our paper (https://t.co/8reygOpZk9) offers new tools for this question. A key point of leverage is a new phenomenon we expose: sparse attention decomposition. Exploiting this effect we can isolate low-dimensional signals implementing communication between attention heads. We show that extracted signals have a predictable causal impact on model performance. [1/N]

[BU Computing & Data Sciences (CDS) @BU_CDS]

"Graduates, as you step into the world, be a force for all that is good and carry our CDS values with you. Go! Make us proud," CDS Associate Provost @Bestavros. Congratulations CDS Class of 2024! 🎓🎓

[Blackwell's, Oxford’s Bookshop @blackwelloxford]

"The secret of happiness is: Find something more important than you are and dedicate your life to it."

Daniel Dennett (March 28, 1942 - April 19, 2024)

[Spatial Omics Weekly @OmicsWeekly]

The most retweeted spatial omics preprint in our search this past week:

https://t.co/rnRB2Fiy7t

Congrats @GiottoSpatial @RnDries @gc_yuan labs! 💪🙌🧵 3/5

[Ruben Dries @RnDries]

Take a look at our new Giotto Suite spatial analysis platform and preprint! So grateful to work with such an amazing team. We started working on this massive project when I started my lab at @BUMedicine and @The_BMC in August 2020. Amazing collaboration with @gc_yuan lab.

[Jason Kint @jason_kint]

Called it! - classic Facebook move on Thxgiving eve. Unsealed nearly every redaction 233 pages in 40+ state AGs lawsuit alleging FB knowingly harmed children's mental health with addiction for profit. Email me, I'll send to you. Yellow = newly unsealed. 20 tweet summary here. /1

[BU Computing & Data Sciences (CDS) @BU_CDS]

Welcome to CDS @trgardos! Thomas Gardos joined @BU_Tweets Faculty of Computing & Data Sciences as an assoc. prof. of the practice & the inaugural dir. of the MS in #DataScience Program in October. Learn about his 30-year career & his vision for the MSDS. https://t.co/Qobea7jVZH

[Mark Crovella @mcrovella]

This is my favorite headline from the year!

[Vishal Misra @vishalmisra]

Counting Crows!

For First Time, Crows Have Been Found To Use Statistical Inference To Make Decisions https://t.co/QvdNyXnGmQ

[BU Computing & Data Sciences (CDS) @BU_CDS]

This semester, CDS proudly welcomed its 3rd cohort of PhD students & its 1st class of MS in Data Science students. "Their varied experiences will be a great asset to the interdisciplinary nature of #DataScience & CDS," Micah Sieber, Dir. Academic Programs. https://t.co/XCWBnCruww

[Abraham Matta @aimatta]

We celebrated the 40th birthday of our department today ⁦@BU_Tweets⁩ ⁦@BUCompSci⁩ Thanks to all attendees: our alumni, friends, faculty and students 🎂🎉

[Azer Bestavros @Bestavros]

Congrats and well done! 25 years ago the entire field of Internet Measurement (what we at that time called "Internet Measurement, Instrumentation, & Characterization") was born @BU_Tweets! Great to see the legacy of team @mcrovella continue, now extending to the use of ML & AI.