My latest substack post describes detailed empirical results for RSA vs ridge regression across a wide range of visual system tests.
I think we're genuinely finally building a better sense of how all these things are related.
Substack: https://t.co/WnPOm6E6YT
We're testing the consequences of RSA contravariance theory: https://t.co/ziCLDd2Thg
TL;DR:
- RSA and ridge regression are actually pretty well correlated, especially in high-quality fMRI data.
- The theoretically predicted Ridge-RSA formula works decently!
- BUT: strong behavioral tests expose issues with raw RSA. Ridge model-brain similarity is uniformly more predictive of behavioral match across MANY metrics.
- And the same is true for low-level visual phenomena: Ridge-based model-brain similarity better predicts 20+ V1 physiology metrics.
enjoy!
The Contravariance theory for RSA (and CKA) is here.
Substack: https://t.co/ziCLDd2Thg
Gory mathematical details: https://t.co/jsMIis9y2B
TL;DR:
- Theiss @ScienceLukas@ermgrant@SaxeLab show RSA decomposes into a "task relevant core geometry" part and a "task irrelevant" nuisance component. The nuisances can dominate the RSA badly.
- The nuisances can be removed by projecting onto the privileged axes. Once you do that, full contravariance applies to the task-relevant core geometry RSA.
- In the hard-task condition, the three metrics (privileged axes, linear regression, and task-relevant RSA) all line up.
Super excited to be back @CogCompNeuro
in NYU - this year presenting new exciting work from Project Simsom with Cameron Ellis, @dyamins and the team. Are we ready to build an AI model of infant development? We think so ... come check it out Thursday 10:30 E39 👶
I am presenting my current research investigating how perception and mental imagery might be implemented in a single visual architecture at #CCN2026 this morning at 9:30 in Poster session A. Come say hi at Poster A96. The Stanford NeuroAILab has a lot of exciting work here, so check it out below!
Will be presenting our work on building a Universal Vision-Language World Model.
We build an 8B autoregressive transformer that unifies World Models and Vision-Language Models using visual abstractions as streams of thought, e.g., camera pose, depth, optical flow, point tracks, text -- generated and read like tokens. Our model solves vision-language tasks using world modeling (novel view synthesis) and inverse dynamics (camera pose estimation), unlike standard VLMs. We optimize end-to-end with a continuous rollout using RL, entering a cycle of recursive self-improvement.
I'm presenting my first-year work, "Structured Early Activity Builds the Visual System's Initial Organization Before Naturalistic Experience," Wednesday, 9:30–11:45, at the Kimmel Center, Rosenthal Room.
Huge thanks to Dan (@dyamins) and Kalanit (@kalatwt) for advising this work!
@CogCompNeuro Satellite Event:
COMPUTATIONAL CONSCIOUSNESS SCIENCE
Saturday, 1 August 2026
New York University
Bringing computational modelling, neuroscience, and philosophy together to understand consciousness
NYU Center for Mind, Brain and Consciousness
#CCN2026
Machine understanding
Feature Review by Huili Chen, Stephen R. Grimm, Olga Russakovsky (@orussakovsky), & Tania Lombrozo (@TaniaLombrozo)
https://t.co/1l6KnKwBxP
Ok -- I've started a substack -- The Principle Investigator! My goal is communicating advanced takes on ideas in NeuroAI+. Here's the first substantive post: https://t.co/V6yQdx8AGx Calculating the aesthetics of DNN models of the brain
Embryonic retinal waves! In new study led by superstar student @CVoufo, @AndyQuaen and Ben Smith, a team led by @atiriac1, we describe waves at early stages of development – long before proper neural circuits are present in the retina. https://t.co/3K0MsoB3re
A great application of the PSI concept to 3D scene understanding and physical word modeling intelligence! It's pretty amazing what you can get from "just pixels"...
Excited to present our ICLR 2026 paper tomorrow: Unified 3D Scene Understanding Through Physical World Modeling.
Joint work with @KlemenKotar@Rahul_Venkatesh@jwhooglee@honglin_c@khai_loong_aw@dyamins.
We introduce 3WM, a foundation model for 3D understanding that treats depth, novel view synthesis, object motion, and geometric reasoning as different prompts to the same physical world model.
If you are at ICLR, come by our poster: Poster Session 6, Pavilion 4, Sat 3:15 PM.
Whither symbols in the era of advanced neural networks?
Opinion by Thomas L. Griffiths, Brenden M. Lake (@LakeBrenden), R. Thomas McCoy, Ellie Pavlick, & Taylor W. Webb
Free access before June 5: https://t.co/qV85r7uoof
Categorization is ‘baked’ into the brain — a Perspective by Lisa Feldman Barrett & Earl K. Miller
@LFeldmanBarrett@MillerLabMIT
https://t.co/gDGnwN2Iom
1/7 How can we develop decoding methods that generalize to novel subjects without any fine-tuning?
Excited to share our recent collaboration to appear at CVPR 2026!
Work led by Andrew Luo's group, with many collaborators (@_jacobprince_ , Mike Tarr, @KriegeskorteLab and others).
code: https://t.co/5o43p1iQpt
paper: https://t.co/hYGH5cSuWw
I think we finally made really significant progress on the biggest unsolved "developmental AI" problem: learning from human-scale data. Key idea: zero-shot world models that support concept extraction via approximate causal inference. amazing collab w/ @khai_loong_aw@mcxfrank
Today's best AI needs orders of magnitude more data than a human child to achieve visual competence.
We introduce the Zero-shot World Model (ZWM), an approach that substantially narrows this gap. Even when trained on the first-person experience of a single child, BabyZWM matches state-of-the-art models on diverse visual-cognitive tasks – with no task-specific training, i.e., zero-shot. 🧵
Hot of the new home-built macroscope. 3 minutes of retinal waves imaged over entire retina. (5x speed). Scope designed and built by incomparable Ben Smith, biology provided by @atiriac1 and @MiahPitcher. So much more to come!
2/ Real brains can predict each other. If your model is a good model, it should fit right into that population — mappable to and from real brains, using the same transforms real brains need. We call this the Principle of the Inter-Animal Transform Class.
Progression from texture to shape along the ventral pathway!
Neuronal tuning aligns dynamically with object and texture manifolds across the visual hierarchy
https://t.co/DDw99wc9sp