1/
Every week in my neurology clinic, I see people lost in the search for a diagnosis.
Sometimes, what’s causing their dementia symptoms is treatable — but was missed for years.
We just published a new AI tool to change that. Here's why it matters🧵
@NaipMayo@MayoClinic#ENDAlz
The hippocampus functions as a latent diffusion engine, compressing sensory, interoceptive, and cognitive inputs into a unified generative model.
https://t.co/yhDxRUaTUT
What do brain waves look like when you have clinical impairment in memory like Alzheimer’s or some other dementia syndrome? Check out our latest preprint: https://t.co/7TpIJEhXjh
The brain uses dimensionality reduction as a built-in feature of its wiring.
Network geometry shapes the transformation of task representations across the human cortex
https://t.co/8ztNt5yHSG
#neuroscience
The outpouring of interest in our paper has been … a bit insane.
May I make four points?
1. We have not shown that “consciousness has a frame rate.”
I can see why people draw this conclusion: we report that inferotemporal and prefrontal cortex alternate between conscious and suppressed states roughly every 60 ms during binocular rivalry.
Consciousness depends on many brain areas, and our paper reports results from only a few select nodes.
We still don’t know how conscious access works--the actual recipe for generating conscious experience. What we have shown is that deep in the brain there is an extremely strong signal, rich enough to reconstruct a high-dimensional face in detail, that is inaccessible to consciousness. As I argued in the original tweet, this makes the problem of conscious access a lot more experimentally tractable. But to claim “consciousness has a frame rate,” one would need to have actually solved the mechanism for conscious access and shown that this mechanism operates at a frame rate.
While we currently don’t know conclusively one way or the other (see VanRullen & Koch, 2003 for a review of positive evidence), I personally am skeptical that consciousness has a frame rate. I believe the brain, within posterior parietal cortex, has continuous tracking mechanisms that actively represent the persistence of matter, including ourselves as physical bodies. The coupling between these persistent tracking processes and feature representations across the brain (like those in face patches) is, I think, the essential mechanism of conscious access.
This coupling need not happen at the rate of inferotemporal alternations (and inferotemporal cortex itself may not even alternate outside of the artificial situation of binocular rivalry). As an analogy: imagine the area crucial for conscious access holding a steady drone note, while inferotemporal cortex alternates between two notes, one in harmony and one dissonant. Inferotemporal content might couple to the drone only at moments of harmony, yet experience would still be continuous, because the drone itself (the tracking process that I believe is at the core of conscious perception) never stops. And areas without conflicting content, like auditory cortex during visual rivalry, could stay in continuous harmony with the drone.
In sum, there is no reason to conclude from our study that consciousness is discrete, with a "frame rate."
2. Our results resemble Jezek et al. 2011 (as @jjsakon notes).
Jezek et al. trained rats in two boxes that were identical except for their lighting cues, so that hippocampal place cells formed two distinct maps, one per box. They then “teleported” the rat by abruptly switching the lights. For a few seconds afterwards, CA3 place-cell activity flickered between the two maps, synced to the theta cycle (~125 ms).
@StanDehaene first pointed this paper out to me, and reading it blew me away. To my mind it is the strongest evidence for a bistable attractor in the brain, because the competing maps are stored within the hippocampus itself.
I think the phenomenon we see may be very similar at the circuit level: a bistable attractor network within inferotemporal cortex that alternately converges to one of two patterns, as cells in face patches work together to build a coherent representation of the input. In our case, both images are present in the input, so the competition could arise within IT or be inherited from earlier areas such as V1--recording there is an important next step. (It also still puzzles me that we didn’t see strong coherence between face patches ML and AM.)
More broadly, what’s different about our paper is the brain area (inferotemporal cortex, often modeled as a feedforward DNN) and the ability to relate the two competing patterns to consciousness, with one granted conscious access and the other not.
3. Please comment on bioRxiv!
The Twitter attention has been cool, but I really hope colleagues will be generous enough to post detailed commentary on bioRxiv, where there are currently 0 comments. As @seemaychou points out, peer review doesn’t have to come from just three anonymous reviewers. We would be truly grateful for your critical feedback.
4. Stay tuned
This work was done at Caltech, Berkeley, in collaboration with @EngelTatiana at Princeton, funded by NIH and HHMI.
I hope to share our vision soon for how we can solve consciousness @AsteraInstitute
BREAKING NEWS
The 2026 #NobelPrize in Physiology or Medicine has been awarded to Karl Deisseroth, Peter Hegemann and Georg Nagel “for their discoveries concerning light-gated ion channels and optogenetics.”
@dileeplearning@foresightinst@AsteraInstitute@doristsao You might be interested in the follow-up to our stochastic latent oscillatory diffusion model #SLOD that integrates variation in neocortical cytoarchitecture into a unified cortical inference with the SLOD mechanisms contributing to it.
@JohnKubie@MillerLabMIT The conclusion this logic leads to, given the aphantasia comparison, is that phenomenal consciousness has causal consequences — not just the underlying computation.
@JohnKubie@MillerLabMIT This precludes causality that takes place in our mental world without producing a physical action. That's where I spend most of my causal energy — it eventually resolves into a reasoned action, but only after many discarded options that never touched the external world directly.
Selective feedforward inhibition.
7 mossy fibers (of 55,000 in the dataset, yellow) converge on one interneuron (green) in hippocampal CA3. Its axon makes 333 synapses onto thorny pyramidal neurons (blue) vs 50 synapses onto sparsely thorny pyramidals (purple).
Paper in @NatureNeuro https://t.co/8HfM4NhW4N
@JohnKubie@MillerLabMIT Since I can’t work through problems by visualizing them internally in my “mind’s eye”, I tend to try and do that externally. I also tend to reason using abstract causal relationships. These tendencies produce different behaviors that are designed to causally influence the world.
1/3 Conscious AI and Biological Naturalism. A veeery loooong time in the making, but I'm happy to say that the full collection is now out in @BBSJournal 🔥 I deeply thank all commentators - of fifty commentaries! - for their wise, broad, and deep opinions on this critical topic. I also thank them for their saintly patience. 🙏🏽
I remain highly skeptical about the prospects for conscious AI (for silicon digital systems), but I've learned an enormous amount from the privilege of engaging with these commentaries. Hands down one of the most rewarding experiences of my professional life.
My hope is that - whether you agree with me or not - having such a diversity of views collected in one place will serve a useful resource for discussions about consciousness and AI.
The original target article is available here: https://t.co/Tsaslytu56
My response "The stuff matters: consciousness, computation, and biology" was published just now, and is here: https://t.co/gtOewuG9MY
And the set of 50 commentaries are collected here: https://t.co/HUhiraO3kG
Unfortunately these are not open access. I will try to remedy that, at least for the target & response.
FWIW there's a more accessible version of my view on the topic here in @NoemaMag https://t.co/LmSiQnT9Wh and also a short @TEDTalks from earlier this year https://t.co/vDvw82ookk
After a long journey through review, our paper is finally out in Nature Communications.
We started with a simple problem: The visual cortex is strongly modulated by movement. So how can it represent the visual world without movement interfering with everything? 🧵
Here it is - the official, revised, peer-reviewed version of my Platonic Space paper. https://t.co/PXNpx5FgFN Of all the many unpopular positions I’ve taken over the decades - bitter controversies around the origin of left-right asymmetry in embryogenesis, bioelectricity and genetics, diverse intelligence, etc., this one has by far generated the most pushback: serious (grateful for those!) and energetic attempts to move me to other views, pleas to just drop it and not talk about it (for several different reasons), nasty emails and accusations, impacts on reviews of papers that have nothing to do with this, etc. Kind of amazing to me how incendiary this is. What can I say... Our job is to call it as we see it, and right now for me, this is it. Apologies to collaborators and colleagues for any shrapnel! Time will tell if this pans out or not; I've placed my bets. And btw, if you think this stuff is weird and uncomfortable, just wait… There’s much more on the way. The knob turns slowly but as long as the data keep coming, I'm going to say what I think it all means and follow it to the next steps it enables. Buckle up!