Doris Tsao

cortical geometer

[Doris Tsao @doristsao]

How does matter become bright inside? I am deeply excited to share our vision @AsteraNeuro for understanding and engineering consciousness.

We believe that perception and thought are constructed from fundamental building blocks, and that by understanding these blocks and how the brain combines them, we can explain the whole mystery of consciousness.

Writing this vision statement helped me think more clearly about questions that have long bothered me: Why did consciousness evolve? What are we actually conscious of? What is the self? How do different parts of the brain work together? How can we tell whether an AI is conscious? What scientific path can lead to real answers?

Part 1 presents a new theory. I propose that conscious experience arises from a coherent internal model of the world and the self built from 'persistent symbols.' I describe the neural representations and dynamics through which the brain could implement such symbols.

Part 2 lays out a roadmap to systematically discover the basic building blocks of perception, cognition, and the self-model, and to build the neural read/write technologies and org culture we need to get there.

Part 3 presents a dream: What will become possible once we acquire the ability to read and write experience--to truly engineer consciousness?

If you are a systems neuroscientist who feels something fundamental is missing from our field, I share this feeling. Understanding the phenomenon that compelled many of us to become neuroscientists in the first place should be a central organizing goal of our field. This vision statement lays out a research program to reach this goal.

If you read the vision statement and recognize that this is the mission you have been longing for, please get in touch, wherever in life you are right now.

I made this stone soup for you.

https://t.co/z8CRrEESce

[Doris Tsao @doristsao]

Thank you for reading! What do you find delusional? 1) I don't think it is...I make a big point about the distinction between simulation and physical convergence, 2) Idk 3) We are not forgoing peer review--we encourage it. We are forgoing publication in traditional peer-reviewed journals.

[Doris Tsao @doristsao]

How does matter become bright inside? I am deeply excited to share our vision @AsteraNeuro for understanding and engineering consciousness.

We believe that perception and thought are constructed from fundamental building blocks, and that by understanding these blocks and how the brain combines them, we can explain the whole mystery of consciousness.

Writing this vision statement helped me think more clearly about questions that have long bothered me: Why did consciousness evolve? What are we actually conscious of? What is the self? How do different parts of the brain work together? How can we tell whether an AI is conscious? What scientific path can lead to real answers?

Part 1 presents a new theory. I propose that conscious experience arises from a coherent internal model of the world and the self built from 'persistent symbols.' I describe the neural representations and dynamics through which the brain could implement such symbols.

Part 2 lays out a roadmap to systematically discover the basic building blocks of perception, cognition, and the self-model, and to build the neural read/write technologies and org culture we need to get there.

Part 3 presents a dream: What will become possible once we acquire the ability to read and write experience--to truly engineer consciousness?

If you are a systems neuroscientist who feels something fundamental is missing from our field, I share this feeling. Understanding the phenomenon that compelled many of us to become neuroscientists in the first place should be a central organizing goal of our field. This vision statement lays out a research program to reach this goal.

If you read the vision statement and recognize that this is the mission you have been longing for, please get in touch, wherever in life you are right now.

I made this stone soup for you.

https://t.co/z8CRrEESce

[Doris Tsao @doristsao]

So excited that @maxsbennett is joining. Something very special about Astera is that it is a place for people who seek principled understanding. And Max totally exemplifies that, as readers of his book know.

[Max S. Bennett @maxsbennett]

I'm v excited to join @AsteraInstitute as a part-time research scientist. @dileeplearning has been a friend for years; in fact, it was his PhD thesis years ago that first got me into neuroAI. Excited to work with him and the amazing team he has assembled. Stay tuned for 🧠+🤖

[Doris Tsao @doristsao]

Looking for a postbac opportunity? Come join us!

[Astera Neuro @AsteraNeuro]

Just graduated and want to work on conscious world models using cutting-edge neurotechnology? You do not have to wait for a PhD.

SHIFT is our paid postbac research fellowship. $75K, full benefits, close mentorship, onsite in the East Bay.

Applications close January 15. https://t.co/LBmOTizfg3

[Doris Tsao @doristsao]

Follow @AsteraNeuro to get our latest updates on career opportunities, datasets, and discoveries 👇

[Astera Neuro @AsteraNeuro]

How does the brain give rise to conscious experience? That is the question we are building Astera Neuro to answer. Come see what we are working on, who we are, and how to join us. https://t.co/GWEjSx4FSK

[Doris Tsao @doristsao]

I think extracting conscious symbols is more than tokenization. 1) I think it is a very specific computation of factorizing persistent features from transformation, including self transformation. 2) I think it requires physical convergence of a coarse-grained dynamical system representing the symbols. So no, I don't think an llm that learns its own discretization will become conscious. At the same time, I don't think the recipe is infinitely complicated either. It is something we will be able to understand and build.

[Doris Tsao @doristsao]

@EliSennesh @AsteraNeuro My hunch is that cortex plays the essential role in generating the structure of our conscious experience (persistent symbols bound to features), and even if thalamus is essential, it is through its effect on cortical signals.

[Doris Tsao @doristsao]

@chang_c_liu @AsteraNeuro Thank you so much, Chang!

[Jacques Carolan @JacquesCarolan]

Advances across neurotech (including today's chemogenetics!) have convinced me that powerful treatments for devastating brain conditions are possible.

The next frontier is making them massively scalable, without requiring 10-hour brain surgery.

Today, my team and I at @ARIA_research are announcing 10 teams we're backing with £50m to tackle this challenge.

A thread 🧵 on their exciting projects!

[Doris Tsao @doristsao]

Imagining myself reading this ten years ago through a crystal globe, and dying of curiosity. How lucky we are!

[David Bau @davidbau]

It's been such a lovely fall and I'm afraid I haven't done nearly enough entertaining. Things have been so hectic, with the singularity going on and everything.

[Kording Lab 🦖 @KordingLab]

Computer vision and connectomics jobs at mindspan just dropped. https://t.co/6bCyhL1Ntz Build deep learning models for the one of the world's largest segmentation jobs. Build great and robust pipelines. Annotate or proofread. Mindspan is hiring.

[Doris Tsao @doristsao]

@dyamins Congratulations, this is super exciting! Can't wait for the science to come. #GoTeamScience #GoDan

[Daniel Yamins @dyamins]

Stanford Digital Brain Project has launched! We're hoping to make a seriously useful model of the human brain. Please come join us! https://t.co/Gnzm6wJS8d

[Dileep George @dileeplearning]

Compute is the helium of the AI era.

In 1920, everyone bet on bigger, better balloons. You can fly without understanding aerodynamics, and you can get intelligent behavior without fully understanding intelligence.

Talked dirigibles and brains with @fuelfive on Progress.

[Doris Tsao @doristsao]

"What we [in neurotech] do in the next ten years will determine how AI and humans co-exist" - Juan Benet.

I couldn't agree more. Juan and team @protocollabs are true visionaries pushing forward this incomparably important frontier.

[Juan Benet @juanbenet]

Neurotech is Humanity’s Next Frontier!

I’ve been investing in neurotech since 2021 and working with teams building BCIs, NeuroAI, and connectomics. It now feels like AI did in ~2019.

We are entering an extraordinary era: neurotech already restores functions, it can enhance our capabilities, and will evolve what it means to be human.

PLC Neurotech is the fund backing the extraordinary founders and defining companies of this booming industry.

[Doris Tsao @doristsao]

@neurexplorer @HesseJanis Hi Julien, I don't think our study provides evidence that perception is discrete, please see https://t.co/SjK9eLEoC1

[Doris Tsao @doristsao]

The outpouring of interest in our paper has been … a bit insane.

May I make four points?

1. We have not shown that “consciousness has a frame rate.”

I can see why people draw this conclusion: we report that inferotemporal and prefrontal cortex alternate between conscious and suppressed states roughly every 60 ms during binocular rivalry.

Consciousness depends on many brain areas, and our paper reports results from only a few select nodes.

We still don’t know how conscious access works--the actual recipe for generating conscious experience. What we have shown is that deep in the brain there is an extremely strong signal, rich enough to reconstruct a high-dimensional face in detail, that is inaccessible to consciousness. As I argued in the original tweet, this makes the problem of conscious access a lot more experimentally tractable. But to claim “consciousness has a frame rate,” one would need to have actually solved the mechanism for conscious access and shown that this mechanism operates at a frame rate.

While we currently don’t know conclusively one way or the other (see VanRullen & Koch, 2003 for a review of positive evidence), I personally am skeptical that consciousness has a frame rate. I believe the brain, within posterior parietal cortex, has continuous tracking mechanisms that actively represent the persistence of matter, including ourselves as physical bodies. The coupling between these persistent tracking processes and feature representations across the brain (like those in face patches) is, I think, the essential mechanism of conscious access.

This coupling need not happen at the rate of inferotemporal alternations (and inferotemporal cortex itself may not even alternate outside of the artificial situation of binocular rivalry). As an analogy: imagine the area crucial for conscious access holding a steady drone note, while inferotemporal cortex alternates between two notes, one in harmony and one dissonant. Inferotemporal content might couple to the drone only at moments of harmony, yet experience would still be continuous, because the drone itself (the tracking process that I believe is at the core of conscious perception) never stops. And areas without conflicting content, like auditory cortex during visual rivalry, could stay in continuous harmony with the drone.

In sum, there is no reason to conclude from our study that consciousness is discrete, with a "frame rate."

2. Our results resemble Jezek et al. 2011 (as @jjsakon notes).

Jezek et al. trained rats in two boxes that were identical except for their lighting cues, so that hippocampal place cells formed two distinct maps, one per box. They then “teleported” the rat by abruptly switching the lights. For a few seconds afterwards, CA3 place-cell activity flickered between the two maps, synced to the theta cycle (~125 ms).

@StanDehaene first pointed this paper out to me, and reading it blew me away. To my mind it is the strongest evidence for a bistable attractor in the brain, because the competing maps are stored within the hippocampus itself.

I think the phenomenon we see may be very similar at the circuit level: a bistable attractor network within inferotemporal cortex that alternately converges to one of two patterns, as cells in face patches work together to build a coherent representation of the input. In our case, both images are present in the input, so the competition could arise within IT or be inherited from earlier areas such as V1--recording there is an important next step. (It also still puzzles me that we didn’t see strong coherence between face patches ML and AM.)

More broadly, what’s different about our paper is the brain area (inferotemporal cortex, often modeled as a feedforward DNN) and the ability to relate the two competing patterns to consciousness, with one granted conscious access and the other not.

3. Please comment on bioRxiv!

The Twitter attention has been cool, but I really hope colleagues will be generous enough to post detailed commentary on bioRxiv, where there are currently 0 comments. As @seemaychou points out, peer review doesn’t have to come from just three anonymous reviewers. We would be truly grateful for your critical feedback.

4. Stay tuned

This work was done at Caltech, Berkeley, in collaboration with @EngelTatiana at Princeton, funded by NIH and HHMI.

I hope to share our vision soon for how we can solve consciousness @AsteraInstitute

[Doris Tsao @doristsao]

1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness.
New preprint: https://t.co/K8BwiHXvXg (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)

[Doris Tsao @doristsao]

@bobking Thank you, Pinglei, this means a lot!

[Doris Tsao @doristsao]

FYI, for those wondering: we are working on organizing everything, and will be releasing data and code for our paper on Monday 10/26/26.

[Doris Tsao @doristsao]

Thank you, Itzhak, for this thoughtful response! I agree with you that the neural structure of consciousness involves more than the few nodes we recorded from. Indeed, it pains me to see all of twitter attributing to me the simplistic idea that consciousness has a frame rate, when in fact I believe that the core question we need to answer to understand consciousness is the perception of persistence. And if consciousness did have a frame rate, which I am skeptical of, this would be even more the core question. My hunch is that the key computations for persistence actually occur in posterior parietal cortex, where the brain is representing persistent pieces of physical matter multi-modally. So yes, even I can see there is more to heaven and earth...

Re the specific refs you cite: I just read both of them again. Both are super interesting (esp this super early response in mPFC preceding spontaneous alternations!), and it would be great to re-analyze the data to see how it compares to our findings (though likely what we really need is new high channel count human recordings during rivalry). I think the example unit you show in figure 2 of Gelbard-Sagiv does show weaker modulation to rivalry than perception, no?

Re Kreiman 2002: responses were analyzed starting at 100 ms after flash. We found that at this latency in monkey IT the transient associated with the flash was still ongoing, which would lead to artificially elevated perceptual modulation, so we analyzed starting at 300 ms, after the transient was over. Nevertheless, I do think it is compelling that not one MTL neuron showed a significant response to the suppressed stimulus.

But more broadly, do you think finding a pure representation of consciousness at the level of a concept cell would vindicate the quest for spatial localization of consciousness? I think the concept cells simply don't have the resolution to represent our qualia of an object (nor do IT cells on their own, for that matter...I think the whole quest was doomed, even independent of our new paper).