Weinan Sun

NeuroAI @AsteraInstitute
Prev. Asst Prof @Cornell

[Weinan Sun @sunw37]

1/12 How do animals build an internal map of the world? In our new paper, we tracked thousands of neurons in mouse CA1 over days/weeks as they learned a VR navigation task. @nspruston @HHMIJanelia, w/ co-1st author @JohanWinn
Video summary: https://t.co/mMvsT8TVc9
Paper: https://t.co/35k5tWdvOe

[Weinan Sun @sunw37]

@sofroniewn @AnthropicAI super cool!

[Doris Tsao @doristsao]

We show face patches implement the following code through recurrent dynamics:

Detect face
If (face found)
Discriminate face
else
Continue to detect face

IMHO, our paper conclusively resolves a debate that has raged since I was a graduate student, about whether face patches are specialized for processing faces or not. It turns out domain-general folks were right early on, domain-specific folks were right later in the response.

So proud of @Yuelin_Shi and the entire team!

[Weinan Sun @sunw37]

@karpathy I think you will enjoy reading about the hippocampal-neocortical system in neuroscience, a lot of continual learning algorithm hints can be found there. Agree more unconventional research is needed.

[Andrej Karpathy @karpathy]

There was a nice time where researchers talked about various ideas quite openly on twitter. (before they disappeared into the gold mines :)).

My guess is that you can get quite far even in the current paradigm by introducing a number of memory ops as "tools" and throwing them into the mix in RL. E.g. current compaction and memory implementations are crappy, first, early examples that were somewhat bolted on, but both can be fairly easily generalized and made part of the optimization as just another tool during RL.

That said neither of these is fully satisfying because clearly people are capable of some weight-based updates (my personal suspicion - mostly during sleep). So there should be even more room for more exotic approaches for long-term memory that do change the weights, but exactly - the details are not obvious. This is a lot more exciting, but also more into the realm of research outside of the established prod stack.

[Weinan Sun @sunw37]

Great opportunity in an amazing team. Highly recommended! @E11BIO @JohanWinn

[Johan Winnubst @JohanWinn]

🚀 Looking to hire a light-sheet microscopy scientist to help us design, build, and optimize the next-gen platform that will power mammalian connectomics at scale. If you’re excited to join a fast-paced team doing high-impact science, take a look: https://t.co/wRN2INtnwd🔬⚙️

[Weinan Sun @sunw37]

Everything is open-source and designed as building blocks, not just for our lab. We're relicensing under Apache 2.0 in the coming weeks to make adoption even easier.

We built this for 2-photon imaging + VR behavior in mice, but the patterns should transfer to any hardware-intensive lab. If your science hits a wall at the hardware layer, this is for you.

Paper: https://t.co/t4ken72VD5
Code: https://t.co/4SXZ8Vlr1p

[Weinan Sun @sunw37]

Closed-loop AI scientists will need to talk to lab hardware. Towards that goal, we released Ataraxis - an open-source framework that gives AI coding assistants direct access to physical instruments. Built by our talented Ph.D. student @InkarosEng @CornellNBB @Cornell.

Paper: https://t.co/t4ken72VD5
Code: https://t.co/4SXZ8Vlr1p

Demos videos:

🎥 Pre-session validation: https://t.co/SnbG8VH5wk
🎥 Hardware troubleshooting: https://t.co/1cAc97Fyq0
🎥 AI-guided hardware integration: https://t.co/ip6tAtvOle

[Weinan Sun @sunw37]

Key design choice: AI helps at configuration time only. During actual experiments, everything runs deterministically with no AI in the loop.

Network goes down? API rate limited? Doesn't matter - your experiment keeps running.

[Victor M @victormustar]

I just made my own Star Wars clip using AI (text-to-video)!

[Toon Van de Maele @toonvdm]

(1/n) How do we generalize knowledge across similar experiences? In our new preprint, we introduce S-HAI: a hierarchical active inference model that captures "schemas" used by humans and animals to generalize task abstractions.
https://t.co/hwv6y4GBbA 🧵

[Alice Ting @aliceyting]

Janelia is hiring Group Leaders! Work alongside superstars like Luke Lavis, @JiefuBiol, @mengwang939, @JLS_Lab, @ERSchreiter. If you have ideas that are perhaps too ambitious or crazy for a traditional academic setting, that is exactly who they are looking for.

Tool-builders at all career stages with original, transformative ideas for experimental approaches in imaging, molecular engineering, protein chemistry, mass spectrometry, and methods that don’t yet exist: Apply by Feb. 3, 2026: https://t.co/WcEOWPoxw5

[Niels Rogge @NielsRogge]

"Before Transformers, RNNs were the thing. These were a big breakthrough. Suddenly, everyone started to work on improving RNNs. But the results were always these slight modifications on the same architecture, like putting the gate in a different spot, with improvements to 1.26, 1.25 bits per character on language modeling."

"After the Transformer, when we applied very deep decoder-only Transformers to the same task, we immediately got 1.1 bits per character. So all that research on RNNs suddenly seemed a waste of time".

"We're currently in the same situation where a lot of papers are taking the same architecture (Transformer) and making these endless tweaks, in a local minimum, and we might be wasting time in exactly the same way."

- Llion Jones, co-author of the Transformer on @MLStreetTalk

[Dileep George @dileeplearning]

If no one builds it, you're never born.
..not building AGI is a risky thing.

Wilbur Wright, inventor of airplanes, died in 1912 at the age of 45 from typhoid fever because antibiotics didn’t exist then. Just imagine how our world would be without the medical and technological advancements of the past century! Actually, you wouldn’t have to imagine, because you wouldn’t exist!

Inventing technology and advancing science is how we overcame our challenges and managed to support 8 billion human souls on this planet, escaping the Malthusian trap of famines, diseases, and conflicts.

Automation of knowledge acquisition and thought is the next step, and the best tool humanity can build. The risk of not building AGI is that we won’t be prepared for the challenges the world throws at us, some of which would be challenges that our own existence creates.

A(G)I safety is important, and here are my thoughts about it.

1. Scaling up current techniques is not going to lead to AGI. It will lead to powerful AI systems, but these will be supported by a lot of engineered scaffolding. In these cases, making AI work usefully is almost exactly the same as making AI safe. Since scaling has already proven to be useful, we are naturally on the path to exploiting it to the maximum, and we should.

2. We will eventually figure out how to build and scale AI that uses principles of human intelligence. These systems will learn causal structure and reliable world models that can be used for counterfactual thinking. This will lead to much more capable AI systems and AGI. But for these kinds of systems, increasing capability can also come with increasing controllability.

Where powerful AI and AGI is going to help us

Earthquakes, wildfires, hurricanes, floods: Despite all the technological advances, we are still at the mercy of nature when it comes to these disasters. Where are the army of robots helping to dig out people from collapsed buildings? Where are the ones to manage fires and help people? Having AGI means we will have robots that will help us in these cases to save lives and to recover faster.

Health: Antibiotic resistance, pandemics, …, we don’t know what challenges we will face in the future and it would be great to have powerful tools. In general, having much better understanding of how our bodies and minds work, and curing of diseases.

Flora and Fauna: Instead of conforming to the requirements of dumb machines, we might finally be able to do more organic multi-crop agriculture, reduce the amount of pesticides we use, and abolish factory farming. Intelligent machines will free us from the economic necessity of these.

Climate change, Energy, Materials, Education, Transportation, Space …. examples like these abound in each of those areas. Things that we accomplished crudely with dumb machinery will be done with more finesse with intelligent machines, and that will be important for humanity to thrive at scale.

Balancing the risks…

Of course the title is a play on the Yudkowsky and Soares book “If anyone builds it, everyone dies”. While I disagree with many things that the book asserts, their work has brought attention to the important problem of AI safety. Smart people working on AI safety is a good thing. It is important to continue that work, even if the specific x-risk scenarios in the book can be taken apart. In the midst of all the talk about the risks of AGI it is important to realize that not building AGI has risks as well.

[Weinan Sun @sunw37]

You should interview @dileeplearning. Can’t think of anyone else who can connect between neuroscience and AI as well as he can. From co-founding Numenta and Vicarious to getting acquired by Alphabet, he had an interesting journey building neuroscience inspired AI. Our recent Nature paper was heavily influenced by his work.

[Machine Learning Street Talk @MLStreetTalk]

Wouldn't it be great if neural networks knew a spiral... as a spiral? 🍥

[Weinan Sun @sunw37]

hi! based on my testing on various file formats with different sizes, Kosmos could analyze all kinds of files (see below). Claude 4.5 could deal with some of them, but often limited by upload size limit, and for some files, the reasoning pushes to max token limit. This reading ability could be a major unlock for science as some of these data were buried for more than 10 yrs with software license expired. Analysis is not perfect yet but feels this is a major step to the right direction.

[Weinan Sun @sunw37]

@SGRodriques wow, congrats Sam and your team on this milestone! super impressive. just dumped an old whole-cell recording file into it, and got this beautiful summary:

[Weinan Sun @sunw37]

@dileeplearning @RichardSSutton "computation" in the first sentence?

[Adam Marblestone @AdamMarblestone]

The US should do something equally big in brain circuit mapping
https://t.co/qZerDAjbJf