Post-Transformers jam session between @SakanaAILabs and @pathway_com on AI and the brain - the recording is now live!
In a guest lecture for Sakana @adrian_pathway, Pathway’s CSO, asked what can natural intelligence teach us about memory, sparsity, locality, and reasoning over time? The Sakana team then followed up with questions that will define the Post-Transformer Era. Thanks to @YesThisIsLion for inviting us!
@pawell I still have my decks and binders from ~20 years ago somewhere in the basement. Always mill deck type for me. some peeps just want to see decks burn :)
“It would take someone with a lot of patience, budget, and conviction to push forward approaches that go beyond Transformers, approaches that don’t give you immediate gains on benchmarks. Those approaches are unfortunately cut out because of incentives in Machine Learning.” — Peter Norvig at the Open Source AI Summit in SF.
And then @adrian_pathway took the stage to talk about the Post-Transformer Frontier :)
For me, it was pretty incredible to see the three “big moments in AI” jamming together at a cosy meetup in the middle of a calm afternoon in the Presidio. Peter (shaped the field of AI) - Illia (co-invented Transformers) - Adrian (leads Post-Transformers).
This event was an absolute gem. I genuinely had fun, took notes and photos of slides (!)
The next generation of AI is inevitable. It is being built on efficiency, not brute force. The first generation of AI won by massively scaling the Transformer. At Pathway, we are here to change this.
Thank you to @ilblackdragon, @ekang426, @build_planet, @near_ai, and @open_intel_fdn for bringing the community together!
150M params. 29.5% on ARC-AGI-1. $0.0007 a task, ~11x cheaper than GPT 5.6 Luna.
No chain-of-thought, it reasons in latent space. The bet is that the wall isn’t compute, it’s the architecture.
Congrats to the awesome team behind this ! @zuzanna_pathway@pathway_com
Pathway’s BDH-CQ model redefines the Cost-Efficiency Frontier on ARC-AGI-1: $.0007 at 29.5%. This is made possible by in-context learning and latent reasoning. Welcome to the Post-Transformer Era.
Pathway’s BDH-CQ model redefines the Cost-Efficiency Frontier on ARC-AGI-1: $.0007 at 29.5%. This is made possible by in-context learning and latent reasoning. Welcome to the Post-Transformer Era.
Today is a big day for Routine! 🎉
After 5 years of building, we're finally unveiling Routine AI.
Routine AI transforms your workspace — tasks, calendar, notes, projects etc. — into an AI-powered assistant with voice commands (a.k.a super Siri), AI agents, and automations.
Transformer vs Post-Transformer: The 5-minute KO compilation is live now. 🥊
@lukaszkaiser (co-invented Transformer & co-created ChatGPT)
@adrian_pathway (invented BDH and is CSO of Pathway)
@mlech26l (co-invented LNNs & is CTO of Liquid AI)
@YesThisIsLion (co-invented Transformer with Łukasz, now CTO of Sakana AI)
Moderated by @dexhorthy (CEO, HumanLayer) and me.
Full debate drops soon. Turn on notifications to catch the complete fight. This is the ultimate source of truth on the subject.
Proactive agent that thinks and acts like you.
Multiplayer AI Brain for teams.
Proper GUI for commanding 50 agents.
https://t.co/zsDxICHezv is all three. Sauna goes live today. First 2000 people, use access code LAUNCH for $80 of weekly(!) credits.
Let’s explain. Multiplayer only works once the personal brain is powerful. So let's start here.
Personal AI Brain
3,800+ tools connected. State of the Art memory. Skills and schedules you teach once that get repeated forever on cheaper models. An AI first CRM.
Lives on the cloud so you can initiate tasks from anywhere: iMessage, Slack, Email. Also no need for a Mac mini 😉
GUI
AI agents have been stuck in their MS-DOS era. A chat box, a scroll buffer, no way to command 50 of them. We built the first GUI: Live sessions on one side, work waiting for your sign-off on the other, plus the things Sauna kicked off while you were asleep waiting for review. Game mode helps clear the queue with actual joy.
Okay so far so good, but how to give benefit of what you built to more people or whole team?
Multiplayer
Once your Sauna actually knows you and you gave her access to your tools, you can use multiplayer.
Two modes:
- Brain access. My co-founder Robert plugged my brain as a tool into his Sauna last month. He can ask it about pricing while I'm in other meetings, gets a sourced answer back, never has to interrupt me. That’s read only. Yolo mode gives him access to all my tools too :O
- Communal Saunas extend that to whole companies with proper permissioning. Folder owners decide what's true for the whole company and build skills. Most get their personal brain + read access to communal files and memories. Works also for group planning my best friend's bachelor party.
The Way
I’ve been obsessed about AI Brain since 2016. Our human brains suck at some things like memory and are brilliant at others like creativity. We are also particularly bad at thinking we are all on the same page and then realising weeks later that we weren’t. Most leaders spend their days being the human diff tool, catching contradictions in hallway conversations and Slack threads. Repeating themselves 50 times. Now every company is spinning up hundreds or thousands of agents that drift faster than humans do, feeding each other their drift as context. Compounding rot.
After 10 years and one failed company in this space we are launching the solution.
The Launch
We thought about a celebrity launch. Margot Robbie in a bathtub explaining agents. But then we realised the same money gives the first 2,000 people free daily credits, every day, until we burn through that $1,000,000.
Sauna runs Claude, GPT-5, Gemini, GLM-5.1, DeepSeek, Kimi so you can budget yourself. The labs are racing to lock you so they can milk you in a year. We picked your side.
Onboarding
It’s live at https://t.co/8ekc1ZtzoJ We’ve preheated saunas based on a niche. Use the access codes in the comments to get a better experience.
Inventors of Transformer and Post-Transformer architectures are stepping into a boxing ring in San Francisco!
This is the battle between innovations that shape trillion dollar markets - presented by their very authors (!).
May 5. Thread 🥊
Today we are sharing a new result from BDH: 97.4% accuracy on Extreme Sudoku puzzles while maintaining language fluency.
No chain-of-thought
Current LLMs → nearly 0% accuracy.
If a model can write beautifully but still cannot reason through a hard constraint space, that is not a side issue.
That is the issue.
As @Forbes's @iamVictorDey writes, Pathway's BDH “may have sparked the beginning of a new era in AI — one where machines don’t just imitate the brain, but begin to think like it.”
https://t.co/UmgWRAOL1M
Breaking news: Adrian Kosowski speaks to @JonKrohnLearns about #machinelearning’s entry into neurology research via a biologically inspired architecture.
Watch the episode here: https://t.co/oApQSeJf2y
Pathway just published Dragon Hatchling (BDH), and I’ve been digging into the paper.
It’s a new LLM architecture where continuous learning is native—via synaptic plasticity and Hebbian updates—rather than bolted on with scaffolds.
Technical breakdown ⤵️