Origin (@origin_bio) is using AI to make safer cell & gene therapies for diseases like cancer.
Their model designs DNA switches & dials to program precise gene expression patterns in disease cells.
Congrats on the launch @YashRathod_75 and @malhar317!
https://t.co/yEHyoafehC
New blackboard lecture w @reinerpope
How do chips actually work – starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do.
0:00:00 – Building a multiply-accumulate from logic gates
0:16:20 – Muxes and the cost of data movement
0:25:59 – How systolic arrays work
0:39:00 – Clock cycles and pipeline registers
0:51:40 – FPGAs vs ASICs
1:03:14 – Cache vs scratchpad
1:07:16 – Why CPU cores are much bigger than GPU cores
1:11:49 – Brains vs chips
1:15:22 – A GPU is just a bunch of tiny TPUs
Look up Dwarkesh Podcast on YouTube/Spotify/etc to watch. Enjoy!
LLM training is built on fast MatMuls. But many surrounding ops still run as memory-bound kernels.
CODA reparameterizes them to hide in the matmul’s shadow, fused into its epilogue before results leave the chip.
Bonus: LLMs can write fast CODA kernels too (approaching SoLs).
Craziest part is we all knew each other already in high school! Along with @randomjohnnyh (Perplexity cofounder), @demi_guo_ (Pika CEO), @stevenkplus1 and Andrew (Cognition), and many others. We all grew up in different states but met thru the olympiad scene.
Vividly remember this line from @alexandr_wang when we were around 19: "I hear people saying they want to find the next Paypal mafia. Why shouldn't it just be us?"
Glad to see @chameleon_jeff get the recognition he deserves :)
AI x Bio teams like Origin have coding agents, scaling laws, and a wave of big biotech deals all at their backs. This is barely touched territory. Crazy what this small team can do now.
Today, we're excited to release 10,000 fully AI-designed enhancer sequences for the research community. Axis was prompted to design sequences with targeted activity in one of three widely used cell-lines.
AI allows us to explore a vast design space, going beyond the natural genome.
Muon can accelerate LLM training, but does that benefit transfer to regulatory DNA sequence modeling with its different data distribution? 🧬
Our results show that Muon with independent weight decay (MuonW) hits our validation perplexity target in ~37% fewer FLOPs than the best Adam configuration.
One of my favorite lessons I’ve learnt from working with smart people:
Action produces information. If you’re unsure of what to do, just do anything, even if it’s the wrong thing. This will give you information about what you should actually be doing.
Sounds simple on the surface - the hard part is making it part of your every day working process.
Going on TBPN at 1:45 pm PT to launch @origin_bio and our model, Axis. Axis is the first model that generates regulatory DNA sequences and predicts its function.
Axis already outperforms DeepMind's AlphaGenome on various benchmarks!
From bits to biology. It's been an incredible journey working on the core compute infra for a new AI model. We spent countless hours maximizing GPU utilization with custom kernels, turning raw hardware power into a tool that understands DNA. 💻 -> 🧬
Today, we are thrilled to announce Axis. It’s the first AI model to unify generative design with predictive validation for regulatory DNA. In short, it can both design novel DNA sequences and predict their functional properties.
The result? Axis already outperforms Google DeepMind’s AlphaGenome on benchmarks. This is a huge first step towards building unified AI models for Biology, and I'm so proud of what our team at Origin has built.
Learn more on our blog: https://t.co/MuhiCLwIUQ
Sign up for early access: https://t.co/RXf1Ke6OhW
Introducing Axis: the first AI model that generates regulatory DNA elements and predicts their function.
Gene therapies suffer from poor efficacy, toxicity & specificity. Models like Axis can help overcome such risks.
Axis beats Google DeepMind's AlphaGenome at predicting regulatory element binding activity by 6.7%.