Taleb is helpful here: it doesn’t really matter if you’re wrong a lot about minor survivable things, if you’re really right about the big things. (and inversely, it doesn’t matter if you’re right about a lot of things, if you’re wrong about the thing that kills/ruins you)
We think computational depth is the missing scaling axis, i.e. we should be doing a lot more deep learning!
Every other axis has been scaled by OOMs over the past few years (params, data, sparsity, test-time reasoning), but depth has been stuck at ~100 layers since GPT-3.
We've found that LLMs are both *severely* depth-bottlenecked and bad at using the depth they have, and that architectural interventions that lift this bottleneck efficiently lead to gains that increase with compute.
w/ @akshayvegesna
biggest startup learning so far: ignore literally everything and everyone that is not immediately ultra critical to what the biggest problem is right now
everything that feels like it's "probably super useful in 3 weeks" is completely useless, NEVER do it
there are three world changing products of the ai era so far:
1. chatgpt which made intelligence universal although with little monetization (hence the openai pivot to enterprise).
2. claude code which made ai an agent that performs actual work. it took over the globe in a single swoop (likely the biggest reason for $100b arr for anthropic).
3. midjourney which made visual imagination executable very early.
what are your top three?
We discovered a third pretraining axis beyond parameters and data: exploration.
Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation.
In the simplest case, it's just a for loop.
Introducing Explorative Modeling.
TLDR:
- Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute
- Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet
- Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is
- End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute
🧵Thread:
you have five (5) more days to exploit the ultra mega dangerous superintelligence to solve all your health problems, win friends, influence people, make money, and find God for a mere 20 bucks
I almost feel like we solved CUA with 5.6.
For the new dictation model launch, Sol completely did the roll out over 15 hours using browser and computer use.
It came up with a plan for increments of rollout %, moving GPU’s, checking metrics and asking for input when needed.
An app can be a home-cooked meal (2020)
personal software was a bit early in 2020 but in 2026, it really can be as personal as a home cooked meal, or a handwritten letter
https://t.co/vQLa9wxUzq
Most research has a strongly aesthetic and emotional quality to it. As Hadamard talks about in The Mathematician's Mind, the subconscious is where research directions and their solutions arise from, and it is strongly motivated by aesthetic considerations.
The role of logic / the conscious mind is only to verify that these aesthetically-pleasing solutions that arise from the subconscious are correct.
The fallacy of this is that more creates more. More hours, more hiring, more something.
And it is true in a sense. If you put in more work, more work will happen. But I think for most startups, the leverage is really in how differently you approach the problem, how well you cultivate your team, and the strategy.
Any large company can outspend you on hours. They have thousands or tens of thousands more people, spending more hours. If hours worked were the metric, every large company and government organization would always win and do the best work. More hours, better output.
This thinking is often representative of younger founders, where the startup becomes their identity and life. They have a hard time doing anything else, and cannot understand that your work is not the person that is you. But activities outside of work can grow you as a person too and make you do better work.
I’ve never worked this way. As a designer, I always saw the need to take a step back, to take a break. At times, I might work 12 hours or 16 hours, or whatever amount was needed, but it wasn’t the norm. You just can't grind design, you need inspiration. But taking that step away from the work, would give me more perspective, inspiration and I could approach the problem differently or I could just see the solution.
Grinding is never good for any creative problem, and startups or creating new products are often mostly about creative problem solving. Grinding works ok for email jobs, or where you just executing on very clear playbook.
With Linear, we’ve never worked this way. We work reasonable hours, 5 days a week. All of us founders have families. Many of our employees have families. I personally stop every evening, spend time with the family, cook dinner for the family, eat dinner together, and focus on things outside of work. Sometimes I work in the late evenings or weekends, but to me the pride is that I don’t need to. Company should be succesful without it.
My goal is to build a company that is sustainable in the long term, and doesn’t require heroics or personal sacrifices every single day.
There are times when our team is heroic. Launches, incidents, some other work that just needs to be done. They will work late into the night because they know it is the right thing. But we don’t require that every day or every week, and the more this happens, the more I think it is a failure of our company and leadership. The team and the leaders should always keep a reserve to use when something is needed.
Our thinking was also that quality, which we value, doesn’t emerge from working more or stressing people more. It emerges when you create the conditions for it to emerge. Often it is the appreciation, space, time, and how the person feels. A person who is rested will do better work.
I wouldn’t attribute much of our success to working a lot. The success came from having clear thinking, ideas, and focus to do the right things.
I sometimes wish we could move the culture more toward a Zen master.
Real mastery is not exerting the most effort. It is achieving the outcome with the least necessary effort.
This quote from Peter Thiel in 2014 has always resonated with me. Finite and infinite games aren’t only for business; they also apply to life generally.
“People always say they want to live every day as though it will be their last. I always have this contrasting view that I think I’d like to live every day as though it will go on forever. If we had an indefinite life span, we would continue to work and start great new projects, we would be very careful about how we treated the people around us because we would encounter them again.”
1/ Controversial take: hard work is more important than smart work.
It's a myth that we only have a few hours of good creative work per day. Train yourself to grind long hours first. You will surprise yourself. The work naturally become higher quality, less distracted.
Imagine every pixel on your screen, streamed live directly from a model. No HTML, no layout engine, no code. Just exactly what you want to see.
@eddiejiao_obj, @drewocarr and I built a prototype to see how this could actually work, and set out to make it real. We're calling it Flipbook. (1/5)
Introducing ml-intern, the agent that just automated the post-training team @huggingface
It's an open-source implementation of the real research loop that our ML researchers do every day. You give it a prompt, it researches papers, goes through citations, implements ideas in GPU sandboxes, iterates and builds deeply research-backed models for any use case. All built on the Hugging Face ecosystem.
It can pull off crazy things:
We made it train the best model for scientific reasoning. It went through citations from the official benchmark paper. Found OpenScience and NemoTron-CrossThink, added 7 difficulty-filtered dataset variants from ARC/SciQ/MMLU, and ran 12 SFT runs on Qwen3-1.7B. This pushed the score 10% → 32% on GPQA in under 10h. Claude Code's best: 22.99%.
In healthcare settings it inspected available datasets, concluded they were too low quality, and wrote a script to generate 1100 synthetic data points from scratch for emergencies, hedging, multilingual etc. Then upsampled 50x for training. Beat Codex on HealthBench by 60%.
For competitive mathematics, it wrote a full GRPO script, launched training with A100 GPUs on https://t.co/udm7xGpNzR, watched rewards claim and then collapse, and ran ablations until it succeeded. All fully backed by papers, autonomously.
How it works?
ml-intern makes full use of the HF ecosystem:
- finds papers on arxiv and https://t.co/brvCC7fLPa, reads them fully, walks citation graphs, pulls datasets referenced in methodology sections and on https://t.co/hrJuRkRyzi
- browses the Hub, reads recent docs, inspects datasets and reformats them before training so it doesn't waste GPU hours on bad data
- launches training jobs on HF Jobs if no local GPUs are available, monitors runs, reads its own eval outputs, diagnoses failures, retrains
ml-intern deeply embodies how researchers work and think. It knows how data should look like and what good models feel like.
Releasing it today as a CLI and a web app you can use from your phone/desktop.
CLI: https://t.co/l3K1PslZ1n
Web + mobile: https://t.co/orko5srL4H
And the best part? We also provisioned 1k$ GPU resources and Anthropic credits for the quickest among you to use.