It is time → https://t.co/YlX841ArFS.
2026, the year of 𝚎𝚏𝚏𝚎𝚌𝚝.
I'm extremely proud of how this turned out. There are many chapters to add, but I do hope you enjoy what's there so far. Let me know what you think. 🫡
Why can AIs code for 1h but not 10h?
A simple explanation: if there's a 10% chance of error per 10min step (say), the success rate is:
1h: 53%
4h: 8%
10h: 0.002%
@tobyordoxford has tested this 'constant error rate' theory and shown it's a good fit for the data
chance of success declines exponentially
Good post from @balajis on the "verification gap".
You could see it as there being two modes in creation. Borrowing GAN terminology:
1) generation and
2) discrimination.
e.g. painting - you make a brush stroke (1) and then you look for a while to see if you improved the painting (2). these two stages are interspersed in pretty much all creative work.
Second point. Discrimination can be computationally very hard.
- images are by far the easiest. e.g. image generator teams can create giant grids of results to decide if one image is better than the other. thank you to the giant GPU in your brain built for processing images very fast.
- text is much harder. it is skimmable, but you have to read, it is semantic, discrete and precise so you also have to reason (esp in e.g. code).
- audio is maybe even harder still imo, because it force a time axis so it's not even skimmable. you're forced to spend serial compute and can't parallelize it at all.
You could say that in coding LLMs have collapsed (1) to ~instant, but have done very little to address (2). A person still has to stare at the results and discriminate if they are good. This is my major criticism of LLM coding in that they casually spit out *way* too much code per query at arbitrary complexity, pretending there is no stage 2. Getting that much code is bad and scary. Instead, the LLM has to actively work with you to break down problems into little incremental steps, each more easily verifiable. It has to anticipate the computational work of (2) and reduce it as much as possible. It has to really care.
This leads me to probably the biggest misunderstanding non-coders have about coding. They think that coding is about writing the code (1). It's not. It's about staring at the code (2). Loading it all into your working memory. Pacing back and forth. Thinking through all the edge cases. If you catch me at a random point while I'm "programming", I'm probably just staring at the screen and, if interrupted, really mad because it is so computationally strenuous. If we only get much faster 1, but we don't also reduce 2 (which is most of the time!), then clearly the overall speed of coding won't improve (see Amdahl's law).
AI PROMPTING → AI VERIFYING
AI prompting scales, because prompting is just typing.
But AI verifying doesn’t scale, because verifying AI output involves much more than just typing.
Sometimes you can verify by eye, which is why AI is great for frontend, images, and video. But for anything subtle, you need to read the code or text deeply — and that means knowing the topic well enough to correct the AI.
Researchers are well aware of this, which is why there’s so much work on evals and hallucination.
However, the concept of verification as the bottleneck for AI users is under-discussed. Yes, you can try formal verification, or critic models where one AI checks another, or other techniques. But to even be aware of the issue as a first class problem is half the battle.
For users: AI verifying is as important as AI prompting.
Fully intact Chinese PL-15 long range air-to-air missile recovered from Hoshiarpur, Punjab. Obviously fired from a PAF jet, likely a JF-17—Failed to detonate.
Yesterday HQ-9B AD system gets smoked in Lahore by an 🇮🇳Harop, now this. Tells you a bit about Chinese weapons quality🥲
@satyanadella Oh wow, Microsofts Majorana 1 finally, it will probably be pre-loaded with Clippy, Solitaire, and a dozen unskippable Windows updates to slow it down to regular CPU speeds. Cant wait for the quantum blue screen of death 😂 with all the bloatware
I helped Orange launch the first Windows Mobile phone in 2002, the SPV. I worried MSFT might end up controlling mobile like they did the PC, I wanted something more open. So, sorry Bill, you're more responsible for loosing the $400B than you realize
https://t.co/xuWu7paunM
I literally helped create Android to prevent Microsoft from controlling the phone the way they did the PC - stifling innovation. So it's always funny for me to hear Gates whine about losing mobile to Android 🤣
Retweet if you think I should share our original pitch deck
1/2
If you're not going to listen to me then atleast listen to the master. If you're a student or an academic dreaming of making LLMs for Indian Languages, stop wasting your time. You're not going to make it. Instead focus on more fundamental and focused problems like:
1. How to get LLMs to do a specific set of things. Niche problems rather than general capabilities.
2. Solving the data problem for LLMs. (IndicLLM Suite)
3. Solving the issue of transfer learning from English. This will need some fundamental analysis of how LLMs see multilingualism. (RomanSetu)
4. Efficiency. This is so damn important.
5. How to incorporate external knowledge and memories into LLMs.
But for god's sake, if you don't have both tons of compute and extremely talented hands, don't waste time, talent and money thinking that you will be the one to put out the next big LLM for Indian languages. Wasting all that time and compute to get a 7B LLM to the same few shot classification performance as a 200M parameter Bert, but with terrible performance for actual generative tasks only proves that you don't know what you are doing. Indian languages based NLP is extremely important but it's going to mean anything if we do it right and for that we need to solve fundamental issues.
No more IndicLLaMas pls.
When's the last time you got an error so absurd it makes you feel like you know less than 1% of web dev?
Today, mine was "Tainted canvases may not be exported"
The fact that sealed bottled gardens can last for decades with trivial effort makes it hard for me to credit the idea that space colony ecosystems will be extremely difficult to sustain. https://t.co/nKGAFmQYdz
"If someone hates me—that’s their problem. Mine—to be patient and cheerful with everyone, including them. Ready to show them their mistake. Not spitefully, or to show off my own self-control, but in an honest, upright way. ” — Marcus Aurelius, Meditations