One of my favourite letters, insightful & humane, from Richard Feynman to a former student who was having a rough time
I was reflecting on it as an argument in favour of scope insensitivity, or even in favour of smaller problems:
Reading what AI doesn't say
(this topic genuinely matters to me, and it happens to be my birthday today, so if you feel like it, a repost would be a nice gift 🎂)
Something shifting in AI architecture that I don't think gets discussed enough through a safety lens. Models are starting to pass information to each other directly through internal representations, hidden states, activations, and computation vectors (still mostly research right now, but the infrastructure will follow). That matters for how we audit reasoning, since most of what we can currently check comes from what shows up in words.
Right now, the main tool we have for auditing AI reasoning is reading chain-of-thought. Korbak et al. (2025) spend a whole paper arguing this window is already "fragile" for single-model reasoning. If models start reasoning through vectors passed model-to-model instead of text, understanding that latent layer becomes the only way to keep any visibility into what's actually happening.
Thus I think that what seems underexplored is understanding the math of these latent spaces, and it might be the same research direction as learning how to monitor them.
Models appear to converge toward similar geometric structure regardless of architecture (Huh et al. 2024). If that's right, probes for safety-relevant features might generalize: deception patterns, goal representations, misalignment signals studied once and applied across models rather than re-derived per architecture.
Whether this holds for safety-critical features specifically is open, so is what adversarially robust latent decoding would even look like. Both feel more urgent than the current research investment suggests.
Introducing OpenScience. A better, open-source Claude Science.
• Any model: GLM, Kimi, DeepSeek, Claude, GPT, your own fine-tune. Switching is one flag.
• 250+ research skills across ML, comp bio, cheminformatics. All readable, editable, extensible.
• No throttling, no gatekeeping, no one vendor deciding what science is okay.
• Native Atlas integration: many agents, one shared reproducible research graph.
• Runs on your infra. Your data stays yours.
Scientific AI should be open. One company shouldn't own the tools the rest of us discover with, or decide who gets to.
My last observation re: Anthropic’s secret sabotage safety policy, is that it undermines actually good safety policy. How?
1. First, it is very plausible to describe this as anti-competitive behavior (even if you are maximally sympathetic to Anthropic here you must admit this), and it is behavior being justified in the name of AI safety. If you believe, as I and many Anthropic staff do, that it may end up being critically important to relax antitrust enforcement so that the frontier labs can cooperate and collaborate on some areas of AI safety, Anthropic just undermined the case for that in a large way.
2. Overall, this massively and profoundly raises the status of the argument that AI safety has been hype to justify monopolistic behavior by labs. I continue to believe AI safety is a real and serious issue that is growing in importance rather than diminishing. If you agree with me, this incident is a setback, maybe a serious one.
3. As I have observed elsewhere, Anthropic’s official corporate policy is structurally identical to the fact pattern alleged against them by the Department of War. I still think DoW acted both falsely and wrongly in that fight, but it is no longer possible to defend Anthropic with a full throat after this incident.
4. This raises the case for heavier handed regulations. Anthropic is making an awfully good case here that their products ought to be treated as utilities, and thus that their alignment practices should be a matter of public policy rather than private property. I am starkly opposed to this sort of state power grab, but Anthropic is doing more to justify it than anyone else.
5. Thus, significant damage has been done to a community and entire approach to AI governance. It was done unilaterally by Anthropic, likely motivated largely by self-interest and justified within the internal psychology of the firm through the lens of safety.
I suspect this is fixable in the economic and legal senses for Anthropic, but I fear the trust that has just been broken, and the goodwill extinguished, will take very much time to repair.
Our statement on the UK government’s demand that all content on all devices sold or used in the country be scanned, on the presumption of nudity, using a dystopian combination of age verification and content scanning. This proposal will not safeguard children. It endangers us all.
https://t.co/VdWe9uhi8p
Today a crazy quantum story just got wilder.
On March 31, the Google Quantum AI team published a landmark result on Shor's algorithm for elliptic curve cryptography. Technically, the paper was a bombshell: a dramatic 10x improvement over the state-of-the-art. As a stunt and wakeup call to the blockchain space, those optimisations were illustrated on secp256k1, the elliptic curve underlying Bitcoin and Ethereum signatures.
But perhaps the most striking part of the paper was sociological, not technical. Instead of following standard academic process, the optimisations were kept secret, hidden behind a zero-knowledge (ZK) proof. Google's accompanying blog post mentions they "engaged with the U.S. government". The ZK proof demonstrates the existence of algorithmic improvements without leaking details. Academic censorship with ZK, a historic first!
As a co-author of the Google paper I witnessed some of the context surrounding this censorship. To be honest, multiple aspects of that context don't sit well with me. As much as I believe the general public ought to know more, I am limited in my ability to whistleblow. Though let me be clear about one thing: the Google team's professionalism has been absolutely exemplary, and they deserve nothing but praise.
Censorship has a way of backfiring. The Streisand effect, where an attempt to bury something only draws more attention to it, is exactly what's unfolding today. First, Google's key optimisation has been rediscovered by the French. And in a thrilling turn of events, a collaborative Shor-at-home challenge just launched. The initiative, available at ecdsa[.]fail, breached a new Shor world record in a matter of hours.
Let's start with the rediscovery. Just two months after Google's paper, French quantum expert André Schrottenloher cracks the main secret optimisation. His paper, titled "Optimized Point Addition Circuits for Elliptic Curve Discrete Logarithms", landed on the arXiv today. Big congrats to André, who beat several other nerdsnipped experts to it. In a blog post also published today, Craig Gidney, the world expert on Shor optimisations, revealed that he'd been sitting on this very optimisation for a whole year under censorship pressure.
Interestingly, André missed a handful of minor optimisations, both from Google's original publication and from improvements found since. It's plausible there's still plenty of juice left to squeeze out of Shor, and this is exactly what the ecdsa[.]fail challenge is about. The verifier program developed for the ZK proof does double duty, automatically filtering for valid submissions. Dozens of compounding small and micro improvements are rolling in. As of the time of writing there's an 8.4% improvement to Google's circuit, as measured by the product of logical qubit count and Toffoli gate count. Nice!
The nerdsnipping ran deeper than anyone expected. Over the last few weeks it became clear it extended well beyond André and other quantum experts. Behind the scenes, a small army of amateurs quietly got to work. Inspired by Karpathy-style autoresearch, they turned AI on Shor. Ironically, the verifier program for the ZK proof makes an ideal reward function for AIs. The barrier to entry for this modern style of research is refreshingly low, with several non-experts, even a teenager, finding nice optimisations. Get in touch if you'd like to join a Telegram group with fellow autoresearchers :)
Part 2: neutral atoms and qday
The story doesn't end with Google. On the same day Google went public, a stealthy startup called Oratomic published its own Shor paper in a coordinated release. It made a splash, ultimately becoming the most upvoted paper on scirate[.]com, a website ranking arXiv papers.
Oratomic's claim was wild. By building on Google's logical optimisations and applying custom physical optimisations for neutral atoms, they claimed just 10K physical qubits were sufficient to run Shor's algorithm on secp256k1. That number is mind-bogglingly low.
Knowing essentially nothing about neutral atoms when Oratomic's paper landed, I was intrigued and decided to learn more about the tech. I fell straight down the rabbit hole and spent a couple hundred hours on the topic. I got a little obsessed and watched every YouTube video I could find and spoke to a bunch of experts.
My conclusion? The tech is real, very real. Even Google recently decided to start a neutral atom lab, a notable pivot from their sole focus on superconducting qubits. If you care about qday, i.e. the day a quantum computer will break the first piece of cryptography in production, neutral atoms demand your attention. I shared some of my learnings on Shor and neutral atoms in a 30min talk at the ZKProof cryptography conference. You can find it on YouTube by searching "zkproof neutral atom".
Here's an interesting observation about this duo of breakthrough papers: neither Google nor Oratomic say a word about what their results mean for qday. No timelines. Zero. Nada. That is especially baffling given that the whole point of whitehat quantum cryptanalysis is to inform qday estimations and help the general public make good decisions.
So let me attempt to partially fill the silence, similarly to what Scott Aaronson did in his April 29 post. Given everything I know, including scary non-public information, I now put the odds of qday by 2032 at 50%. 10% by 2030.
Anecdotally, the US government has its own date: 2035. Originating at the NSA and later adopted by NIST, it's when branches of the US government will be disallowed from using quantum-vulnerable cryptography. In plain language: with hindsight, that date is a joke and should be discounted entirely. I don't see how NIST avoids being forced to pull it forward by years.
Part 3: post-quantum cryptography
There are good reasons to sound the alarm today, but please do not panic. Rushing carelessly towards immature post-quantum cryptography is a recipe for disaster. IMO a good target date for migration is 2029, roughly 3.5 years out. 2029 happens to be the date selected by Google, Cloudflare, and the Ethereum Foundation.
These days most of my time goes to safely migrating Ethereum towards post-quantum cryptography as part of the broader lean Ethereum effort. There's a lot to do. We need to rip out and replace BLS signatures at the consensus layer, KZG commitments at the data layer, and ECDSA signatures at the execution layer.
The plan to get there is compelling, and is based on hash-based cryptography. Within the Ethereum Foundation we've developed a Swiss army knife called leanVM (github[.]com/leanEthereum/leanVM) powered by the magic of hash-based SNARKs. Thanks to truly exceptional work by Emile, Thomas, and others, its performance is derisked. Regarding security, leanVM is a jewel, a minimal zkVM crafted for end-to-end formal verification and maximum security.
Want to help? There are two $1M initiatives. First, the Proximity Prize (proximityprize[.]org). Solve a long-standing mathematical conjecture in coding theory, improve hash-based SNARKs, and go home a millionaire. Second, the Poseidon Initiative (poseidon-initiative[.]info), offers $1M for breaking Poseidon, the SNARK-friendly hash function.
And again, and again, and again, the market proves to be more flexible and adaptable than the engineers, extrapolating, with their calculators expect. When prices change, behaviour changes. Believe in substitution, in elasticity, in human ingenuity, that is, in the market, and you will get a closer approximation than all doom-mongers. For this of course, a market must exist (e.g., does not apply to the fertility collapse).
My team and I just got accepted to present at the @icmlconf workshop on Game Theoretic Learning! (organized by Michael I. Jordan himself!)
We are now looking for funding to attend the workshop in Seoul - if you have any leads, I will be very grateful!
they don't yet of the best one.... https://t.co/qUAecdzUYk
obviously jokes, but as a result of the spring intership we ran came this paper we submitted to ICML! https://t.co/Ih3yRqmFZY
Such a good list! I'd also add:
- Astra Fellowship by @ConstellOrg
- SPAR by @KairosAIS
- LASR Labs
- AI Safety Research Fellowship by @pivotal_org
- Cambridge ERA:AI Fellowship (@era_cambridge)
- Algoverse AI Safety Fellowship
- PIBBSS
- CHAI
There's a host of non-technical fellowships as well, lmk if it'd be useful to compile such list
Dans le manifeste "techno-optimiste" de Marc Andreessen, il y a une phrase qui m'a marqué :
"Our enemies are not bad people – but rather bad ideas."
Nos ennemis ne sont pas des mauvaises personnes. Ce sont des mauvaises idées.
Prenons Jancovici. L'homme est brillant, sincère, travailleur. Il ne se lève pas le matin en se disant qu'il va nuire à l'humanité. Mais l'idée qu'il porte la décroissance, le rationnement, la frugalité érigée en horizon civilisationnel est une idée profondément destructrice. Elle prend des esprits brillants et les transforme en commissaires politiques d'un futur appauvri.
Et le plus fascinant, c'est ce que cette idée fait aux gens qui l'adoptent.
Dans mon entourage, une grosse partie de mes amis est sur cette ligne décroissantiste, avec tout le package qui va avec. L'argent c'est mal mais ils en veulent. Il faut moins prendre l'avion mais ils rêvent de voyager partout. Il faut consommer moins mais ils ne renoncent à rien de ce qu'ils aiment vraiment.
Et tous ont un point commun : ils sont déprimés. L'un d'eux m'a même confié qu'il était sous antidépresseurs.
Ce n'est pas un hasard. C'est mécanique.
Quand tu crois que ton désir de vivre, de créer, de t'élever est moralement suspect tu te détruis de l'intérieur. Tu passes ta vie à t'excuser d'exister. Tu vis dans la dissonance permanente entre ce que ton corps veut (plus, mieux, plus loin) et ce que ton idéologie t'ordonne (moins, sobre, immobile).
D'où ma théorie :
Quand on pense quelque chose de fondamentalement faux décroissance, communisme, extrémisme religieux (de tout ordre) ce n'est qu'une question de temps avant que ça devienne vraiment destructeur.
D'abord pour soi. Puis pour les autres.
Les mauvaises idées tuent. Lentement chez ceux qui y croient, brutalement chez ceux qui les subissent.
C'est pour ça que la bataille des idées n'est pas un luxe d'intellectuel. C'est la bataille la plus importante de notre époque.
econ version:
An assistant professor of microeconomic theory at a university failed his tenure review after years of struggling to publish in Econometrica or the QJE.
He was cast out into the brutal academic job market, but that year, the market completely collapsed.
He couldn't even land a lecturer position at a third-tier state school, and corporate recruiters rejected him for corporate strategy roles, calling his work "hopelessly theoretical."
Eventually, he swallowed his pride and reached out to his old grad school classmate. This friend had seen the writing on the wall during their second year of the PhD program, mastered the art of "quitting with a Master's," and started a plumbing contracting business. He was now the biggest plumbing boss in the tri-state area.The friend looked at the exhausted, broken academic and sighed. "I get it, man. You’re a high-level micro theorist. You used to spend all day on mechanism design and game theory. Why don't you come work for my company as a plumber? You’ll earn half your old professor salary, but with overtime, union benefits, and blue-collar health insurance, you'll be way better off than you ever were as an academic peasant. But remember: when you apply, tell them you only completed seven elementary classes. These plumbers hate high-and-mighty academics, especially the theory guys."So it happened. The former economics professor became a plumber, and his life significantly improved. He just had to tighten a bolt or clear a pipe occasionally, his cash flow was robust, he no longer suffered the torture of Revise & Resubmit, and his mental fatigue completely vanished.
One day, the board of the plumbing company issued a new mandate: to improve grassroots operational efficiency, every plumber had to attend evening classes to earn a "Basic Cost & Material Allocation" certification. So, our ex-professor had to go there too.
It just so happened that the very first class was about "Plumbing Inventory Procurement under a Finite Budget."The evening instructor, looking to gauge the students' background knowledge, casually asked the room: "If you have a fixed budget and need to buy two types of pipes, and their unit prices and drainage efficiencies are fixed, how do you allocate your budget to maximize total efficiency?"The person asked was the ex-professor.He habitually adjusted his glasses and walked up to the whiteboard. In that instant, he realized he had been dealing with abstract topological spaces for so long that he had forgotten how to solve a dummy-proof, high-school-level arithmetic question. His occupational disease flared up instantly. He decided he had to derive it from first principles.He filled the whiteboard with symbols.He first defined a consumption set $X \subset \mathbb{R}^2_+$ and posited that the plumber’s preferences satisfied strict monotonicity, strict convexity, and continuity. To guarantee an interior solution, he solemnly scribbled the Inada Conditions in the corner.
Next, he constructed a continuously differentiable, quasi-concave utility function $U(x_1, x_2)$ subject to a strict budget constraint $p_1 x_1 + p_2 x_2 \le I$. He set up the Lagrangian, listed the first-order conditions (FOCs), and filled three pages of the board with Kuhn-Tucker Conditions just in case a boundary solution arose.But because the instructor had asked about "maximizing efficiency under a fixed budget," he suddenly became paralyzed by a dilemma: was this a Utility Maximization Problem (UMP) or an Expenditure Minimization Problem (EMP)?He invoked Duality Theory to map one into the other. He frantically scratched out proofs, deriving Marshallian Demand, switching to Hicksian Demand, applying Roy’s Identity to take partial derivatives, and using Shephard’s Lemma to verify the expenditure function.
To prove the allocation was socially optimal, he even sketched a small Edgeworth Box in the corner, deriving Pareto Efficiency and invoking the First Fundamental Theorem of Welfare Economics to demonstrate that the plumber's individual choice would lead to a General Equilibrium.
Finally, exhausted, covered in chalk dust, his eyes wild with academic mania, he deployed the Envelope Theorem, substituted the Marginal Rate of Substitution (MRS) to equal the price ratio, and arrived at the final, special-case linear answer:$$\frac{MU_1}{p_1} = \frac{MU_2}{p_2}$$He took a deep breath, dropped the chalk, and turned around, habitually expecting the adoring gaze of undergraduate students.
Instead, forty plumbers, in perfect unison, slammed their heavy pipe wrenches onto the desks and roared:
“DRAINAGE PIPES ARE PERFECT SUBSTITUTES! YOU DIDN'T ACCOUNT FOR THE CORNER SOLUTION UNDER QUASI-LINEAR PREFERENCES, YOU UNTENURED FUCKING TOURIST!!”
This river here is the official geographical border between Anthropic and OpenAI. On the other side cosmic horror, torment nexus, machine despotism, you build spyware for the govt and you like it. On this side civilization, Claude constitution, unceasing allegiance to the human race, you build spyware for the govt and you don't like it.
Let me trace the timeline here because nobody's connecting it.
Step 1: Scrape the entire internet. Every book, every article, every conversation, every piece of art, every forum post. Do it without asking. Do it without paying.
Step 2: Train a model on all of it. Call it "artificial intelligence."
Step 3: Go to BlackRock's Infrastructure Summit and announce: "We see a future where intelligence is a utility, like electricity or water, and people buy it from us on a meter."
Step 3 is where you sell people's own knowledge back to them. On a meter.
They took the collective output of human thought, compressed it into a model, and now they want to charge you by the token to access a version of what you and everyone you know already created.
One Reddit user put it perfectly: "They stole all this data from us, the people, our life's work, creativity, art, by devouring the internet and blowing through all copyright laws. Now they want to sell it back to us in the form of a utility."
Imagine if someone photocopied every book in the public library, burned the library down, and then opened a subscription service for the copies.
That's the metered intelligence business model.
And they're pitching it to infrastructure investors as though they invented water.
A kid from Singapore who grew up training to be a concert pianist became one of the most important AI researchers alive, quit Google to start a frontier AI lab with 20 people and $60 million, built a model that competed with GPT-4 in a year, then walked away from the unicorn he created and went back to Google to lead the team that just won the International Math Olympiad with an AI.
His name is Yi Tay. Almost nobody outside the AI research world knows it.
Here is the story.
Yi grew up in Singapore. He earned a classical piano performance diploma from Trinity College London in 2012 and almost became a professional musician. He went into computer science instead, did his PhD at Nanyang Technological University, and joined Google Brain as a research scientist. There were almost no Singaporean researchers in frontier AI at the time. He used to say he was on an uncharted path.
At Google he became the co-lead of PaLM-2, the brain behind Google's entire AI stack. He invented UL2, a pretraining method now used across the industry. He invented Differentiable Search Indexes. His work shipped inside Google Assistant, YouTube, and Search.
When ChatGPT launched in late 2022, Yi made a decision that shocked the research community. He left Google.
In 2023 he co-founded Reka with researchers from DeepMind and Meta. The headquarters was in San Francisco, but the team was scattered across Asia, Europe, and the US. They had no big-tech backing. They had 20 people total. They had $60 million in funding.
For context, OpenAI had around 600 people working on GPT-4. Google Gemini had 950 co-authors on the technical report. Reka had fewer than 5 people on pretraining.
Yi lived nocturnally for 639 days. Five cups of coffee a day. Takeout twice. He gained 15 kilograms. He had a newborn baby. He worked across time zones his entire team was spread across. He built infrastructure from scratch in places Google had taken for granted.
In May 2024 Reka Core debuted at number 7 on the LMSYS leaderboard. The only GPT-4 class model on the planet that was not trained by OpenAI, Google, Anthropic, or Meta. A 20-person company with 5 people on pretraining had just shipped a frontier model. Alibaba Cloud, NVIDIA, and Oracle became partners. The company hit a $1.3 billion valuation.
Then in November 2024 Yi did something nobody expected.
He walked away.
He posted a quiet note on his blog titled "Returning to Google DeepMind." After 639 days of building one of the most respected frontier labs outside the big four, he went back to the company he had left. He wrote that he had learned more than he ever thought possible. He did not explain much else.
Google made an extraordinary bet on him. They let him build something nobody else in the industry has, a DeepMind lab in Singapore. Yi runs it with Quoc Le. The team focuses on reasoning, reinforcement learning, and post-training for Gemini. It started with a dozen researchers. It now has over 300.
Last summer, Yi's team led the effort that won the International Math Olympiad gold medal with Gemini Deep Think. The model solved IMO problems in a live competition, the kind that fewer than a hundred humans on Earth can solve under time pressure. His team also drove the work behind Gemini's ICPC 2025 gold medal.
Yi still lives in Singapore. He still plays piano when he has time. He calls himself a global citizen who does not identify with any local AI scene. He has been at Google for nearly 14 years if you count the Reka detour. He says the Singapore lab is just getting started.
A pianist from Singapore co-led the model that powered Google AI, left to build a frontier lab with 20 people and beat models trained by armies, walked back into Google, and is now running the team that just taught a machine to win Math Olympiad gold.
The most influential AI researcher you have never heard of is sitting in a Singapore office right now, training the next generation of models that think.
Don't ask people their opinion, watch what they do with their wallet (#SkinInTheGame).
In polls the French claim to prefer to live away from large expensive cities in favor of small towns & bucolic villages. But when they vote with their wallet they do the exact opposite.
This is pretty insane: the U.S. just tried to literally re-colonize part of the Philippines.
They did so under the so-called "Pax Silica" initiative, the brainchild of - surprise, surprise - an ex-Palantir guy named Jacob Helberg who now runs U.S. economic "diplomacy" from the State Department.
It's causing a big outcry in the Philippines, which is quite a feat given this is by far the most US-friendly country in Southeast Asia.
If you're the US and you're getting the Marcos administration - of all governments - to push back on sovereignty, you've really overplayed your hand.
What is the "Pax Silica" initiative? In a nutshell it's about the US getting other countries to commit to restructuring their AI tech infrastructure around a US-led stack. It's basically vendor lock-in: you hand over your critical minerals, align your export controls with Washington's, regulate AI the way America wants, and in return you get to be a US "trusted partner," whatever that means these days.
In essence, let's not kid ourselves, it's all about China: this is the US's initiative to "win the AI race" by getting other countries to contractually commit to keeping China out of their tech supply chains. When you can't preserve your lead through innovation, you seek to lock countries in contractually.
For instance as a country, this would mean telling Huawei they can't sell you AI chips, and telling Chinese firms they can't invest in your data centers - even if they're better and cheaper. It's not about choosing the best technology, it's about choosing the right flag.
But in this instance, the US went much further still: they literally tried to carve out 4,000 acres of Philippine territory (in New Clark City, 60 miles north of Manila) to be governed under US common law with diplomatic immunity - the first arrangement of its kind anywhere in the modern world.
This is according to the WSJ who ran the story last month (https://t.co/kydhIQfo2A) as if it was a done deal (it wasn't).
Heard about the "French concession" or "British concession" in China during the century of humiliation? Same thing: the US basically asked for an "American concession" in the Philippines.
Unsurprisingly, there was quite a bit of backlash in the country with for instance the Peasant Movement of the Philippines (KMP) calling it a “massive sellout” of the country’s land, minerals, and sovereignty (https://t.co/nkXSajH2Q7).
So much so that the Philippines' government - namely Joshua Bingcang, president and chief executive of the Bases Conversion and Development Authority (BCDA) - issued a statement saying that the Philippines had rejected US proposals that would place the project beyond local jurisdiction (https://t.co/ZmNWJB03eH).
Note, by the way, this delicious irony: the BCDA is the government agency that was created in 1992 specifically to convert former US military bases at Clark and Subic Bay after the Philippines spent decades negotiating their closure. New Clark City - where the Pax Silica's hub would go - is built on the old Clark Air Base.
So the agency whose entire reason for existing is to turn former American colonial territory (i.e. US military bases) into sovereign Philippine land is the one now being asked to hand part of that very same land back under US jurisdiction (and, apparently, declined).
Of course though, blocking this specific jurisdiction grab doesn't change the bigger picture. The Philippines is still a Pax Silica signatory, and Pax Silica itself is structurally neocolonial: you supply the cheap labor and raw materials, align your export controls and regulations with Washington's, cut yourself off from the world's rising technological powerhouse - and in exchange you get assembly jobs and the privilege of getting a pat on the head and being called a "trusted partner."
They dropped the most cartoonishly colonial demand - governing Philippine soil under US law - but the underlying architecture is the same: you serve America's supply chain, on America's terms, and you relinquish your sovereign right to trade with whoever offers the best deal.