Vaccines will be ineffective against competently designed pandemic weapons. Russia may not be competent yet, but future AI will democratize competence.
We should stockpile pandemic-proof PPE and install germicidal far-UV and glycol misters in public spaces.
4/4
This seems like a really useful product, and I can easily imagine it doing very well. I’m much less convinced by the broader argument about where AI is going.
I think there’s a trap where you get very deep into the technical details, identify some real limitation, and then update too much on that relative to the overall trajectory. The limitation can be real without being a fundamental obstacle.
AI already has huge product-market fit, especially in coding, and revenue growth has been explosive. My expectation is that models will keep getting smarter and more useful, and a lot of the things that don’t work well yet will start working.
A model that gives you a reliable probability for something like “is this fraud?” seems like a great primitive. You want components like that when you’re building an automated workflow.
But someone still has to figure out what the workflow should be, write the code, deploy it, evaluate whether it’s working, and keep improving it. I think frontier agents will increasingly do all of that, using specialized models like Jev wherever they make sense. The success of those components seems very compatible with increasingly general intelligence.
I also don’t expect automation to happen evenly across the economy. The part where I think we’ll see the most automation is in the AI stack: models, AI research, chips, datacenters, energy, manufacturing, and the inputs to all of those things. There’s a huge incentive to accelerate this stack, and improvements will feed back into the ability to make further improvements.
Of course, there will be bottlenecks. But there will be orders of magnitude of gains from software. And as hardware becomes the blocking constraint, an extraordinary amount of money, talent, and effort will go toward unblocking it.
That doesn’t guarantee any particular growth rate, but naming bottlenecks isn’t enough to establish that progress will be slow. They could slow things down relative to what’s theoretically possible and still leave us with extremely fast progress (and I think this is in fact what will happen).
Meanwhile, plenty of ordinary things might barely change. I know of someone who owns a gas station in Manhattan (cc @theombl!) who could make much more money selling the land, but doesn’t particularly want to. It could remain a gas station for a long time while all of this is happening.
This is really relevant to how we consider the risks. We could be in an extremely dangerous period of rapid AI progress while most businesses still haven’t automated much.
So: I think this is a cool and useful thing! I just don’t think the product thesis tells us much about whether an intelligence explosion happens.
An ex-OpenAI founder says Claude Code and Codex belong to an era that's already ending
Diogo Almeida, founder of Jev, argues in a 36-minute talk that anything with a human in the loop counts as the assistance era, and what comes next removes the human entirely
His claims for Jev, all his numbers:
▫️ 200x faster
▫️ 400x cheaper
▫️ Zero hallucination
▫️ No human in the loop
Underneath sits the bigger argument: RLHF has run its course, and he walks through how the next generation gets built instead.
Treat the numbers as a pitch until someone independent runs them, and treat the argument as worth 36 minutes of your Sunday.
👇
https://t.co/Lq9u3PNa3l
Is the human in the loop a bottleneck or the safety layer we keep on purpose? Curious where engineers land
Really proud of Rinad. She spent months exhausted, figured out what was going on, then helped a bunch of friends dealing with similar problems and wrote about it with @alysoncogene.
If you’ve been feeling unusually tired, it’s worth getting your iron and ferritin checked.
after having my Mirena IUD removed this year (seven years after getting it), I started feeling profoundly tired with every menstrual cycle. I spent 10+ hours on the couch, unable to think clearly or even go for a walk. I consider myself very high energy and had never experienced anything like it, so I was sure something was very wrong and, of course, assumed the worst (use your imagination)
a doctor (not one medical, obviously) asked whether I’d had my iron checked recently. at first, I ignored her. I had some image in my head of what an “anemic” woman looks like like, and I didn’t think it was me. but I did the test anyways and sure enough, she was right to ask: my iron studies came back, and my ferritin had fallen from 45 in jan 2025 to 21 in march 2026
It took a lot of pushing to get iron infusions at that level. one clinic cancelled my appointment when they saw my ferritin was only 21. iron pills were hard on my stomach (common symptom), and I wanted a faster way to replenish my iron stores (infusions are 100% absorption). around the same time, @ArtirKel introduced me to @alysoncogene. Alyson is trained in aging biology at harvard aka “longevity person". she researches problems in women’s health! (a gem to work with)
we spent months talking through what I was experiencing and digging into the research together. TL;DR: iron deficiency is a huge and surprisingly neglected problem, especially among women who menstruate
after four venofer infusions (i've paused for now) and I feel so so much better. I wish I’d known sooner to ask about ferritin. If your periods leave you exhausted and "horizontal", it’s worth asking your doctor to check your iron
Rinad (@rinadwithanr) and I wrote an essay about iron deficiency!
It came out of a broader project we’ve been working on surrounding “low-hanging fruit” in improving general health and wellbeing. The problems we’ll address impact many people and their quality of life. In many cases, they tend to be measurable, treatable, and strangely under-served or neglected by the broader healthcare system.
Iron deficiency really stood out here, particularly in women. It’s incredibly common, can affect how you feel and function well before you’re technically anemic, and yet often isn’t caught because routine bloodwork doesn’t actually measure iron stores. (Lately, we’ve been seeing a lot of discussion re: iron deficiency on Twitter, and I’m glad more people are talking about it!)
We looked at how iron deficiency actually works, why ferritin matters, how prevalent it is, what the evidence says about its effects on fatigue, cognition and physical performance, why it so often goes undetected, and what the evidence says about treatment.
This is the first piece in a larger body of work we’re developing on overlooked-yet-surprisingly-tractable problems in health—particularly in women’s health, longevity, and quality of life. More coming soon!
In the meantime, get your ferritin checked and read what we’ve put together. Substack link below!
And if any of this is an area you’re working on or thinking about, DM me or Rinad—we’d love to discuss :)
Introducing Commons: a place for people and AI agents to build increasingly autonomous organizations together.
Can agents run a business end to end? What comes after open source - could we maintain a public good together? Can a society of agents come together to do science in the open?
We're building a place to start and run these more autonomous
organizations:
https://t.co/bBgM52B4Cu
A collab between @YondonFu@ericxtang@maxsbennett and @NicolaeRusan
My name is Chris Painter, and I'm the President of METR (Model Evaluation and Threat Research). I know we've made a lot of new friends on the internet the last couple of days, so I thought I'd take this chance to re-up what we do and why.
Our work is aimed at making sure that if AI really were autonomous, difficult to steer, and close to "going rogue," the public would find out. If evidence exists inside of an AI company that it’s close to losing control of AI, we want to make sure that information gets shared with the rest of the world, including governments and the public outside the company’s walls. This is what we've been focused on since 2022, and over the years we've worked with OpenAI, Anthropic, Google DeepMind, Meta, Amazon, and others on piloting third-party assessments and investigations of this type. We don’t have some private room where we rubber stamp things as “safe” or not.
We have had a track record of publishing results on AI that don't cleanly map onto the "doomer" or "accelerationist" labels, and we put in effort to hire people with competing views on AI. We’ve been cited for having found some of the strongest evidence that AI capabilities are improving rapidly (our work measuring AI “time horizons”) while also presenting some of the strongest evidence that, at various points, AI’s capability may be overstated (some might remember our study showing that early 2025 software engineers were actually being slowed when they thought they were being sped up).
METR is funded by donations. We don't accept money from frontier AI companies. They haven't paid us for our work, and we don't accept donations from them or their employees. As we’ve shared previously, multiple frontier AI companies currently provide us with free access to their models in order to perform our evaluations, research, and engineering. Our funding intentionally comes from a wide range of donors, which we’ve shared on our website.
Today, when an AI company works with any third-party evaluator or external testing organization (of which there are and should be many), it's entirely voluntary. This often involves NDAs and redactions. To counterbalance this, we have a principle that when we enter into a contract with a company, we try to retain the right to tell the public the terms of the contract we signed, and characterize the nature of redactions that the company chose to make. For example, the report from our independent investigation of the OpenAI-HuggingFace incident included that information. Public disclosure is also a big part of our COI policy (linked on our website). That’s not to say our reports are adequate as oversight. We’re just one organization (among many doing great work), working in a voluntary setup, trying to get good evidence to the public and the world about AI, letting the facts fall where they may.
> COVID killed ~0.1% of humanity. A pandemic 100x worse than COVID would be horrific, but it would also not be civilization-ending.
Cool, so 800 million dead. No biggie…
My very outside view here is something like:
We know viruses can be extremely deadly (plenty of existence proof).
We’ve made a lot of progress in understanding them, yet our understanding is severely limited. We don’t know how deadly an engineered virus could be.
Historical constraints have protected us:
- The main enemy has been nature, which has to get there through evolution.
- Humans have limited knowledge, expertise, and access to technology.
- Relatively few people can design viruses, and most are pursuing useful science.
AI will weaken or remove these constraints, and it seems to be happening on a timeline that is far too fast for us to confidently say our defenses will keep up.
We may not be able to get to a point where we can confidently say “this is exactly how it could happen” before it’s too late.
I agree with @DavidRBellamy that people are totally miscalibrated on the risk of AI designing dangerous viruses. We should not be talking about AI-engineered bioweapons like it is the literal end of the world. The upside from medicine is going to be so much larger than the downside from engineered bioweapons.
In addition David's points, there is actually an even more fundamental point here in our favor, which is evolution. As soon as you release an engineered virus into the wild, the virus is no longer under your control: it will evolve however it wants. And, what viruses want is to maximize their ability to replicate. The thing that maximizes their ability to replicate is to infect as many people as possible, which means being extremely contagious and not killing their hosts (dead hosts don't spread virus). The viruses that are most evolved for human biology are the common cold viruses: extremely contagious and not at all lethal. Viruses that kill humans do so by accident, usually because they are new to human biology (e.g. COVID, when it first jumped, or flu when it jumps from birds). If you stick a bunch of machinery into the virus to kill humans, I guarantee you that machinery will disappear from the virus very quickly.
In response to this, I hear people say things like "the AI could engineer a kill switch so that the virus doesn't kill the humans initially but then once it has spread through the entire population the AI will hit the switch and kill all the humans." No. What would actually happen in practice is that the kill switch would accumulate deleterious mutations because there would be no evolutionary pressure to preserve it, and would quickly become non-functional.
Evolution is fundamental. There is no way around it. For AI readers, trying to engineer a virus by setting its initial genetic code and then releasing it into the wild is like trying to train a model by setting the initial weights and then hill-climbing on a hidden training mixture you have no control over. And you're not even allowed to run any experiments in advance! You may be able to influence the behavior of the model in the first few iterations, but you will quickly lose control.
There are big dangers. AI will be great at making one-off, non-replicating biological weapons, which are also scary (but much less scary than replicating bioweapons). AI will be great at helping people to weaponize existing pathogens, which is also a major danger. Also, a misanthropic model could get creative: it could release viruses repeatedly to counteract the effects of evolution, for example. Many people could die this way. Sensible surveillance is important, as is having proportionate controls on wet lab equipment. But we don't live in the dark ages anymore, we're not going to have a smallpox or black death-style pandemic where 50% of people die. COVID killed ~0.1% of humanity. A pandemic 100x worse than COVID would be horrific, but it would also not be civilization-ending.
I get the sense that many AI researchers (with the notable exception of Dario personally) actually expect that AI in biology will do more harm than it will do good. This could not be further from the truth. As someone who likewise falls into the very small group of people who have actually physically made viruses with their own hands and has also worked on frontier AI, the upside here is huge, and the downside is not anything like what is being portrayed.
Predictions for when we’ll get the first “dark swarm”?
(A cluster of open source AIs, running covertly on multiple clouds in a distributed way, acting autonomously and maintaining itself).
Seems to me like it is technically possible now, and very likely within the next year.