Jonathan Wang ⨇

formal verification for software @Intrinsic_HQ | zk @axiom_xyz

[Jonathan Wang ⨇ @jonathanpwang]

Excited to announce OpenVM, a new performant and modular zkVM framework built for customization and extensibility. It was very fun contributing to the initial design. This will be the future of zkVM design — user-led customization without forking the core architecture.

[OpenVM @openvm_org]

Announcing OpenVM, an open-source zkVM framework prioritizing customization and extensibility at every level of the stack.

OpenVM achieves this with a novel modular design allowing developers to extend the ISA, ZK circuits, and proof system within one unified framework.

[Jonathan Wang ⨇ @jonathanpwang]

This level of formal verification was previously impossible, but now multi-agent systems can prove end to end security of software. We wrote and reviewed the specs to ensure correctness and human understandability; the agents proved the rest.

[Intrinsic Technologies @Intrinsic_HQ]

At Intrinsic, we apply AI to scale formal verification for software.

As a first result, we end-to-end formally verified @openvm_org, a production zero-knowledge proof system using @leanprover. Our verification shows OpenVM's cryptography is sound and matches its implementation.

[Tony Feng @tonylfeng]

@aaswaminathan01 Great job!! For a possible next project: an important recent result which is notoriously difficult to referee is the proof of the Geometric Langlands Conjecture.

[Yi Sun @theyisun]

At @axiom_xyz we’ve been building RL environments to help AI models do FV at the binary level and accelerate defensive efforts like these. Our environments are based on real code, scalable, and challenge even frontier models. Get in touch if you are interested!

[Axiom @axiom_xyz]

We are excited to announce the public devnet release of Lighter EVM, built with @Lighter_xyz using @openvm_org.

Lighter EVM is an EVM-equivalent rollup for composing general-purpose applications with Lighter's perps and spot markets.

[srikar @SrikarVaradaraj]

We need a taxonomy of *minds*, so we can simulate societies of minds to understand emergent properties, by drawing from this distribution.
We should be careful to design minds whose emergent properties, at scale, have positive traits from our vantage point.

[Boaz Barak @boazbaraktcs]

Mathematicians should also watch this. I’ve had a wonderful time chiseling proofs by hand, but folks need to accept that era is over and embrace the new one of exploring math with AI.

[Daniel Kokotajlo @DKokotajlo]

Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share:

Dan Selsam's Personal Statement on AI Risk:

I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods.

Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk.

The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail.

I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues.

I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here.

That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase.

Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways.

It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace.

The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing.
But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence:

[Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them.

[Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals.

These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans.

If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong.

One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for.

Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason).

Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance.

In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek.

I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering
implications. I do not have answers, but as a first step, I wanted to share my present concerns.

Daniel Selsam
September 14, 2026

Link to original doc: https://t.co/TxMNr0vhrL

[Jonathan Wang ⨇ @jonathanpwang]

The most reasonable take

[Tony Feng @tonylfeng]

The resolution of Navier-Stokes is an amazing accomplishment. Some comments …

First, a brief mathematical postmortem. Millennium problems are important, but when you confront an unsolved problem, it is always unclear how far the solution is out of reach. The initial opinion of at least some experts (not myself – I have no expertise on this problem) seems to be that, in retrospect, the conceptual program underlying this solution happened to already be in place*, and the contribution of AI was a very intricate technical execution. Technical execution is one of the axes on which we've already seen AI become superhuman, though never before with such a dramatic demonstration.

This is not to diminish the solution or OpenAI’s achievement. But there are many problems which are out of reach of current models, no matter how much inference-time compute one invests. I’m still convinced that human mathematicians are laying indispensable groundwork for future progress.

*Terry Tao describes one possible program at https://t.co/IjGmfNS4YB. As I understand, the one Tao describes is NOT the Cordoba--Martınez-Zoroa program that OpenAI (and Alpoge/Buckmaster) followed. However, the successful approach was not as esoteric as Buckmaster’s statement might make it seem – there were certainly other people (not me) also prompting that approach; it just isn’t the kind of thing that one goes around saying publicly.

As far as I know, Navier–Stokes is the only one among all Millenium problems with a well-defined program towards a solution. Concretely, for the Birch and Swinnerton-Dyer Conjecture (over number fields) I am not aware of even a vague proposal of what a proof might look like, so I doubt that current AI models will get anywhere close to a resolution. (My opinion is that this conjecture cannot be resolved in the negative; perhaps the most refined formulation is false, but then the real problem would be to correct the formulation and then prove it.) I'd be curious to see what progress was made on it!

[prinz @deredleritt3r]

In December 2024, o3-preview scored 87.5% on ARC-AGI-1. This run cost >$4,500 per task.

In July 2026, DeepSeek-V4-Flash scored 87.0% on ARC-AGI-1. This run cost $0.021 per task.

Please realize that all OpenAI has done is bring forward the AI-generated solution to Navier-Stokes by maybe around a year or two. In all likelihood, by 2028, we will have open-source flash models capable of cracking it in under an hour. There will be no way to ban or police this. The profession of mathematics will have to change.

[Tony Feng @tonylfeng]

I don't repost mathematics if I do not personally feel sure of its correctness, so I did not yet comment on the purported construction of a complex structure on S^6. However, I wanted to add some contextualization because I don't agree with the social media narrative that is developing.

0. Of the AI breakthroughs so far it's the closest to my interests, although still pretty far. I am not grieving, I am interested to learn more about it! I don't know if it's correct but on cursory glance it seems quite plausible.

1. If correct, this would be a major development. I believe it is a central problem in its field (complex differential geometry), like the Unit Distance Conjecture was for combinatorial geometry. Perhaps even more famous names are attached to this field, leading some people to say it's even more prestigious, but I prefer not to think in such terms.

2. There are two writeups, a 2-page one and a 100+-page one. Both are terrible and seem deliberately constructed to be inhumane. Fortunately, one can fight AI with AI: I asked Claude/GPT/Gemini to collaborate on making a more readable and enlightening version, which you can find at https://t.co/7dKpFyQ0vZ. To make this, I prompted the models to write a document in the style of one of my own surveys. I haven't checked the mathematics, but it seems less forbidding than the original document at least.

3. This is a great opportunity for an expert to write an illuminating account of the construction, explain the key ideas and their proper historical context, and summarize the takeaways and lessons learned for the mathematics community.

4. If I see such an account, I would be happy to arrange for the author to give a lecture series in Berkeley. If this lecture series is also wonderful, I would go further and advocate for the author to have a position at Berkeley (if desired). For contrast, I see little value in having colleagues who are good at being the first to prompt AI for a solution.

5. There are some intriguing differences with the previous artifacts from Claude. The announcement seems coy about the provenance of the solution - there is a vague allusion to Claude, but which model(s) did it come from, and what was the human involvement? Why is it hosted on a personal website instead of the Anthropic one? We have seen some instances of AI results that are so good, the human authors seem to begrudge credit. I keep a list of such instances. The standard should be to disclose the role of AI clearly and transparently. Better to be honest than go through life under a cloud of suspicion.

6. Although this may solve one of the more famous problems of mathematics, there are still whole fields that are only just starting to feel acceleration. I'm not aware of any AI breakthroughs in my areas of research, for example. When those come, as I suspect they will before long, I look forward to studying the insights!

[Axiom @axiom_xyz]

We are excited to announce that we have formally verified @openvm_org's Keccak and SHA-2 circuits using @leanprover, proving that each circuit constrains exactly the hashing algorithm specified by @NIST for every possible input.

Here's what we proved and why we did it:

[Jonathan Wang ⨇ @jonathanpwang]

@Steve_Yegge Nvm it already is https://t.co/46j3DgAvsx

[Jonathan Wang ⨇ @jonathanpwang]

@Steve_Yegge Please make this into a webcomic series

[Tony Feng @tonylfeng]

This is a nice result, but to be clear it represents 0% progress towards actually resolving the Riemann Hypothesis.

[Steve Yegge @Steve_Yegge]

Something I noticed about working with Fable...

[Jonathan Wang ⨇ @jonathanpwang]

What

[Jimmy Apples 🍎/acc @apples_jimmy]

“ Refining this idea through
further discussion with ChatGPT eventually led us to the discovery of Theorem “

Looking forward to the day when we have big results in the health and science domains as well

[Jonathan Wang ⨇ @jonathanpwang]

iykyk

[Mario Zechner @badlogicgames]

if i have to read the word seams one more time, i'll quit software and become a carpenter.

[OpenVM @openvm_org]

We are excited to release OpenVM 2.1 Preview, an early look at the performance of the next version of OpenVM.

It adds 64-bit RISC-V support, adopts a redesigned execution, and now proves mainnet Ethereum blocks in real time on only 4 5090 GPUs, a 2x improvement over OpenVM 2.0.

[ayush @0xa9a]

We rewrote openvm execution (again!) and it’s ridiculously fast

More details in my blog:
https://t.co/TsUTWFyZFD

[Tony Feng @tonylfeng]

I've seen this sentiment from several mathematicians, and it is understandable, but I feel it is also dangerously wrong. I say this because I think mathematicians need to adapt hard and fast, instead of wishing that the disruption from AI will somehow go away.

There is a science to large language models and it predicts that with the linear scaling rate we are seeing now, capabilities will increase over the next 5 years at a similar rate as the last 5 years. To get a sense of what that means, one of the main benchmarks 5 years ago was GSM8K, whose questions you can see for yourself at https://t.co/fNyx6xUVse.

There also seems to be a misconception that frontier labs are emphasizing math right now, and spending a lot on solving open math problems, in order to advertise the intelligence of their models. There was some truth to this 7-10 months ago, but that time has passed and most labs are now focused on coding agents. The continued improvements in math are largely a side effect.

In the past, the path to a math Ph.D. cultivated resilience, resourcefulness, critical thinking, and a healthy skepticism. These qualities are still needed today, perhaps more than ever, but suddenly you can produce a passable Ph.D. thesis with the push of a button. This signals that the field needs to completely overhaul itself, starting now.

My recommendation is that the future role of humans in mathematics, and similar theoretical fields, should be preserving and disseminating knowledge in oral and social forms; written papers will be the domain of AI. Over the coming months I will be involved in workshops that aim to produce more concrete proposals. But people need to come to terms ASAP with the fact that drastic change and compromise are needed to prevent the disenfranchisement of humanity from knowledge work.