Yesterday, I posted a GitHub discussion in the OpenShell repository about some research directions one of my students is considering. It got two encouraging comments, so at first I was excited! But both comments are AI. Wish someone at Nvidia would look!
https://t.co/TDZ6qc8wWR
New blog post: On the Unreasonable Effectiveness of Property-Based Testing for Validating Formal Specifications.
https://t.co/Lfrjyao3sY
The gist: randomised testing can validate formal specs. It's very cheap and powerful: we found bugs in specs of VERINA and CLEVER benchmarks.
Rota, Dyson, Gowers, etc, wrote about problem-solving vs. theory-building (with recognition that most math contains both). This is old hat to mathematicians, but non-mathematicians might find it useful to read, to broaden their understanding of what mathematicians do and math is.
Yesterday, John Morgan and I completed our “Covering Spaces Project”, whose goal was to formally (in Lean @leanprover, of course) prove the Fundamental Theorem of Algebra (that polynomials have complex roots) using only continuity: covering spaces, trivializations, and winding numbers, instead of the Mathlib proof that goes through complex analysis / Liouville.
We started the morning about 30% through the proof, and tried playing with the Codex VS code extension (running GPT-5.4, extra high reasoning). We had already written a complete natural language blueprint (last June, at the Simons @SimonsFdn Lean workshop) and were meeting for about an hour a month, leisurely working through the details by hand (John wanted to learn the process). We’d used AI before (mostly Claude) to help move things along, but this was the first time that we met after I turned on Codex.
I started by asking for Codex to give me just the formal *statement* corresponding to our next unfinished leaf in the dependency graph. It thought for 5 mins, and gave what looked like a reasonable statement. So I said, ok, can you now give a formal proof? It thought for 10 mins, and came back with a full proof, including helper lemmas. I asked it to add natural language around the helper lemmas so we’d see what they’re doing in the blueprint. It thought for 5 mins, and did it. We went on to the next statement, then the next proof, and it got those too.
It continued like this for an hour or so, and we jumped to about 60% done with the project, amazing progress! I had to run to get on a train to DC (for the DARPA meeting). Once in my seat, I decided on a whim to just tell Codex to keep working on the whole file and get as far as it could, stopping to ask me for help if it got stuck. I left VS Code running in the background, while working on other things. After an hour, I remembered to check back on what progress Codex had made.
It was done. And so was the project!
Now I’m having Codex go through the whole proof all over again, remove all our bespoke definitions and statements, and make it clean and Mathlib-ready. Many more hours of iteration later, and it’s ready to go as a PR.
Wild wild times we’re living in!
AI is writing a growing share of the world's software. No one is formally verifying any of it.
New essay: "When AI Writes the World's Software, Who Verifies It?"
https://t.co/8zjS9FkdA8
Here's what I actually got in the yearbook, which was on display at the reunion. I showed up and had the same conversation with everyone about how, the PhD I had just quit at the ten year reunion, I eventually finished ten years later.
I used to do this too, but I would talk about math. And it wasn't in a coffee shop. And I came to suspect they weren't really my friends. It was fun, though, when they were interested in more than just getting the answers to homework. Though 90% was just answering homework.