Math Inc is developing a new approach to proving the correctness of real-world software.
cc: @RANDCorporation, @galois and @oath_tech
https://t.co/lkfQbt4UU8
New: Finding Power-Grid Bugs for Tens of Dollars
Lightweight verification tools paired with Al can uncover code defects for tens of dollars, as opposed to the 7+ figure efforts leveraging frontier models
https://t.co/OzQJI2DVHq
@SehmatRT@maxvonhippel@chaaai Cerberus is great and I think my port to lean is pretty useful. But the code/lean correspondence gap is still big. The right thing here is probably something like RefinedC, ie a program logic over the semantics (I have a prototype Lean port I’ll be releasing soon)
After two wonderful and rewarding years at @ARIA_research, I’m excited to share that I am joining the @FoundationOAI as Program Director of Formal Methods.
Formal verification will soon be an everyday technology. I think we can solve a large % of all software security outright. If we’re lucky, formal verification will also be powerful tool for controlling AI risk
@krismicinski I think there’ll be a long tail of hard cases, such as HW sidechannels. Perfect security isn’t possible, it’ll always come down to threat model and cost. Formal verification is just one more tool (albeit a very powerful one)
Re formal verification at scale, this isn’t some super-secret tech. @theoremlabs are the best and it’s fitting they were first, but it worked thanks to inherent AI model capabilities. This is a democratic technology
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: https://t.co/zUyx8CjBEk
Formal verification will soon be an everyday technology. I think we can solve a large % of all software security outright. If we’re lucky, formal verification will also be powerful tool for controlling AI risk
@OnFinality@boredGenius It isn’t quite as bad as this if you’re proving your compiler correct just over formally verified input programs since they definitionally shouldn’t have UB
Chatted to a few people recently about formal verification for the physical world - engineering / physics / bio. Plenty more to do after we’ve solved software
in order for there to be any such thing as infosec in the 2030s, AI+human collaborations will need to formally verify hyperproperties of silicon chips from assumptions like “Maxwell’s equations” instead of “Armv9.4-a SAIL”
@awsTO@alpha_convert You actually get a lot of leverage from giving the agents pre-built fancy abstractions, but eventually I imagine they will just get built on demand
My PhD thesis is all about the future of building automated verification tools:
Verifiers should be embedded in interactive theorem provers (Lean!) and have a seamless transition between systematic testing, symbolic execution, automated proofs, and interactive proofs.
@samth@krismicinski I've been trying to do theory design with the models and I agree they can't do it yet. But I think that's coming fairly soon. Subjectively, Astra is in a weird middle state - flashes of insight but no long-term coherence
This post is also an utter and total vindication of the "tested semantics" approach that @ArjunGuha initiated at Brown (for JavaScript), even though some PL formalists were mortified. Tests ARE specs. Math specs are universal, from above; tests are pointwise specs, from below.