New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: https://t.co/zUyx8CjBEk
AI-assisted research is killing embargoes. Coordinated disclosure will never be the same.
The industry used to have the luxury of testing or sitting on a fix for months. Not anymore. When everyone's using the same tools and the same intelligence to find vulnerabilities, you're not the only one finding that bug.
Product security and PSIRT need a serious rethink: less gatekeeping, more proactive evaluation of security posture, and shipping fixes continuously at scale.
The biggest impact will be in firmware and hardware, where disclosure timelines have historically stretched past 12 months. That mindset is a leftover from outdated processes and slow testing and bring-up environments.
@Rolezn I guess both trends are visible at scale. I've personally run into this a few times with my own disclosures, where the bug was independently rediscovered by other people, possibly using similar AI-assisted tools.
@Cyburgerim OSS vuln disclosure was always kinda broken, but AI takes it to a whole new level. Every fix in a popular OSS component now opens a real exposure window for everyone downstream.
And honestly, the target name + bug class + a bit of context is enough to /goal the rediscovery.
Reminder: It has been 16 years since Stuxnet was found -- an autonomous self-replicating worm designed to spread across multiple platforms and sabotage nuclear enrichment equipment.
If small models might soon be able to copy themselves around that's not a step-change.
We made an anime about SBOMs (with a lot of help from @claudeai). Then shipped v0.3.0 🎉
CBOMs now get checked against CNSA 2.0 + post-quantum rules, and if sbom-tools can't tell what an algorithm is.
https://t.co/CN073co9yJ
Exciting work from @willzhou97 (Fellow, now full time!) and @bucketofkets on the Frontier Red Team. One of the cool parts of the job is seeing a future world early. We think this one is ~pretty underpriced.
"We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay."
To secure open-source software, we’re launching OSS Scanner.
We’ll use our frontier models to periodically scan opted-in open-source projects for vulnerabilities, at no cost. Our reports will provide a proof-of-concept, explanation, and suggested fix.
https://t.co/UFFrLUd25U
Agree on verification, but I'd push it further. Memory safety is a symptom, not the cause. We got here because of design decisions and trade-offs we made all along, and you can see it in pretty much every recent Linux kernel disclosure.
Throwing more tokens at the attack surface won't save us either. Nothing we have today is ready for AI-driven vuln research at this scale, and finding and fixing bugs with AI doesn't change that.
What we actually need is redesign and architecture validation with AI, formal methods where we can, and not accepting attack surface at the design phase in the first place. That's just not how we build software today. Patching known bugs doesn't touch the root cause, and hardware has the same problem.
Security will become a larger and larger function in software companies. Security is verification engineering (eg: “my code is probably memory-safe”), as well as capital allocation (“what surface should I throw most tokens at?”)
For startups, it’s a challenge and opportunity. As the world’s paranoia grows given ever-more-sophisticated AI adversaries, how can you trust a 2-person-and-a-dog company?
But conversely, given the dire state of global cybersecurity and increasing reliance on centralization, as always, there’re vast areas that small teams can now disrupt that I’m excited to invest in.
We’re expanding our Cyber Verification Program to give security professionals broader access to our most capable models.
Through this program, verified security professionals can access Claude Mythos 5.1, Opus 5.5, and Sonnet 5.5 with safeguards designed for defensive work.
We’re also opening up new tiers to allow for authorized offensive work, like penetration testing and red-teaming.
https://t.co/blaJtvxhJh
I wouldn't treat it as a silver bullet, but it does help in complex systems and gives strong guarantees for the critical parts. What seems realistic to me is formally verifying the places where hardware-driven specs connect with software layers like hypervisors. That said, AI is making formal verification much more scalable, easing the human bottleneck that used to limit it.
@msuiche It feels like everything's been broken for decades, and we've just been hunting different attack surfaces because of the complexity. I don't think software as complex as KVM is fixable. It needs to be redesigned from scratch and formally verified. This bug-fix loop is infinite.
@dinodaizovi That's quite a thing, I bumped into this in my recent disclosures too. But seriously, just the name of the target and the type of bug, plus a little context, is enough to /goal the rediscovery, specifically in the OSS ecosystem.
Yeah, the seams are the part that worries me. Verification works well inside each block, but bootROM + configurable policy + third-party IP is where the assumptions quietly stop lining up. Curious if you think formal methods ever reach across those boundaries.
Totally agree on tokenmaxxing too. Burst-mode vuln hunting with a frontier model loses to an attacker grinding away with a cheaper one 24/7 on a narrow slice of your attack surface.