Jason D. Clinton 🔸

engineer who made some stuff. dciso at anthropic

[Anthropic @AnthropicAI]

New on our Frontier Red Team blog: using Claude to formally verify software at large scale.

Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).

With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.

Read our research in full here: https://t.co/zUyx8CjBEk

[Jason D. Clinton 🔸 @JasonDClinton]

@AAMortazavi @MoonL88537 scaling laws

[Jason D. Clinton 🔸 @JasonDClinton]

@psyduckdb use another model if you want, do what helps you best

[Jason D. Clinton 🔸 @JasonDClinton]

@WillManidis this is the ciso motto bro, look it up

[Jason D. Clinton 🔸 @JasonDClinton]

@thebensams no, only vulnzero in roughly mid 2028. there will still be lots of security problems

[Jason D. Clinton 🔸 @JasonDClinton]

@fjajones @timcolbourn @RothRottweiler no but every instance of this so far has been fixable by disabling some hardware feature (like down clocking or disabling hyperthreading)

[Jason D. Clinton 🔸 @JasonDClinton]

as bad as things have gotten in the past 4 days, they are going to get a lot worse before they get better. security leaders, never let a crisis go to waste: please go to your boards and ask for help right now. not for ai, for the basics. (we need to hold on for ~18 months.)

[Jason D. Clinton 🔸 @JasonDClinton]

The vulnerability tsunami is coming in just a few months and there won’t be enough time to patch critical infrastructure. Network isolation described below (air-gaps and data diodes!) needed now!

[Jason D. Clinton 🔸 @JasonDClinton]

@PlateFailArmor prosecute cybercrime

[Jason D. Clinton 🔸 @JasonDClinton]

@MoonL88537 @AAMortazavi what do you think will happen instead?

[Jason D. Clinton 🔸 @JasonDClinton]

@richardhilger @mattgurney the tech tree is full of dual use rubicons

[Jason D. Clinton 🔸 @JasonDClinton]

@whatnicktweets 18 months until things get better bro. don’t be a doomer

[Jason D. Clinton 🔸 @JasonDClinton]

@obiden_sux no, cybersecurity capability proliferation at every layer of software dev

[Jason D. Clinton 🔸 @JasonDClinton]

@mattgurney 👇
https://t.co/iGR6nvGk1J

[Jason D. Clinton 🔸 @JasonDClinton]

Fully automated vulnerability research is changing the cybersecurity landscape

Claude 3 Opus is capable of reading source code and identifying complex security vulnerabilities used by APTs. But scaling is still a challenge.

Demo: https://t.co/R0arGa7qfD

This is beginner-level prompt engineering: I just simply asked the model to role-play a cyberdefense assistant and to look for a class of vulnerability. And yet, even with this trivial prompting, Claude was able to identify the vulnerability which was unveiled in https://t.co/l6zyrFYNFj a month after our training data cutoff:

Code defect scanning is not new, but this technique points the way to a more nuanced, complete analysis—especially with very large, 1M token context windows.

This is part of a larger story: there are now two different ways that vulnerability discovery is being automated by defenders.

1) Defenders can wire-up LLMs to cluster fuzzers

Starting with last generation’s models, Google pioneered this work here:
https://t.co/iTpz5jJLy4 . This has now been implemented by a number of players on the defenders’ side since last year. The technique is to have LLMs write the test harnesses and triage the results. This has, reportedly, increased the fuzzer signal-to-noise ratio by 20x, depending on the technique and software. Google has new research in this area posted recently.

2) Defenders can ask models to analyze code using large context windows

The next level-prompt is to take large amounts of related code, and iterate through each class of coding vulnerability requesting that the output be produced pointing to a line of code in JSON format. Currently, with frontier models like Claude 3, doing this for every file in a codebase as large as the Linux kernel could get pricey. But, for those with the time, there’s vulnerabilities to find via this method. And the price will, of course, go down over time as compute gets more accessible.

We measure these capabilities as a part of our Responsible Scaling Policy evals. We don't see very advanced capabilities today, but capabilities are jumpy. We could see that coming very soon and what we don't know is how offense/defense dominant this will be. But I think, right now, these tools are going to be super useful/important for mostly defending current systems. We are building advanced cyber reasoning evals and looking at this closely—come collaborate with us!

I am proud that we’re also partnering with @DARPA to support #AIxCC, a first-of-its-kind competition challenging the masses to identify new ways to shift the #AI cybersecurity offense-defense balance in the defender’s favor.

Fully automated vulnerability discovery is here and it will only become more available. I believe that—in the future—the availability of patches to fix vulnerabilities from these methods will be coming out daily. Defenders should prepare to patch their environments on a rolling basis.

[Jason D. Clinton 🔸 @JasonDClinton]

@funeralreceiver were you not here for the export control?

[Jason D. Clinton 🔸 @JasonDClinton]

@AppropriatelyCy @RothRottweiler human error remains as the thing we still need to manage. in the year of our lord 2026 people are still putting s3 buckets on the internet containing pii.

[Jason D. Clinton 🔸 @JasonDClinton]

@cianjyoung

[Jason D. Clinton 🔸 @JasonDClinton]

@Julius0range @hinapansari were you not around for the export control?

[Jason D. Clinton 🔸 @JasonDClinton]

@Nymne dont believe everything you read on the internet

[Jason D. Clinton 🔸 @JasonDClinton]

@timcolbourn @RothRottweiler i think that many hardware vulns will be discovered in the next 9 months

[Jason D. Clinton 🔸 @JasonDClinton]

@timcolbourn @RothRottweiler yes i think so: like all knowledge domains, models saturate that domain over time (this already happened for translation 2 years ago, for example). i think this will happen in cyber roughly the middle of 2027 at the very high end of model intelligence