chrispyroberts

Building AI Bug Hunters @ Cantina, Computational Physics @ CMU

[chrispyroberts @chrispyprojects]

The 3 minute talk I gave at @Stanford AI Cybersecurity conference.

If you want to know what the data is telling us about the direction of AI Cybersecurity tooling, check this out, maybe you'll learn something : )

Had to hand draw these graphs because slides weren't allowed.

[chrispyroberts @chrispyprojects]

If I had a billion dollars, first thing I would do is get OpenAI’s Dota 2 AI gym, port the architecture to transformers from LSTMs, and train it on as many GPUs as I can get, then use it to go from top 5k NA to top 1 NA.

[chrispyroberts @chrispyprojects]

Guys, Anthropic's expensive Opus 4.8 router is back!!!!

[chrispyroberts @chrispyprojects]

IMO you are probably fine to test on public competitions. Sample efficiency learning in frontier LLMs is orders of magnitude worse than in humans, and I think just because the code was in the train set and had it's gradient passed over once (or multiple times if it's in a fine-tune/curriculum dataset) doesn't necessarily mean the results are testing recall over bug finding. A true test would be two independent benchmarks, one in-distribution and the other OOD. But I agree that if you want to guaranteed avoid this issue you can just test on recent/private competitions/audits.

[Recon @getreconxyz]

AI Audit, WTF am I buying?

Here's my RANT on why AI Audit benchmarks don't mean much beside being an indicator of a race to the bottom.

And an invitation to use Coverage Classes as a way to concretely show the work done by auditors, agents and tools as a means to make shipping code with 0 bugs a concrete reality.

[chrispyroberts @chrispyprojects]

I do like your idea of coverage classes, and this concept is (at a high level) used inside of our AI Audits. I only hope this promotes more discussion and questions in the industry because the future here is very uncertain. Anybody's guess about what it looks like and the correct way to perform an AI Audit could be right.

[chrispyroberts @chrispyprojects]

@getreconxyz Great video, and thanks for taking the time to read through and discuss the blog I wrote! The higher-level purpose of it is to promote visibility, credibility and discussion amongst security researchers building their own versions of Apex. I do have some clarifying points though:

[chrispyroberts @chrispyprojects]

3) Maybe I went to far with some of my arguments, or they weren't worded properly, but the key statement I was trying to get across, which I think you agree with based on what you said in the beginning of the video, is that you can get the same/better performance (measured across 8 audits) on coverage of bugs for significantly less $$$ (and way faster) compared to what you would get in the past with a team of human researchers, and it's fully AI end-to-end.

[chrispyroberts @chrispyprojects]

2) The blog actually proposes an answer on how much money you should burn on an AI Audit, and it's measured by predicting the saturation of scans - which we find we can reliably measure across any AI Audit. Some codebases are massive, and require significantly more tokens (compute) to exhaustively cover, and this is captured by the saturating nature of the AI Audit, while some are a few thousand lines of code, and don't require $30k of tokens to find all the bugs.

[chrispyroberts @chrispyprojects]

1) When it comes to the severity distribution slicing, I think the conclusion you draw isn't fully aligned with what the data is saying, or at least not with the point I was making. The severity slicing is aggregated over ALL 1600 scans. The variance is insane on a per-scan basis, sometimes all the crits come in the first 10%, sometimes in the last 10%. This points to the conclusion that at any point in the scan, a critical can come. It's non-deterministic. The point I made in the blog is that it tells you that it is worth spending more because the next $ of compute could find a critical.

[chrispyroberts @chrispyprojects]

@sayinshallah

[chrispyroberts @chrispyprojects]

@0xTomass @pashov check my blog on compute scaling laws for ai bug hunters, definitely applies here and looks like you'll need to engineer a more efficiently scaling harness if you want all the bugs at a reasonable price point

[chrispyroberts @chrispyprojects]

@Bharg4v @cantinasecurity thank you! let's play dota 2 sometime ;)

[chrispyroberts @chrispyprojects]

@GeorgeKarr52692 @cantinasecurity GEEOOOOOO

[chrispyroberts @chrispyprojects]

I think it’s very hard to defend that Web3/Blockchain Security Audits are not solved by Autonomous AI Bug Hunters like Cantina’s Apex.

Not only are we #1 on the HackerOne US business leaderboard, but we also (a very small team) used Apex to farm nearly $1M in bounties in the span of a few months, abusing the free money machine while competition was scarce.

We took every scan we’ve ever run (1500+) and looked carefully at the data and found some interesting trends we think are worth sharing with everyone.

1. # of valid bugs scales log-linearly with compute across 3 orders of magnitude. Double the compute and you get ~40% more findings on the median scan.

2. Scans saturate. There is a finite number of bugs in a codebase, and we can consistently predict once we’ve hit the ceiling of Apex’s capability to find more bugs in a codebase.

3. The indeterministic properties of AI bug hunters goes away as you scale compute. Apex more reliably converges on the same set of findings as you spend more compute.

4. The data supports that audits that would normally cost more than half a million dollars and take months, can be completed in a few days for a few thousand bucks, end-to-end by AI products. Solving the coverage problem took engineering an efficiently scaling harness. We are able to hit superhuman capabilities in our benchmarks for entirely automated security audits and the economics of it are ridiculous. For 1/100th, or in some case 1/1000th the cost, you get the same/better performance than a set of expert researchers.

Moving forward, we are seeing a new trend that is potentially scary. As more security researchers integrate AI more deeply into their workflow, at what point does hiring a researcher just mean you’re hiring their custom built harness? We’ve run scans side by side with recent audits comparing results after, and often see that the researcher findings are a subset of Apex findings. It’s very difficult now to measure the set of out-of-distributions bugs because of AI. Maybe manual hunting can make a comeback if your particular style is difficult to replace by AI.

Check out the full blog here: https://t.co/8xrqE7ckNc

[Anthropic @AnthropicAI]

The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.

The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.

Access to all other Claude models is not affected.

We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.

Read our full statement: https://t.co/bwn0sximKZ

[chrispyroberts @chrispyprojects]

@armaniferrante doesn't work.

[Hari @hrkrshnn]

1. Labeling a product by its worst users is a dangerous slippery slope. Where does that line get drawn? Like for Ethereum, Tornado Cash and the internet itself.

2. Security professionals have something close to a Hippocratic Oath. In this case, anyone can now burn tokens to find out the remaining bugs given the information that an AI tool has found more bugs. This is now doing harm to any user of thorchain who now have money at risk.

3. Threatening to publicly disclose live bugs unless a bounty is paid is extortion. It's also a sensitive time with many crypto companies running out of money, laying off staff and shutting down their bounty programs.

[chrispyroberts @chrispyprojects]

@adisingh @agentcardai this is honestly such good marketing, and kind of acts as a bug bounty too lmao

[chrispyroberts @chrispyprojects]

@Michal_Archacki @laurensnys nope https://t.co/5G5ZEgsQdJ