Aidan Homewood

Technical Governance @AISecurityInst
Formerly @GovAIOrg
🇳🇿

[Xander Davies @alxndrdavies]

At @AISecurityInst, we tested the cybersecurity safeguards on GPT-5.6 Sol. In all rounds of testing, we found universal jailbreaks that allowed for long-form agentic task completion in domains like vulnerability discovery and exploit development. 🧵

[Toby Ord @tobyordoxford]

Excellent post from @AISecurityInst on how AI agents' time horizons scale with the number of tokens they're allowed.
For me the most interesting point is the first graph below. Note how it is requiring much more than 10x the tokens to get 10x the time horizon…

[Marie Davidsen Buhl @MarieBassBuhl]

Very excited about our new paper on neglected risks of automated alignment research (aka getting AI systems to figure out how to make other AI systems safe). Some of my main takeaways in 🧵

[AI Security Institute (AISI) @AISecurityInst]

We conducted cyber evaluations of Claude Mythos Preview and found that it is the first model to complete an AISI cyber range end-to-end. 🧵

[kamilė @kamilelukosiute]

AI models' cyber capabilities keep getting meaningfully better, and fast. To determine how AI capabilities will impact cybercrime, we first need a baseline for global cybercrime damages.

In a new @GovAIOrg technical report with John Halstead and @lucafrighetti, we arrive at a baseline estimate of global cybercrime damages: $500B (with 90% CI of $100B-$1T) per year.

Existing estimates of global cybercrime damages range from tens of billions to tens of trillions of dollars. Most have serious problems: they rely on reported damages only (missing the vast majority of incidents that go unreported), or they don't publish their methodology at all. We tried to do better by extrapolating mostly from survey data, which captures unreported incidents, and by being transparent about every assumption we make.

Our total estimate: ~$500B a year. This includes direct losses to individuals, direct + response costs to businesses, and defensive spending. Notably, this does not include costs that are even harder to quantify, such as IP theft, espionage, and national security costs, so the real yearly damages are presumably higher.

As AI gets better at cyber, even a modest additive effect on the volume of cybercrime is a big deal. A 20% increase would mean ~$100B in additional yearly damages.

Our estimates have extremely high uncertainty ranges. If we want to understand how AI is shaping cybercrime, we'll need to build new ways of measure the effects by looking at real world indicators of threat actor AI usage.

Read the full report here: https://t.co/ldM53aJqhX

[kamilė @kamilelukosiute]

New post on what open source means for cyber misuse: nothing good. The best open-weight models are about 3–6 months behind frontier closed models on cyber. That gap is close to the threshold where models become operationally useful for attackers.

Once open-weight models cross that line, attackers will prefer them. Even a slightly worse model is worth it if it means not handing evidence of criminal activity to a US company that can be subpoenaed.

In light of the distillation news from the past week from Anthropic (idk if anyone at this point remember that particular bit of Anthropic news...), it's worth noting that anti-distillation might be of the highest-leverage cybersecurity interventions available right now.

In any case, the lab-based safety paradigm buys time, but not much. We need to be investing in societal-level defences that work regardless of where the model lives.

[Michael L. Chen @miclchen]

Here’s what’s new in Anthropic’s Responsible Scaling Policy, version 3:

- No more implication of unilateral commitment to pause AI development and deployment in relevant conditions
- Public roadmaps of safety and security goals
- Risk reports and third-party review
- Advocacy of industry-wide safety standards based on safety cases

[Active Site @ActiveSiteBio]

We ran a randomized controlled trial to see if LLMs can help novices perform molecular biology in a wet-lab.

The results: LLMs may help in some aspects, but we found no significant increase at the core tasks end-to-end. That's lower than what experts predicted.

Our findings 🧵

[Miles Brundage @Miles_Brundage]

If anyone really wants to fill out an apology form today, the obvious places to look are Paul Christiano (re: slow takeoff, distributed capabilities/deployment causing various weirdnesses, basically everything else...) and Alan Chan (re: urgency of agent infrastructure).

[Scott Singer (宋杰) @Scott_R_Singer]

A year ago, the Western world was stunned when DeepSeek released its R1 model.

Chinese AI is going to stun the world again. Only the next time, it won't be at chatbot. It will be AI-powered robotics.

For @washingtonpost, Pavlo Z. and I explain why China is winning the embodied AI race. 🧵

[Miles Brundage @Miles_Brundage]

The world if frontier AI auditing were effective and universal

[Aidan Homewood @adnhw]

Audits are probably important for making AI go well. Excited to have co-authored this paper, and to see @AVERIorg launch! 🚀

[AVERI @AVERIorg]

AVERI’s goal is to make third-party auditing of frontier AI effective and universal.

Today, together with coauthors at dozens of organizations we set out our vision in "Frontier AI Auditing" — a framework for rigorous third-party audits of safety and security practices at leading AI companies.

[Aidan Homewood @adnhw]

@joodalooped @grok is this true

[Markus Anderljung @Manderljung]

Is it cheaper to build AI models in the UAE than in the US?

The answer seems to be... no. Total costs are roughly comparable to the US.

So why are Microsoft, AWS, and OpenAI investing billions there?

New GovAI technical report from @amelia__michael

[Luca Righetti @lucafrighetti]

There's been intense debate about AI and lone-wolf bioterrorism. How worried should we actually be?

My new report tackles this question—investigating the threat model, why experts disagree, and proposing a concrete framework for risk assessment.

[Alan Chan @_achan96_]

New GovAI policy brief!

How should we deal with the risks that AI agents could pose to information ecosystems? Article 50 of the EU AI Act offers one possibility.

[Dylan HadfieldMenell @dhadfieldmenell]

This is an important point: marginal risk makes sense for comparing AI systems to non-AI systems, but if we compare to competitor AI systems it’s easy for risk to compound across releases.

[Jonas Freund @jonasfreund_]

Frontier AI companies increasingly assess how much their models increase risk compared to competitors’ models.

In our new policy brief, we argue that this particular approach to marginal risk is problematic.

🔗 Learn more: https://t.co/YccNS81t9K

[Aidan Homewood @adnhw]

@_S_Williams @NoemiDreksler @Manderljung @jonasfreund_ Read our summary and the paper here: https://t.co/ib08C2f1y5

[Aidan Homewood @adnhw]

New policy brief from GovAI:

AI companies taking a “marginal risk” approach might lead to inaccurate risk assessments. But even when done well, we could still see safety standards erode.

[Aidan Homewood @adnhw]

@_S_Williams led this work. @NoemiDreksler @manderljung @jonasfreund_ and I co-authored.