There's no better time to pivot into AI governance than now (other than a few years ago)!
Apply for the UK or DC GovAI Summer fellowship, deadline Nov 22nd.
https://t.co/oEDy95TTMN
https://t.co/NQvSJRiQXa
I share my theory of a “partner run” with Joe Weisenthal @TheStalwart and @tracyalloway of @Bloomberg Odd Lots to explain why Weil Gotshal’s collapse was more predictable than it seemed. https://t.co/3DO8yBPAiK
Rapid AI progress has many people imagining a world where general-purpose AI models autonomously control military drones at scale. But there’s little analysis on what this would look like in practice, and how soon it might be feasible.
My latest paper at @GovAIOrg asks what it would take to make this vision a reality, and what the path there means for AI policy.
Read it here: https://t.co/eiTWhtRHRQ
The AI-enabled drones widely fielded today use narrow AI, not general-purpose models like LLMs. AI is typically only used to handle limited portions of a flight when the link to a pilot is broken, and these models are brittle in unpredictable conditions on real-world battlefields. Because of this, today’s drones still require significant pilot involvement to be most effective.
General-purpose AI models might make it possible to field autonomous systems that can receive plain language commands and make complex decisions in unpredictable environments without human control. Many people see frontier LLMs as a path toward this end - and rapid progress in robotics capabilities by frontier models like GPT-6 Astra and most recently Claude Fable 5.2 supports this case.
But two things currently stand in the way:
- The biggest and most powerful models are still not capable enough even in controlled environments without compute and latency constraints
- Military drones will have to operate “on the edge” in contested environments without consistent cloud compute access - which makes these constraints even more severe
If and when general-purpose AI plays a major role in drone autonomy, it’ll likely be through models that are far smaller than today’s frontier, and those that militaries can rapidly modify. And, notably - when it comes to these types of small open-weight models, China currently dominates.
This has a variety of implications for the US military, frontier AI developers, and US policymakers.
Thread below with my key takeaways and recommendations.
1/
Re point 1: once you accept that Chinese customers can switch to data centers in countries the rule doesn’t cover, you end up reinventing something like the diffusion rule, as @alasdairpr and I argued back when the administration repealed the rule. https://t.co/KJGwL8ZsQX
Commerce is reportedly considering new regulations to restrict China's access to AI chips for data centers overseas.
This is a long-standing gap in US export controls.
However, the rule in its reported form has three massive flaws:
1. It only applies to certain countries. This makes it fairly straightforward for Chinese customers to switch to data centers in non-covered countries.
2. It only applies to future AI chip exports. This means that all existing AI data centers used by Chinese customers like DeepSeek are unaffected.
3. It only applies to chips that are at least as powerful as NVIDIA's Blackwell generation. This means that data centers in covered countries can still provide chips like NVIDIA's H200 to Chinese customers. For reference, Huawei is not planning to produce a chip as powerful as the H200 until the end of 2027 at the earliest.
If we apply these conditions to our estimates of China's remote access from different countries* in 2027, the rule would affect at most 14% of China's total end-of-year 2027 compute, or 44% of remote access.
This is almost certainly a strong ceiling, however, as it assumes that all of this compute is Blackwells or better, and that Chinese customers do not shift usage to non-covered countries.
* country-level estimates are fairly rough - they estimate shares of our existing regional data using country-level data center pipeline statistics from the World Bank
Today, I'm delighted to share our report: Trust but Verify: Implementing Frontier AI Policy in California and Beyond.
Governing advanced AI is hard. Policymakers face an evidence dilemma: AI evolves so quickly that responding prematurely risks entrenching ineffective policies, while delaying action could leave society vulnerable to serious risks.
With no federal legislation focused on frontier AI safety, California is shaping frontier AI governance nationally. But passing legislation is only the first step. Whether California's approach succeeds depends on how it is implemented.
That's why, shortly after SB 53 passed, Carnegie assembled 13 world-leading experts to examine the most consequential implementation choices, identify issues for future legislation, and provide a blueprint for other jurisdictions.
"Trust but verify" AI governance approaches are dynamic by design: they allow governments to generate more evidence on AI capabilities and risks to inform what comes next. SB 53 was always supposed to be the first, not final, step in frontier AI policy. Assessing its strengths and weaknesses can clarify where frontier AI policy should come after—in California and elsewhere.
🚨 @CNASdc just launched the AI Policy Lab, a new hub to fuse in-depth AI and national security expertise to help meet ever growing demand from policymakers. We're hiring for *nine* open roles as we staff up. Encourage applications and referrals for exceptional candidates!
‼️ We're Hiring ‼️
The Carnegie Nuclear Policy Program is seeking a Stanton Nuclear Security Fellow!
This year-long, 2027-2028 post or pre-doc fellowship is an excellent opportunity to work alongside leading experts in Nuclear Policy.
Learn more & apply below! ⬇️
One of the great entry level jobs in media, politics, and international affairs. Come work at @ForeignAffairs as our next Rubenstein Editorial Fellow. https://t.co/DmL0LE8FzR
@discoplomacy One nuance on your first point: obviously electricity costs matter, but delays (planning uncertainty, grid connection queues etc) matter more.
We are hiring a Head of Private Office and two Private Secretaries at @AISecurityInst to work with me (Chief Strategy Officer) and @HZoete (Director).
If you’re sharp, ambitious, and want to help us run an important and fast-moving organisation, read on!
“The notion of military victory is a dangerous myth,” write Stephen Biddle, @SamWinterLevy, and Yuri Zhukov. “Terms and bargaining always matter: no war ends unless both sides choose to accept a settlement and stop.”
https://t.co/1epzwbSsHA
I’m building a team in DC. Everyone in Washington has a stake in ensuring that reliable info about alignment and RSI reaches the public.
We need trustworthy communicators who can explain METR's loss-of-control risk assessments and goals to policymakers.
Some believe that we need work for purpose and dignity, while others see freedom from work as utopic.
New paper: What does the *empirical evidence* tell us about work and wellbeing? And what does that imply for AI futures?
No modern conflict has ended by destroying the enemy’s ability to resist—not even World War II. To end a war, both sides must choose to stop fighting, and those choices always require nontrivial bargaining. Misunderstanding the relationship between bargaining and fighting produces systematic failures to integrate negotiating strategy with campaign planning, from Afghanistan to Ukraine to the US war with Iran.
New piece in @ForeignAffairs with Stephen Biddle and Yuri Zhukov on the myth of military victory.
No modern conflict has ended by destroying the enemy’s ability to resist—not even World War II. To end a war, both sides must choose to stop fighting, and those choices always require nontrivial bargaining. Misunderstanding the relationship between bargaining and fighting produces systematic failures to integrate negotiating strategy with campaign planning, from Afghanistan to Ukraine to the US war with Iran.
New piece in @ForeignAffairs with Stephen Biddle and Yuri Zhukov on the myth of military victory.
There are enough policy opportunities that we need more editors at @IFP!
Shape op-eds, reports, and private memos that let us increase the upside (e.g. infrastructure) and lower the downside (e.g. biosecurity) of a dynamist future
($5k referral bonus)
https://t.co/KKdKkswQiA