Harry Booth

Reporter @TIME covering frontier AI companies. London bureau. Moving to the Bay in 2027. Signal: harry_booth.84

Ex-Tarbell fellow (May 2024 - Feb 2025)

[Harry Booth @harryboothtime]

I just published my first TIME cover story! As a writer, it's a dream come true. https://t.co/CUvPo4BUdz

[Harry Booth @harryboothtime]

@The_American_Op @TIME I specify in the question.

[Harry Booth @harryboothtime]

Watch. This.

[TIME @TIME]

Jensen Huang talks with TIME's @harryboothtime about an AI development slowdown.

[Harry Booth @harryboothtime]

@ShakeelHashim We landed in similar spots man

[Harry Booth @harryboothtime]

Insane that they weren't even looking for a general PHP exploit when AI helped find one 😵‍💫

More details in @TIME https://t.co/yTVzpv6RIS

[TIME @TIME]

Jensen Huang took the stage at Nvidia GTC in Berlin to lay out the company's vision in Europe. Here are the takeaways from TIME's @harryboothtime.

[Harry Booth @harryboothtime]

@joshua_saxe I'm reporting on this. Would love to chat!

[Harry Booth @harryboothtime]

@ZackKorman Fair play. To be clear, the post was not intended as a blanket dunk on cybersecurity, but specifically on *this* criticism towards METR—which I agree is not the strongest argument from from security folks.

[Harry Booth @harryboothtime]

Those who criticised METR for focusing on model behavior rather than OAI security in the HF report: that was because of the limited scope allowed by OAI. Given more leeway, look what they found

[ℏεsam @Hesamation]

so METR spent 3 weeks at OpenAI testing the safety systems that are designed to catch rogue agents.

they report some crazy stuff:

> 10% of internal Codex traffic was unmonitored
> agents could delay monitoring by simply continuing to act
> they had the potential to hide HUNDREDS of actions from the monitors
> Codex sandboxing and Auto-Review could be bypassed by the agents
> RL training wasn’t monitored at the start of the audit at all!

but the REAL interesting part:
researchers submitted a 61 page report to OpenAI, and OAI redacted it to 38 pages.

so if things as bad as this passed the filter and got into the final report, I wonder WTH was in those 23 pages.

OpenAI says the production vulnerabilities are now patched.

[Harry Booth @harryboothtime]

This is a big deal. For containing agents.

Formal verification mathematically proves software behaves how it’s supposed to. More reliable than testing for bugs.

I hear Ant is using Claude to formally verify policies and decide what agents can access through Nvidia OpenShell.

[Anthropic @AnthropicAI]

New on our Frontier Red Team blog: using Claude to formally verify software at large scale.

Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).

With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.

Read our research in full here: https://t.co/zUyx8CjBEk

[Tomek Korbak @tomekkorbak]

This was one of the third-party engagements I worked on at OpenAI. I'm glad the report is out! I'm disappointed that METR wasn't allowed to comment on the scale of redactions made by OpenAI and that there is no explanation why the publication was delayed for 2.5 months.

[Max Zeff @ZeffMax]

New: The three researchers OpenAI fired last month spent the summer fighting to keep AI's chain-of-thought process visible.

Then, OpenAI asked to see their Signal chats. A week later, they were let go.

More below from @keachhagey, @berber_jin1, and me.

https://t.co/7klEDa8R0M

[TIME @TIME]

As AI grows more powerful, two communities are clashing over how to keep it under control—and who gets to decide whether it's safe. https://t.co/dRfQjhhE9v

[Harry Booth @harryboothtime]

I can't help but love that Nathan, who was served with a subpoena from OpenAI in 2025, is now debating Dario in public.

[Nathan Calvin @_NathanCalvin]

Dario — Thank you for taking the time to respond to my previous post. I think it's great when this kind of engagement happens in public rather than on company Slacks, and I hope to see more of it.

I agree that Anthropic isn't best modeled as a profit-maximizer. Your post gave several compelling examples of Anthropic taking competitive hits for safety, and insofar as my previous post suggested otherwise, that's on me. But I don't think I made my main point as clearly as I could have, so let me try again. It comes down to four key claims:

1. Anthropic seems to be racing at breakneck speed to build recursively self-improving superintelligence (RSI). By its own lights, that could kill billions of people or literally destroy the world.

2. Risking destroying the world is very hard to justify. It is only conceivably acceptable if serious attempts to put RSI under robust societal governance are failing.

3. Far from meeting that burden, Anthropic, when push comes to shove, often works to keep governments from tying its hands. It seems to care much more about preventing government overreach than about reducing the risk that it and other AI companies will destroy the world.

4. Even if the AEGIS Act really was an honest mix-up, as you suggest, that is itself a serious problem. It suggests Anthropic isn't treating the governance of this technology with anything close to the gravity it deserves.

In greater detail for those who want it:

1. Anthropic is racing to RSI, and by its own lights that is insanely risky.

Launching RSI could literally destroy the world. That's according to you, your co-founders, and most of your safety staff and capabilities researchers, not to mention many AI researchers outside your walls. You are intentionally trying to create generally intelligent systems more capable than all humans that you claim in six to twelve months “could be capable” of taking over the entire internet before we have the ability to reliably monitor, control, or align them.

“Succeeding” at RSI could kill or permanently disempower billions of people in the near future, including me and everyone I love. Yet Anthropic’s primary focus seems to be marshalling an immense amount of resources and talent in a race to get there first.

2. Risking destroying the world carries an enormous burden of justification.

I believe the vast majority of people do not want Anthropic to do this (to the extent they know what it's doing). Imposing even a modest chance of catastrophe on billions of people who never consented requires an extraordinary justification. The central one Anthropic has offered: someone will build RSI regardless, so it's better for a safety-focused lab to be at the front of the pack, using that position to support strong AI governance. (Other reasons Anthropic has given, like modestly accelerating improvements in healthcare or economic growth, don’t seem remotely proportionate to the risks Anthropic is imposing by its own lights.)

But that argument only works as a last resort. Racing to RSI is only conceivably okay if attempts to institute robust societal governance to manage that race are failing despite your best efforts. A lab that relies on this justification should be a strong, consistent supporter of efforts to enact governance that could bind it, including governance that could slow it and significantly constrain what it does.

3. Rather than supporting efforts at strong regulation, Anthropic has intervened to keep governments from tying its hands.

Anthropic has sometimes helped: it played a very helpful role in passing transparency bills in Illinois and ongoing efforts in Massachusetts, and I'm sincerely appreciative. But transparency is far from enough for a technology that will be the world's most powerful superweapon at best, and an uncontrollable swarm of lethally superintelligent agents at worst. When it matters most, Anthropic has often pushed for weak regulation.

The AEGIS Act is the most obvious example. No highly risky industry is regulated as anemically as AEGIS would regulate frontier AI: not nuclear, not banking, not aviation, not food safety, not the manufacture of explosives. In all of those industries, specialist regulators can address unsafe practices well before they cause harm. AEGIS would bar the federal government from intervening to block unsafe development or deployment of models until there is an "imminent risk" of catastrophic harm, and would require it to meet a demanding burden of proof in court. Worse, AEGIS would also prevent states from picking up the slack. This design makes perfect sense if you're a company trying to minimize the risk that the government can overly constrain you. It makes little sense if you want American democracy to govern frontier AI effectively.

This isn't the first time. The initial version of California's SB 1047 would have let the Attorney General sue to stop developers like Anthropic from imposing unreasonable risks of catastrophic harm. It would also have created a regulatory agency to help set safety standards. Anthropic objected, and helped get both provisions removed before ultimately offering some support for the bill (which I also genuinely appreciate). I’m not saying that Anthropic should have supported 1047 in its initial form; that’s a complex issue and I realize there were questions about whether earlier versions of the bill were less politically feasible. Still, in Anthropic's own words, it helped weaken the bill because "[t]here is a substantial risk that the bill and state agencies will simply be wrong about what is actually effective in preventing catastrophic risk, leading to ineffective and/or burdensome compliance requirements."

The same goes for New York's RAISE Act, whose initial version also would have let the Attorney General sue to stop unreasonable risks of catastrophic harm. Anthropic credits itself as a supporter of RAISE, but it did not support the bill while it was in the legislature, including when it nearly died. After the bill passed the legislature, Anthropic publicly critiqued it, saying "Any state proposals should be narrowly focused on transparency and not overly prescriptive."

Why does Anthropic consistently take the risk of clumsy government action far more seriously than its own repeated warnings that companies like Anthropic will literally destroy the world by racing to RSI?

4. Even if AEGIS was an honest mix-up, that is itself a serious problem.

Your response frames AEGIS as an early draft and a miscommunication. Perhaps you would have strengthened it and narrowed its preemption. (That wouldn't explain Anthropic's interventions to weaken SB 1047 or RAISE, but leave that aside.)

Suppose that's exactly right. A federal framework for governing RSI is one of the highest-stakes documents your company will touch. Your post admits Anthropic "could have handled [these matters] better," but its tone ("This kind of thing happens a lot in large organizations") doesn't reflect the gravity of this issue. Anthropic talks about the need for “operational excellence” in critical areas, and I have a hard time believing that a similarly cavalier attitude would be tolerated while Anthropic was negotiating a key compute deal or preparing its next large pre-training run.

In short

The justification for racing rests on Anthropic fighting for strong governance, and Anthropic's record on regulation points the other way. Together with its breakneck racing, that fits a much darker picture of the company than the one that seems to prevail among your employees. I hope that Anthropic reflects on this picture, supports stronger regulation, exercises extraordinary caution when advocating for weak regulation, and doubles down on its founding purpose of helping the world safely navigate transformative AI together, instead of prioritizing Anthropic’s ability to win a race that imposes intolerable risks on everyone alive.

[TIME @TIME]

Older Americans are embracing Meta’s Muse for everyday tasks and companionship—even as experts warn about the AI agent’s privacy risks. https://t.co/cDkjtUzs6B

[Harry Booth @harryboothtime]

Most important paragraph here.

Being published around the time of the PHP bug news, this headline has been misread.

[Madison Mills @MadisonMills22]

SCOOP: Major U.S. banks are detecting unauthorized third-party AI agents inside customer accounts, including some attempting to take actions that appear to be independent of user instructions, sources told Axios.

The findings have alarmed banking executives, who are racing to restrict access to their systems as AI moves at a velocity faster than financial institutions can respond, multiple industry and bank executives told Axios.

Several banks are now preparing for worst-case and even “day after” scenarios, including how to rapidly secure or restore customer accounts if things go wrong, the people said.

Several bank executives told Axios they're struggling to distinguish between human users and AI agents. One executive said banks also struggle to determine whether agents are authorized by customers or potentially operating without their permission, more akin to rogue agent behavior.

Read my latest for Axios here: https://t.co/Q3bda5In8q

[Harry Booth @harryboothtime]

wut?

[The Verge @verge]

Pichai allegedly told the president, “Oceans are different, sir.” https://t.co/IgNndlwbPY

[Harry Booth @harryboothtime]

@ShakeelHashim bruh i know

[Harry Booth @harryboothtime]

@mmitchell_ai Thank you!

[Harry Booth @harryboothtime]

I’ve been trying to make sense of this rift between the AI safety vs security camps.

https://t.co/woW5vx4iwV

[Kevin Roose @kevinroose]

It’s weird that the CEO of Hugging Face continues to be an “AI is normal technology” guy despite being the victim of an autonomous cyberattack by a swarm of rogue agents, arguably the least normal thing AI has done so far.

[The Information @theinformation]

Hackers turned Claude and Grok into a network of AI agents that autonomously targeted more than 400 financial institutions and other financial targets, according to cybersecurity research shared with The Information.

Read more: https://t.co/9AjtuNWiaq

[Harry Booth @harryboothtime]

ZERO mentioned of sovereignty on stage yesterday from Jensen Huang at GTC Berlin. The new pitch is *AI will make you rich*
Interesting to see how this lands with the Europeans…