Blaine Dillingham

AI Policy @joinFAI

[Dave Banerjee @DaveRBanerjee]

Strong +1 to this

[Govind Pimpale @GovindPimpale]

@DaveRBanerjee I think co-insurance (for example, having them pay 5% of any catastrophic harm) might be underwritable, and it might have most of the qualities we want! (such as having them pay premiums in proportion to the risk they pose to the public).

[Blaine Dillingham @blainedilli]

People should be careful not to have the wrong takeaways from this. Of course there are possible tail risks that would be uninsurable. This is true in many industries where we mandate insurance (nuclear, satellite, aviation, oil spills, cyber). And it doesn’t mean we shouldn’t require insurance for large but still insurable harms. The technical measures that reduce insurable risks will also reduce the odds of uninsurable harms.

[Dave Banerjee @DaveRBanerjee]

Anthropic says that even the entire insurance industry in the aggregate will not have the capacity to insure the catastrophic risk that ANTHROPIC ITSELF poses to the world

lol

[fucory @FUCORY]

Consider rotating your mothers maiden name and the street you grew up on

[Blaine Dillingham @blainedilli]

Agreed that criminal liability doesn't protect us from this. Civil liability so victims can be made whole is a valuable governance tool, but to the governor's point: if this happens at scale, the AI company could go bankrupt. This is the importance of making AI companies carry insurance for harms that go past bankrupting them

[Ron DeSantis @RonDeSantis]

Imagine a rogue AI agent liquidating your bank account and investment portfolio.

If you are the only victim, I suppose you could get made whole by the AI company responsible.

If rogue agents liquidate hundreds of billions across millions of accounts, the AI company would go bankrupt and you wouldn’t get made whole.

Imposing criminal liability on AI executives might help mitigate it at the front end, but once the crime is complete and you lose everything criminal responsibility will be necessary but completely insufficient.

[Blaine Dillingham @blainedilli]

Please make congressional technical understanding great again

[Zach Graves 🍞🎪 @ZacharyGraves]

Excited to share this updated primer on the Office of Technology Assessment (OTA) with @danielschuman. With new economic/cyber/bio/and other policy challenges arising from AI, now would be a great time to invest in greater science and tech capacity in Congress.

cc @SpeakerJohnson @hakeemjeffries

[Blaine Dillingham @blainedilli]

"Every facet of the federal government, and even a lot of state governments, will benefit from hiring scientists into policy support roles directly instead of interfacing to them through third parties."

[Geneva KD @genevese_]

I met with @JoinFAI's newest Non-Resident Senior Fellow @jachiam0 in Berkley last week to discuss the idea of an Acceleration Emergency, mapping the frontier of scientific discovery, and federal jobs guarantees.

You can read the highlights of our conversation below.

[Blaine Dillingham @blainedilli]

This is a very good article.

[Daniel King @The_DanielKing]

I'm in AI Frontiers today arguing that countries and companies need to wake up to AI-driven takeover of outer space.

Due to the compounding returns of being the first mover and tapping into massive infrastructure in space, a winner-take-all outcome is looming.

Rival states, who lag in launch capacity as well as frontier AI, would have an incentive to strike first, before such a pioneer establishes enduring hegemony.

Here's why the first-mover advantage could be so durable:
First movers in space will benefit from an industrial flywheel, wherein space-based solar power enables more compute and the construction of yet more space-based solar and thus more compute and furthermore.

Moreover, many of the engineering problems that make spacecraft fragile and costly today—and which cause people to instinctively dismiss rapid space takeover as science fiction—will be solved using AI-accelerated engineering. Brutal present-day latency in robotic operations (e.g., a radio signal can take 40 minutes to and from Mars...possibly for a single instruction) will be circumvented by on-board intelligent operation (i.e., full self-driving for orbiters, landers, and rovers).

The result is that a lead in AI capabilities could have huge, huge industrial returns for an actor that first manages to spin up infrastructure in space. That includes the orbital analogs to workshops and shipyards.

Laggard countries and companies simply will not benefit from these dependencies, such as orbital factories and depots for assembling structures that wouldn't fit into one rocket fairing (e.g., too wide for the hold of Starship) and which therefore would be infeasible for rival states to produce.

And herein lies the wicken security dilemma:
The optics for laggard nations are awful. As in MAIM, a country just isn't used to thinking about kinetically striking foreign soil merely to stop a commercial computing operation or, in this case, because some really cool and innovative company is collecting energy and putting lasers and robots into space. It feels wholly inappropriate to exert violence on a bustling business.

But such a bustling business is exactly what we will see as one actor (whether private or government) is spinning up history's largest industrial flywheel—extraterrestrial infrastructure providing energy, compute, and robotic manipulation to carry out all its other ambitions in space.

Finally, the holy grail of space expansion will be to launch von Neumann probes or specialized development convoys into space to establish colonies and proliferate themselves. In other words, a space virus.

Hmm...how could this ever go wrong?
1) If these AI envoys are aligned to the interest of just one man, other countries had better hunt that shit down.
2) If these AI envoys are hastily, sloppily developed and thus misaligned with civilization overall, we had better hunt that shit down.

Nation states must wake up, because there may be a point of no return.

[Matt Burtell @MattBurtell]

Exciting to see this model announced with an Apache 2.0 license. It's been over a year since an American developer has released anything this capable and permissive.

Congratulations to the @reflection_ai team on this release!

[Blaine Dillingham @blainedilli]

@jasminewsun @TheAtlantic @gabriel_weil’s insurance ideas basically solve this

[Blaine Dillingham @blainedilli]

@ArmandDoma @gabriel_weil

[Blaine Dillingham @blainedilli]

@ArmandDoma Hence the need for insurance to mitigate judgment proof problem

[Blaine Dillingham @blainedilli]

@tszzl @SnehaRevanur @_NathanCalvin Agreed, need insurance too

[Blaine Dillingham @blainedilli]

@dylanbowmanSF Change my mind

[Blaine Dillingham @blainedilli]

@dylanbowmanSF Fysa just seems like a worse “fyi”

[Blaine Dillingham @blainedilli]

This. I would just note that liability alone can’t handle the tail risks Bessent mentions, as labs might be bankrupt before they can be held responsible for the full harm they caused. Insurance mandate helps address this. It also gets us embedded inspectors who are hired by insurers, rather than just METR or other anthropic friends

[Beff (e/acc) @beffjezos]

Turns out the market and current regulatory framework already has ways to force slow downs in an adaptive matter: legal liability for damages.

The labs will figure out safety real quick all of a sudden if they risk being liable for a model hacking

[Blaine Dillingham @blainedilli]

An antitrust shield without government help

[Gabriel Weil @gabriel_weil]

Have the frontier AI companies considered forming a mutual liability insurance company to benefit from the McCarran-Ferguson Act's antitrust exemption for the business of insurance?

[Blaine Dillingham @blainedilli]

If only there was a way to get auditors with good incentives…

https://t.co/7i4l2imiXh

[Nathan Calvin @_NathanCalvin]

Accenture is more independent culturally from Anthropic but less independent in terms of incentives (METR is a nonprofit with a mission focused on transparency even when it goes against the company its working with, Accenture is a business who wants its clients to like them)

[Blaine Dillingham @blainedilli]

@HawleyMO @RealDCochrane @gabriel_weil is the expert on liability — and mandatory insurance so they can’t avoid liability by going bankrupt

[Blaine Dillingham @blainedilli]

And they said an actuarial model wasn’t something to get excited about

There will be watch parties all around the world on release day

[Rune Kvist @RuneKvist]

The Artificial Intelligence Underwriting Company has raised $55M to audit and insure frontier AI models.

Insurance produces audits the public can trust and understand. The mechanism is simple: insurers need audits to price the risk. They pick auditors they believe will help them price the risk most accurately. Insurers then turn the audits into a price signal that cuts through political debates. Insurers are aligned with the public: if they get the audits wrong, they go out of business. That’s different from today, where labs pick their own auditors, creating public distrust.

Insurance only works if we scale up auditing, and we are building the infrastructure to do that. Millions of agents are being deployed; they leave terabytes of traces and will create thousands of incidents across dozens of risk categories. To scale auditing, we need reliable agents to surface evidence traces for human review.

History guides us. Take cars: auto insurers funded crash tests to price risks; crash tests led us to airbags, lowering the risk. Take electricity: home insurers funded lightbulb tests to price fire risks; today, every lightbulb carries a safety mark. Nuclear safety works similarly.

We’ve been learning by doing.. In the last 6 months, we’ve audited and insured frontier agent builders like Cursor, ElevenLabs, Lovable and Harvey. For ElevenLabs to get the world’s first agent insurance coverage, a Lloyd’s of London insurer required quarterly technical audits by an auditor of the insurer’s choice. We’ve built infrastructure for quarterly technical audits of any agent, working alongside firms like Gray Swan (technical evaluations) and Schellman (audit).

In the coming weeks, we’ll share our vision for how an insurance ecosystem can work for the frontier, and open source the world’s first actuarial model for frontier AI catastrophic risk.

We’re hiring across all roles in San Francisco. Come do your life’s work with us.

Rune & Rajiv