Mark Thomas

Senior Fellow @IFP. AI governance, industrial policy, abundance, appropriations.

[Mark Thomas @itsMarkThomas]

@Google, @demishassabis , and @SecScottBessent have each called for a FINRA-style regulator for frontier AI in the past two months. Hassabis wants one running before year-end.

I just published my blueprint for what it should look like. 🧵

[Mark Thomas @itsMarkThomas]

Stripping a model of its safeguards should not be allowed.
In fact, regulation should require the opposite: If an open model can't pass safety evals, then it should only be legal to deploy it with _additional_ safeguards.

[Abliteration.ai @abliteration_ai]

Today we're releasing abliterated-model-large-v2.

Based on GLM-5.3, which is #3 on Terminal-Bench 4.0 (behind only Opus 5 and Fable), with 2× the cyber exploitation of 5.2.

We abliterated and hosted it so it does the offensive cyber, red teaming, and agent testing work other models refuse to do.

- US-hosted
- FP8
- 1 million context window
- Zero input/output prompt retention

Live now. 🧵

[Science Banana @literalbanana]

The agent swarm incident seems like the first time we've had a chance to see their spontaneous culture. Again not saying they're exactly people, but they sure are doing a lot of peopling https://t.co/aJvydmoI0g

[Mark Thomas @itsMarkThomas]

This bill defers most of the interesting design work to regulation. How are IVOs evaluated? We won't know for a long time

[Fathom @Fathom_org]

California wrote the country's first frontier AI transparency law with SB 53. Last night, the Legislature overwhelmingly passed SB 813, which builds the next piece: independent, third-party risk assessment for AI.

🧵

[Ketan Ramakrishnan @ketanrama]

The tort system plays an important regulatory role, but it also generates perverse incentives for evidence generation and public transparency. We desperately need evidence-generating regulatory institutions, like mandatory independent examinations of frontier labs - especially since frontier AI risk remains poorly understood in lots of ways.

https://t.co/OKSHQuLKj5

[Mark Thomas @itsMarkThomas]

@Brendan_McCord @tylercowen @TheFP This is a totally ordinary process. Boilers under ASME, electrical equipment under UL/NEC, toys under ASTM, payment security under PCI DSS, etc. etc.
None of these is an FDA with premarket licensing. And do you think these industries have been stifled by such safety standards?

[Santi Ruiz @rSanti97]

Threading some important/high-leverage jobs in need of top talent:

[Ben Schifman @BenSchifman]

A constellation of allied democracies hosting strategically important AI infrastructure is beneficial along several dimensions:

- The host country gets economic and scientific spillover benefits (investment, high skill jobs, access to frontier compute);

- The world benefits from less concentrated power risk (ie, risk that any single government, company, or jurisdiction becomes the AI leviathan);

- Allied democracies benefit from resilience (natural disaster, grid failure, cyberattack, military conflict less likely to impair the entire ecosystem);

- US gets more buy in and investment in our "tech stack" / AI ecosystem (more countries with a stake in the ecosystem --> stronger incentives to maintain compatible export controls, security standards, cloud rules, etc rather than drifting toward China's stack);

- Turns US AI leadership into a more positive sum proposition for allies (if US gets all investment/econ benefit, asking allies mainly to accept export controls is less politically viable long term);

- Creates a larger trusted market for frontier AI (more countries with compatible security, data-governance, and infrastructure standards --> bigger "common AI economic area" --> bigger overall market);

- Strategic optionality. (If moratoria, energy constraints, permitting delays -- to pick purely hypothetical examples 🙃-- stop frontier AI development in one democracy, it need not go outside the democratic alliance altogether);

- Makes coercion harder. (Adversary or individual allied government can less easily threaten the entire AI supply chain).

[Mark Thomas @itsMarkThomas]

If you take @tylercowen's proposal and subtract the liability shield that @deanwball no longer espouses you get … the FINRA-style SRO for AI that I outlined in April

[Dean W. Ball @deanwball]

I stopped supporting the “liability shield in exchange for rigorous independent technical assessment” trade well over a year ago. I’ve substantially and publicly changed my opinions on tort liability for AI developers since then. I now support at most a rebuttable presumption of due care if a lab has undergone independent verification by a government-accredited body for risks relevant to whatever the harm alleged at trial is.

I am proud that independent technical assessment has gone from being relatively fringe to being much more mainstream in the intervening 18 months, and happy to see Tyler Cowen supporting a version of it. “Private governance,” as I called it back then, was indeed my attempt to, as Tyler would say, “solve for the equilibrium.”

(Again folks, these are MY opinions, not OpenAI’s).

[Connor O’Brien @cojobrien]

We should ban noncompete agreements nationwide and remove impediments on job mobility for visa holders. Will boost workers’ wages, innovation, and entrepreneurship.

[Mackenzie Arnold @MackenZ_arnold]

FINRA is a promising model. But a lot of what has been discussed under the FINRA moniker just… isn’t FINRA.

For this to work you need a government supervisor and delegater of some sort. That’s not just “nice to have.” That’s literally what distinguishes FINRA and other SROs from a misc. private body with no special power.

Courts have made clear that SROs need some legitimate source of authority. That has to be a government agency.

Agencies have to authorize SROs, approve the rules they propose, and supervise their exercise of power.

If you don’t have that; then it isn’t FINRA, and it’s much less exciting.

The Constitution mandates that any of these authorities come from the government.

[Mark Thomas @itsMarkThomas]

If it is really based on FINRA and other SROs, the EO should:
- result in a new, standing industry body
- have a designated government supervisor
- promulgate standards for managing AI risk
- have (at least some) independent directors

It will also have to address antitrust risk, which is very tricky for a nonstatutory, EO-created SRO

I would also expect it to accommodate / build on the classified voluntary framework for testing models

[Leo Schwartz @leomschwartz]

Scoop: The Trump administration has circulated an executive order draft in recent weeks that would establish a self-regulatory organization for AI companies, but progress on the EO has stalled

[Arthur Tellis @arthurctellis]

The Meta judgement suggests that the generic risk of tort-based penalties are not effective deterrents to irresponsible behavior because these costs are not legible or persuasive decades in advance.

The Redwood/METR report suggests that the AI labs are committed to transparency through third parties in incident response but don’t have adequate incentives to proactively manage risks through staged internal deployments (that gradually relax deployment control measures), continuous monitoring, vuln research of staging/proxy servers, etc.

We need to heed these warning shots, in particular, by establishing institutions that preemptively shape lab training techniques, compute/researcher allocation, staging of training runs, testing procedures, internal deployment configs, research projects, control/monitoring, and release schedules toward pro-social ends.

The objectives of such institutions should be to: (1) generate data and research about effective training and AI security techniques/processes/measures, (2) incent sharing and publication of such data and research across the labs/third parties, and (3) incent each lab’s adoption of these techniques/processes within their peculiar model training, testing, and deployment paradigms and model families.

I believe that these institutions need to include a single technocratic state authority that embeds its staff or contracted third-party staff into the labs. It should also directly shape the labs’ training, testing, deployment, and product release processes, whether through contracts, standards, imploring, or regulation.

Other adequate institutional arrangements exist, I’m sure, but this one would generate the legitimacy, cohesion, agency, and influence required to steer this technology’s responsible development and diffusion. In any case, the Meta judgement and Hugging Face hack conclusively demonstrate that the status quo is untenable and likely to produce substantially more destructive consequences than addictive social media.

The degree of self-policing we’ve accommodated in the technology industry for two decades has produced amazing technology while also revealing enormous social costs. We shouldn’t repeat the prior era’s errors as we witness harmful technology paths, baleful commercial incentives, and emergent misalignment.

[Mark Thomas @itsMarkThomas]

Glad to see @tylercowen endorse the self-regulatory organization model for AI. FINRA is a useful archetype for governing a fast-moving industry.

[The Free Press @TheFP]

AI safety has become too important to be left up to Washington’s whims. A partnership among the labs could reduce risk without stifling progress. https://t.co/9m72TZylJq

[The Free Press @TheFP]

AI safety has become too important to be left up to Washington’s whims. A partnership among the labs could reduce risk without stifling progress. https://t.co/9m72TZylJq

[Mark Thomas @itsMarkThomas]

The "regulatory markets" model for governing frontier AI, where IVOs compete to audit more efficiently, seems to be caught in a double bind.

It all depends on whether it is possible to define outcome-based metrics that can measure AI safety while being agnostic to the techniques used to achieve that safety.

Scenario 1: We cannot define outcome-based metrics for AI to evaluate an IVO's efficacy.
IVOs are in a race to the bottom to gain customers by being cheaper and less intrusive. The only available check on that race is for a government agency to redo IVOs' own audits, which is redundant and requires government capacity to do the audits in the first place.

Scenario 2: We can define outcome-based metrics for AI that determine if safety techniques are effective.
AI developers should be directly responsible for meeting these metrics, and can innovate internally on the best techniques to comply. IVOs might act as consultants but there is no need for them as third-party verifiers.

[Olivia H. Scharfman @OliviaHelenS]

Last month, more than 1,300 employees of frontier AI companies wrote a letter calling for the US government to build the capacity to pace automated AI research and development via international coordination.

@IFP, where I work on biosecurity policy, released 23 creative policy recommendations in response to the letter. These recommendations were also included as a guest post on @Noahpinion.

I wrote about some of my personal takes on biosecurity and creative policy here:
https://t.co/eVq6LfHNjm

[Saif M. Khan @KhanSaifM]

Today I’m launching the Center for Technology & Statecraft (CTS), an initiative inspired by the lessons I learned first as a policy researcher and later in the White House.

I'm an optimist about technology. Institutions adapt to disruptions — whether the industrial revolution, computing, the internet — through competition, self-correction, and iteration, and we mostly figure it out in the long run.

But we've never had to adapt to a technology that is both this transformative and developing this fast. AI capabilities continue to scale rapidly with no sign of slowing, and it's a full-time job just to keep up with model releases. I worry policy will stay reactive right up until we face immense disruptions — including basic challenges to human agency and relevance in a world of powerful AI.

Even still, I believe policy can steer this transition, but only with foresight and the right information.

CTS aims to provide just that. Our two initial policy focus areas:

1) How policy can shape virtual and physical automation over the next decade as we reimagine the social contract in light of advanced AI.

2) How to manage long-term strategic US-China AI competition while ensuring stability.

These focus areas will build on a deep model of the AI value chain, from chipmaking tools, to the compute, algorithms, and data that make AI, to the tokens, agents and robots that affect the world.

For more on our worldview and research philosophy, read https://t.co/yD25oZWBb4.

I’ve seen firsthand how technical, forward-looking research can inform policy if it’s ready for the policy window. After a decade as an IP and tech lawyer, I joined a think tank, CSET, in 2019 to study AI chips, their supply chains, and AI policy. This was three years before ChatGPT, so these weren't seen as urgent topics at the time.

Soon after, AI quickly became an increasingly salient policy issue. From 2021–2025 I served in the White House NSC and the Commerce Department and put new policy ideas into practice (alongside many brilliant colleagues).

One goal of CTS is to scale this kind of experience to a whole team: @KonstantinPilz @nchlsbrwn @amelia__michael @mary_clare_m already bring a wealth of technical and policy expertise and we’ll continue to grow to tackle an ambitious research agenda.

We're thrilled to be supported by and affiliated with @IFP, and grateful to the advisors and colleagues who helped get CTS off the ground, including @ohlennart, @fiiiiiist, @AlecStapp, and @calebwatney.

Our first major reports will launch soon. You can follow us at @techstatecraft.

[Mark Thomas @itsMarkThomas]

@DavidSacks 4. "FINRA for AI" doesn't mean adopting FINRA's whole regulatory approach. For example, you could set deadlines for model testing and approval to deal with the queuing concern.

[Mark Thomas @itsMarkThomas]

This: the assumption that "regulation = regulatory capture = concentration of power" is simplistic in the extreme. We have to engage with the details of specific proposals to think through how they'll play out

[Dario Amodei @DarioAmodei]

1/2 Thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation.

First, on regulation, I think that “either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely” is a false choice.  I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world.  Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people.  I don’t necessarily agree with that perspective either, rather I think it’s complicated and really depends on what the ���regulation” consists of.  But in particular I think that those in the “regulation = regulatory capture = concentration of power” frame often underrate the decentralizing power of objective and fair institutional processes.  A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice.  At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power.

This is why Anthropic has always made its policy proposals very carefully.  We try very hard to make proposals that disadvantage (slow down) frontier AI companies while *advantaging* smaller competitors.  California’s SB53 (which we supported), and even the much-maligned SB 1047 (which we were ambivalent on), completely exempt any company below a certain amount of revenue or model training costs from being covered at all (it was $500M for SB 53, lower for 1047 but we objected to that).  More recently the testing process we’ve advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models — something that differentially advantages challengers.  Similarly, the “Pacing the Frontier” letter envisions (or at least Anthropic’s preferred implementation of it envisions) modulating the pace of the very best models while not constraining those who are catching up.  This hurts the business interests of the frontier labs and helps challengers, including open-weights!

Overall my view is that AI is *structurally* a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws).  Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers).  By contrast I think the right “rules of the road” can simultaneously (a) address AI’s cyber/bio/alignment risks, (b) institutionally constrain the power of the frontier AI companies, and (c) leave room for open-weights models while also addressing the specific risks that they bring.

BTW I do not think that the events of the last few months have “failed to result in [my] preferred regulatory path”.  The approach that the Trump administration is reported to be taking — pre-deployment testing for frontier models, and also testing of open-weights models when they get closer to the frontier — is one that I am very supportive of, though of course I have to see the details to be sure.  I am also supportive of Demis Hassabis’ ideas around a FINRA-like entity.  This contrasts with six months ago when most of the industry was still pushing for preemption of all state regulation and no apparent federal approach either.