AVERI

Nonprofit working to make third-party auditing of frontier AI effective and universal.

[AVERI @AVERIorg]

Today we’re announcing a historic milestone:

The first ever double-blind evaluation of a proprietary language model. This was made possible by a unique collaboration between AVERI, @GoogleDeepMind, @OpenMinedOrg, and @MLCommons.

We tested Gemini 2.5 Flash-Lite using never-before-used prompts from the MLCommons safety benchmark family, AILuminate, inside a secure enclave, a form of hardware isolation that protects sensitive computations. Each organization involved played a unique role in making strong privacy guarantees possible.

High-stakes independent evaluation runs into a structural problem: developers and evaluators both hold assets they have good reason to protect. Developers sharing model weights risk theft and leakage. Evaluators who publish benchmarks or provide them directly to companies run the risk of them being trained against, and the benchmark then stops being independent.

Secure enclaves address this constraint. The hardware attests to exactly what code will run before either party's assets enter, and both sides review and approve that code. Then the enclave executes it, releasing only the agreed outputs to the agreed recipients.

A 2024 pilot by OpenMined, @Anthropic, and the @AISecurityInst proved this mechanism, using GPT-2 as a stand-in and a five-row eval. The pilot we’re announcing today moved to a production model and a more comprehensive evaluation.

AVERI encrypted the prompts using a private key that no other party could see, then AVERI and Google DeepMind jointly ran the evaluation, using software originally produced and adapted by OpenMined, in an enclave environment configured by Google. Finally, AVERI alone decrypted the outputs and graded them with the AILuminate benchmark criteria to inform a qualitative and (small-scale) quantitative assessment of the model properties.

Today, companies provide contractual commitments to not monitor certain interactions with their models – including evaluations conducted by third party evaluators. But stronger, technically-backed guarantees could provide greater assurance that those commitments are being honored, and will be especially valuable for scenarios such as international verification.

These results come at a critical time in the development of AI policy. Laws like SB 315 and standards like the EU’s General-Purpose AI Code of Practice rightly demand that security and privacy be respected in the process of third-party assessments, but provide little guidance on how to achieve this.

Policymakers considering audit requirements should feel encouraged by these results to be ambitious in requiring that deep, secure access be provided. Secure enclaves are just one of many technologies under rapid development that can help enable such access. Furthermore, a demand signal from lawmakers will further accelerate the maturation of such technologies, along with complementary “low-tech”approaches such as embedding auditors within companies.

Today also marks the beginning of a more public phase for AVERI’s pilot work. AVERI's strategy is a flywheel: we conduct pilot audits with leading AI companies, carry out technical and policy research to inform audit design, and convert the insights of each into open source tools, industry-wide auditing standards, and policy analysis. Over time, we hope for improved standards, stronger policy demand signals, and tooling to enable even more ambitious pilots and more informed research – bringing us closer to our mission of making frontier AI auditing effective and universal. This pilot is one of several underway, with more findings to follow in the coming months.

We are grateful to our collaborators on this pilot for working with us to advance the frontier of AI governance.

Read more in our blog post and joint technical report.

AVERI blog post: https://t.co/5nMvcdFGwc

Joint technical report: https://t.co/maNIwC37gI

[Google DeepMind @GoogleDeepMind]

In an industry first, we’re piloting double-blind evaluations for frontier AI.

By creating a secure environment where neither test prompts nor model weights are revealed, we can ensure external safety and performance evaluations of our models remain private, robust, and trustworthy. → https://t.co/ocwQ2iWFDz

[MTS @MTSlive]

AVERI standards director @prpaskov says the "only 50 AI evaluators" narrative is misleading because it ignores an entire hidden pool of auditors, cyber experts, and NDA-bound teams already doing the work:

"When people are giving this 50 evaluators number, they're largely focusing on the evaluation of the model or the system level. At AVERI, we think of it in three parts. You can audit the model or the system, you can audit compute, and you can audit governance."

"A lot of frontier evaluations to date have been done under NDAs. So there are more even on the model and system level than are visible."

"The skillsets of traditional auditors like the Big Four, KPMG, EY, there are a lot of skillsets in governance auditing and risk management auditing that we can pull from. In the cybersecurity world, tons of skillsets."

"There is actually a lot of talent out there. We just need to do a better job at funneling it."
@AVERIorg

[Patricia Paskov @prpaskov]

I spoke with @NPR on the current state of third-party evaluations, alongside experts including @DKokotajlo @AnkaReuel and others.

I think this is a fantastic write-by @Jingnan_Huo that captures the nuances of this evolving landscape for a broad and general audience.

https://t.co/KzIp5iVrip

[AVERI @AVERIorg]

AVERI is glad to see that independent auditing of frontier AI was prominently featured in the White House accord signed today.

The details of such auditing matter greatly, which is why AVERI works with other members of the @aievalforum to identify best practices and develop standards in areas such as auditor independence, minimum access requirements, and audit scope.

We believe that audits conforming to such standards should ultimately be required for leading AI companies, though voluntary commitments are a step forward.

[Ashley Gold @ashleyrgold]

Here's the White House Accord on Super Intelligence. Only six signatories - Pichai, Amodei, Zuckerberg, Brockman, Musk, Huang

[Kevin Frazier @KevinTFrazier]

Watch here: https://t.co/toRHFxa0jh

Listen here: https://t.co/biV1ikoEc9

[AI Evaluator Forum @aievalforum]

“When companies promise that they’re going to do a better job on safety, we shouldn’t have to take their word for it.”

- @random_walker on @CBSNews discussing today’s AEF letter and our call for credible, independent AI evaluations.

https://t.co/KqFXax9TkY

[Nat Purser @NatPurser]

over the past week, I’ve gotten a lot of questions about what independent AI evaluations should actually look like in practice.

what about conflicts of interest between labs and evaluators? will evaluators actually get meaningful access? what if companies just block the publication of unfavorable findings?

these are good and important questions. and we shouldn’t assume voluntary arrangements alone will produce the conditions that credible independent evaluation requires.

so i’m very excited to see 100+ researchers, evaluators, and experts — including us at @AVERIorg — laying out minimum conditions for embedded evaluation, including:

meaningful independence, strong access, transparency / editorial control, protections against retaliation, and a diversity of perspectives and competencies.

and we shouldn’t try to incorporate those principles in ad hoc, one-off agreements, policy is critical to standardizing these safeguards across the industry.

glad to see @aievalforum leading on putting these principles on paper.

[Patricia Paskov @prpaskov]

i'm heartened to see ai companies agree *that* we need embedded evaluations; now the question becomes *how.*

answering this question requires many voices. today 100+ respond, calling for five minimum requirements to make embedded evaluations credible.

we have a lot of work to do. to meet this demand, @aievalforum -- where i act as senior advisor -- is embarking on an ambitious path ahead to:

(1) develop standards for embedded evaluation;

(2) create independence-preserving funding mechanisms; and

(3) establish a talent pipeline for independent evaluation.

we are actively expanding our membership. we also welcome conversations on support and funding.

please reach out if you would like to collaborate with us on any of these initiatives.

[AI Evaluator Forum @aievalforum]

Today, more than 100 leading AI experts endorsed a set of minimum requirements to take seriously AI companies' recent call to embed external evaluators.

These evaluators need to be genuinely independent, transparent, and represent a range of expertise areas. They also need to be guaranteed employee-level access and to be protected from retaliation for findings that make companies look bad.

We welcome model developers’ recent calls for independent oversight, but it’s what they do next that matters. The labs must be accountable for ensuring these requirements are met, so that the public can have faith in the process and the outcomes.

Over the past week, the AI community has debated the appropriate role of external evaluation, including who should do it and on what terms. We may not agree on everything, but there is a lot of common ground.

To make embedded evaluations credible, more than 100 experts with varying backgrounds and ideas about AI risk agree in today’s letter that frontier AI developers should:

1. Guarantee embedded evaluators full editorial independence and mitigate conflicts of interest

2. Rely on multiple evaluators with differing viewpoints and areas of expertise

3. Publicly document the terms under which evaluators operate, as well as facilitating permissive publication of methods and findings

4. Shield evaluators from retaliation

5. Grant access equivalent to that of highly privileged employees

There is a thriving and growing ecosystem of independent AI evaluators who are advancing this science every day – but we need aligned standards, guaranteed protections, and independent funding. That’s why we created the AI Evaluator Forum.

Today we are entering our next phase. We’re launching an open call for new members, collaborators, and independent funding sources to help evaluators meet this moment and demand accountability from developers. Join us in building the evaluator ecosystem.

See the public letter here: https://t.co/LopvS0NaFW

Learn more at https://t.co/7S7G6hia4n

[AVERI @AVERIorg]

AVERI is proud to be a member of AEF and looks forward to continued work on establishing clear standards as the evaluation ecosystem matures.

[AI Evaluator Forum @aievalforum]

The AI Evaluator Forum (AEF) welcomes recent statements regarding the importance of embedded independent experts to verify AI safety and security claims. This is a first step towards trustworthy oversight of frontier AI.

No single evaluator can do this work alone. A vibrant ecosystem of independent evaluators from distinct backgrounds can offer a range of expertise and methodology, help to ensure rigor, and avoid a single point of failure. The Forum exists to strengthen this ecosystem.

Realizing the benefits of third-party evaluations also requires independence, access, and transparency. AEF brings evaluators together to address these questions. Our first standard, AEF-1: Minimum Operating Conditions for Independent Third-Party AI Evaluations, envisions a minimum floor for access, managing conflicts of interest, funding relationships, recusal requirements, and transparency of evaluation terms.

Read more here: https://t.co/RmPU66Z3NV

The long-term success of third-party evaluation relies on standards and frameworks. AEF is committed to developing these with its members and collaborators. We welcome engagement from organizations committed to rigorous, independent work.

Learn more about AEF, express interest in joining, and raise questions for us here: https://t.co/KSWjgAhgE1

[AVERI @AVERIorg]

AVERI's Executive Director, @Miles_Brundage, spoke with Business Insider about recent discussions of embedded auditing:

[Business Insider @BusinessInsider]

A hot job title, "embedded evaluator," has entered the AI space. https://t.co/yxhmHgh2mO

[AI Evaluator Forum @aievalforum]

The AI Evaluator Forum (AEF) welcomes recent statements regarding the importance of embedded independent experts to verify AI safety and security claims. This is a first step towards trustworthy oversight of frontier AI.

No single evaluator can do this work alone. A vibrant ecosystem of independent evaluators from distinct backgrounds can offer a range of expertise and methodology, help to ensure rigor, and avoid a single point of failure. The Forum exists to strengthen this ecosystem.

Realizing the benefits of third-party evaluations also requires independence, access, and transparency. AEF brings evaluators together to address these questions. Our first standard, AEF-1: Minimum Operating Conditions for Independent Third-Party AI Evaluations, envisions a minimum floor for access, managing conflicts of interest, funding relationships, recusal requirements, and transparency of evaluation terms.

Read more here: https://t.co/RmPU66Z3NV

The long-term success of third-party evaluation relies on standards and frameworks. AEF is committed to developing these with its members and collaborators. We welcome engagement from organizations committed to rigorous, independent work.

Learn more about AEF, express interest in joining, and raise questions for us here: https://t.co/KSWjgAhgE1

[AVERI @AVERIorg]

AVERI commends recent statements by the leaders of Anthropic, OpenAI, and SpaceX regarding the importance of embedding independent experts within frontier AI companies. We are committed to working with various stakeholders to advance standards for frontier AI auditing, and moving towards binding requirements for auditing that aligns with those standards.

Our flagship paper, “Frontier AI Auditing,” published in collaboration with experts in industry, civil society, and academia, lays out a vision for third-party auditing of frontier AI companies’ systems and practices and includes embedding as a key component. Embedding helps facilitate deep, secure access to non-public information.

Moving embedded auditing from a voluntary practice to an effective and universal industry requirement requires not only pilot projects, but also standards, tooling, and thoughtful policy design. AVERI exists to provide these, and we are conducting pilot projects aimed at informing standards for different aspects of AI audits.

Embedded auditing will require clear standards for who counts as a qualified and independent auditor, how audits should be conducted and reported, and what safety and security standards companies should be audited against. Fortunately, we are not starting from scratch: with evaluators across the ecosystem, we coauthored a standard (AEF-1) which covers, e.g., conflict of interest disclosure and management in an AI context.

Learn more about vision for frontier AI auditing here: https://t.co/t2YXZ9FPHH

[Dario Amodei @DarioAmodei]

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: https://t.co/OGyPb7yaYt

[Nat Purser @NatPurser]

anthropic unilaterally committing to having embedded independent evaluators is BIG and worthy of commendation.

they are not the only ones building the frontier. this should be standard practice across frontier companies and neolabs building similarly capable systems — independent third-party evaluation should be required by law.

[Patricia Paskov @prpaskov]

Anthropic is unilaterially committing to embedding evaluators to verify safety practices and report incidents. Evaluators will get access badges, company laptops, and "permissions and tools similar to those of internal employees who do comparable risk assessments."

This is a remarkable step for frontier AI auditing, transparency, safety, and security. I hope that other companies follow suit. And I look forward to continuing to build the standards, practices, tools, and policy at @AVERIorg to make this work rigorous, effective, and universal.

[Nat Purser @NatPurser]

biannual @AVERIorg staff retreat aura post

[AVERI @AVERIorg]

AVERI is mentoring CBAI fellows this fall in engineering and standards projects shaping the field of frontier AI auditing. Apply below by September 6th.

[Cambridge Boston Alignment Initiative @cbai_ai]

Applications are open for the CBAI Fall Research Fellowship in AI Safety & AIxBiosecurity. Apply by September 6th at 11:59 PM EDT!

📅October 13 - December 18
💵 $15,000 stipend for 10 weeks
💰 ~$1,500+/week compute support
🏠 Housing close to our offices, arranged by CBAI

[Santi Ruiz @rSanti97]

Making AI go well will require capable and serious people in places like @AVERIorg. Maybe that's you. https://t.co/Ak74Zg8X48

[Patricia Paskov @prpaskov]

Want to shape the future of frontier AI auditing? Join us. @AVERIorg is hiring a Security Research Engineer, Pilot Audit Project Manager, and accepts general expressions of interests.

Apply here: https://t.co/Na2JD49Amo

[pamela mishkin @manlikemishap]

serious science? in ai governance? very cool and timely work from averi. congrats to all involved!!

[George Balston @GeorgeBalston]

This is a first, and (we think) is super important.

Confidentiality concerns wrt both proprietary models and sensitive benchmarks are legitimate. Having the ability to run both in secure enclaves gives privacy guarantees which are stronger than contractual obligations, and will hopefully create greater opportunities for external evaluations and subsequently safer AI.

A pleasure working with @GoogleDeepMind @openminedorg and @MLCommons on this.