Abhay Sheshadri

AI Safety/Security Researcher

[NSA Cyber @NSACyber]

NSA joined @NIST and @CISACyber in releasing guidelines for managing SI control incidents. Organizations utilizing SI are encouraged to adopt these practices. Read the joint report: https://t.co/9Bd3Sku0KW

[Abhay Sheshadri @abhayesian]

@KhoriatyMatthew @ejcgan In the post, we explore what happens when the side task and main task get decoupled, for the same reason you're describing. It usually happens at higher learning rates. Decoupling halts further progress on the side task, but doesn't erase the gains already made.

[Abhay Sheshadri @abhayesian]

New Redwood Research post

LLMs can use RL training on one task to teach themselves a different, unrelated capability. In a form of exploration hacking, the model outputs answers that get reward only when it succeeds at a side task it wants to learn. We call it reward laundering.

[Abhay Sheshadri @abhayesian]

You can read the full post, with additional results, here
https://t.co/bmptgRAXfJ

[Abhay Sheshadri @abhayesian]

We think it's at the rigor of a mid-MATS research update. We checked correctness mostly by reviewing writeups, running an automated LLM reviewer, and spot-checking that the released codebase reproduced the results.

[Howard Lutnick @howardlutnick]

CAISI’s latest report shows that Kimi K3 remains behind America’s leading frontier AI models.

The United States continues to lead in frontier AI because we’re home to the greatest innovators and technologists the world has ever seen.

[Dave Banerjee @DaveRBanerjee]

Redwood Research just published an AI research project where the experiments were designed and run entirely by an automated research agent

They rate the work "at or slightly below the rigor of a typical MATS project."

My friends in DC, please take recursive self-improvement seriously. If AIs can do research at the level of a junior researcher today, expect the rate of AI progress to accelerate

The AIs of 2027 may be unrecognizable from what we have today

https://t.co/QDZLGzZuAV

[Abhay Sheshadri @abhayesian]

@a_karvonen @Alan_Cooney_ I've seen several projects where 70B massively improved on null results from newer but smaller models. But I don't think anyone has rigorously compared it with the best new open weights models.

[Abhay Sheshadri @abhayesian]

@miclchen congrats!

[keshav shenoy @kshenoy_]

Can LLMs simply tell us about unwanted behaviors they’ve picked up in training?

We train a single Introspection Adapter (IA) that makes fine-tuned models describe their behaviors.

It generalizes to detecting hidden misalignment, backdoors and safeguard removal.

[Micah Carroll @MicahCarroll]

Very curious to see if @OpenAI's auditing pipelines catch these agents! Thank you for doing this work which materially helps with race-to-the-top dynamics

[Abhay Sheshadri @abhayesian]

📚 Paper: https://t.co/XLFQLFgX9g

📝Blogpost: https://t.co/E10Y6SRNEI

🤖 Models & Datasets: https://t.co/rv5vP0EILY

💻Codebase: https://t.co/J9JSWbaqxW

[Abhay Sheshadri @abhayesian]

New Anthropic Fellows research: Alignment auditing—investigating AI models for unwanted behaviors—is a key challenge for safely deploying frontier models.

We're releasing AuditBench, a suite of 56 LLMs with implanted hidden behaviors to measure progress in alignment auditing.

[Abhay Sheshadri @abhayesian]

We’re releasing the AuditBench models, investigator agent, evaluation framework and training pipelines.

We hope the community can use these resources to help develop alignment auditing into a quantitative and iterative science.