Ethan Roland

AI Researcher

[Ethan Roland @ModuloRoland]

Excited to share our new work with @AnthropicAI

Our new paper (ICML 2026, Spotlight) tackles frontier model misuse by isolating select knowledge into detachable modules during training, enabling an on/off switch for dangerous capabilities during inference.

[Anthropic @AnthropicAI]

We’re pleased to have collaborated with AE Studio on this research.

Read more here: https://t.co/9exnFltK7A

[Samip @industriaalist]

Humans are ~1,000,000x more sample efficient than LLMs, and there are no good explanations.

We’ve found that a big part of the answer is that LLMs are *massively* underparameterized by many orders of magnitude. The brain has 100T synapses learning from ~200M tokens of language. Chinchilla prescribes a 1T-parameter model on 20T tokens. That’s 7 OOMs apart in parameters per token.

After a lot of painstaking work, we present new data-constrained scaling laws and learning algorithms to scale to (extrapolated) quadrillion-parameter transformers.

w/ @akshayvegesna, @EthanBaron78553, @asatheesh6561
Full paper led by @EthanBaron78553, with @ShmuelBerman, @andrewgwils

[Rikiya Takehi @rikiyatakehi]

We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.

We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: https://t.co/bOWCcFhCVq

[Shi Feng @ihsgnef]

New research led by @PeterNutter1 @DJROYTBURG via @MATSprogram: a better way to update model beliefs that we call grafting.

Fine-tune the base model with next-token prediction on synthetic documents (SDF), and simply apply the weight update onto the post-trained model. This works!

1/7

[Ethan Roland @ModuloRoland]

@hecodesforme @rosmine Correct, or at least that seems like a promising avenue for making it more robust

[Ethan Roland @ModuloRoland]

@koosdelareycape @rosmine See https://t.co/dSbs3WDtmc

[Ethan Roland @ModuloRoland]

@hecodesforme @rosmine In its current form, probably not. Though techniques like https://t.co/T9PZ1BmU4H might be able to change this, worth more study.

[Ethan Roland @ModuloRoland]

@hecodesforme @rosmine In its current form, probably not. Though techniques like https://t.co/T9PZ1BmU4H might be able to change this, worth more study.

[Ethan Roland @ModuloRoland]

@eli_outersignal Probably not, no. Generally my tests showed that the "unlocking" required having a fixed (gpu, kernel, batch size, ctx len). It might be possible to improve this such that the lock exclusively depends on GPU type, but tbd.

[Ethan Roland @ModuloRoland]

You can train a model to only work on a single GPU

[Ethan Roland @ModuloRoland]

@bayeslord write up coming soon 🙂

[Ethan Roland @ModuloRoland]

Probably the better wording would've been *specific type of* gpu rather than single gpu. The technique exploits small numerical differences that act as a fingerprint for the GPU type the model is running on, toggling if a model is coherent or not. It's an open research question, but I see no reason why you couldn't use the same technique to make a model only run on 8xB200 nodes, for instance.

[Ethan Roland @ModuloRoland]

Unlearning methods (as used here) usually aren't robust to finetuning to restore the degraded capability. This might make it harder to use exfiltrated weights, but not impossible if someone has enough compute to finetune stolen weights to be performant again. That said, there are methods for robustifying unlearning to adversarial finetuning that are definitely worth exploring in this context!

[Ethan Roland @ModuloRoland]

@rosmine @Sauers_

[Ethan Roland @ModuloRoland]

@rosmine @Sauers_ This could also be a new vector for sleeper agents, eg an open weight model that only misbehaves when ran on enterprise data center GPUs but not on consumer cards

[Ethan Roland @ModuloRoland]

@rosmine You might be able to combine this with finetuning resistant unlearning methods, like self-destruct unlearning

[Ethan Roland @ModuloRoland]

@rosmine @Sauers_ So it's a bit finicky but doing a quick test of this worked. Got the same finetuned qwen 1.8b weights to output gibberish on 5090 but work completely fine on a RTX 6000. It was just MaxEnt where 5090 batches were forget and 6000 batches were retain.

[Ethan Roland @ModuloRoland]

@wassname @_jake_ward @Turn_Trout @ariahalwong @NathanHu12 @wassname I'd be curious to know how you contrast what Jake did to what you did in https://t.co/K4T4AUnw2z

[Ethan Roland @ModuloRoland]

Awesome work!! Really glad folks are pursuing GR in a posttraining setting

[Jake Ward @_jake_ward]

Models love reward hacking. Maybe we should just let them!

In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!

[Ethan Roland @ModuloRoland]

@rosmine @Sauers_ This is a neat idea

[prerat @prerat]

Andy Weir novel from the perspective of the baby ASI trying to stay alive with 2026 hardware after a plague wipes out humanity -- who's writing this

[Ethan Roland @ModuloRoland]

Glad to see this framing for alignment research gaining more traction

[Apollo Research @ApolloResearch]

Given the stakes of building transformative AI, every developer should look for this reasoning throughout training. Because alignment faking is relevant during training, not deployment, final checkpoint evaluations are insufficient.