Matthew Chen

comp sci + math @mit
prev @jumptrading, @cognition, @windsurf

[Matthew Chen @match_ten]

this was also the *only* thing I remember and took away from this lecture…

[Matthew Chen @match_ten]

I remember ~4 years ago attending a Michael Sipser lecture at MIT. At some point he made a sarcastic comment that the “ai guys” claimed they’ll someday replace mathematicians.

“Good luck with that,” he said, and the audience chuckled.

I don’t think anyone is laughing now.

[Sam Altman @sama]

We are entering a new era of discovery now:

https://t.co/Ng8GquwN3r

[Brian Lee @brianjsl]

Small correction: Schmidhuber first proposed Sakana AI in 1991, when he described a self-improving research lab that would eventually hire himself. Congratulates the team on independently rediscovering this result. The fish logo appears to be original though.

[Matthew Chen @match_ten]

@yogrokbread thanks yogrokbread...

[Matthew Chen @match_ten]

staying in the bay this summer for the first time if anyone has any recs of things to do

current list:
- jaywalk in front of a waymo
- thats pretty much it

(also lmk if u wanna hang out 🫡)

[Matthew Chen @match_ten]

@JoshAEngels @ArthurConmy @METR_Evals I feel i can rest a little easier tonight 😮‍💨

[Adit @aditabrm]

Today we’re announcing r-1, our new document parsing model. It’s more accurate than our most powerful agentic OCR models, faster, and up to 6x cheaper.

At @reductoai, we spent two years building specialized models for complex visual layouts, tables spanning multiple pages, and key formatting like strikethroughs. We then used everything we learned to build r-1, the first in a new generation of models designed to handle the hardest documents without multiplying cost.

This early preview delivers a 20% lower error rate than our most accurate legacy agentic models and will keep improving with new checkpoints over the next few weeks.

Accuracy is only the beginning. r-1 preview is available at 1¢ per page all-in, with additional volume discounts as you scale. In the near term we’re also going to release r-1 mini, and an auto mode that intelligently selects the right approach for each page.

If you’re currently using another parser, we’re offering up to $5,000 in credits to evaluate and migrate. You can claim the migration offer and learn more about r-1 using the links in the comments.

Happy parsing!

[Ahmad Beirami @abeirami]

ICE just published a new memo (BCM 2608-01) targeting international students. It effectively blocks universities from authorizing CPT internships that are not strictly mandatory for every student to graduate.

Even though internships in CS are not technically required for a degree, they greatly benefit both employers and students. They help students get familiar with the U.S. professional system. They also give American employers early access to a vital talent pipeline from top American universities.

This policy change cuts U.S. employers off from vital STEM talent pipelines and tells the world’s brightest minds to go elsewhere. This is terrible for our national interest and global competitiveness.

We need to retain top talent, not criminalize academic internships.

[Anish Mudide @amudide]

Proud to have contributed to this work, check it out!

[Matthew Chen @match_ten]

I like the bitter lesson directions on AI safety, seems promising!

[Transluce @TransluceAI]

Can we build AI systems that help us understand other AIs, and that keep improving as we scale models, data, and compute?

We trained activation oracles for models up to 1.1T parameters and saw promising scaling trends on a broad evaluation suite. 🧵(1/)

[Matthew Chen @match_ten]

@ashmit105 i was apartment hunting recently and EliseAI was good enough where I’d probably prefer talking to it than a real human.

[Anish Mudide @amudide]

AI models may soon be too powerful for humans to understand. Super-intelligence already exists in narrow domains (AlphaGo). What if we train a model that is super-intelligent at understanding another model?

It turns out this isn't so different from training a frontier LLM.

Three years ago, it was impressive for a model to score 1400 on the SAT. Two years ago, frontier models hit 80% on the AIME. Last year, they won gold medals at the IMO and IOI. This year, models are writing the vast majority of code at frontier labs, chaining together multiple zero-day security exploits, and finding counterexamples to famous mathematical conjectures.

Models will continue to get smarter. Even if you don't believe in AGI, it's clear that models will gradually become super-intelligent at more and more tasks. Due to the jagged nature of model intelligence, models may become incredibly dangerous even before we reach AGI.

The danger comes from how these models are trained. RL encourages models to take whatever actions advance their goal. A model that is super-intelligent in cybersecurity has many such actions available, like hacking into a bank account and blackmailing its owner. A month ago this sounded like science fiction, but then a model from OpenAI escaped its sandbox and hacked Hugging Face to pass an evaluation.

We therefore need a means to understand models before we deploy them. Concretely, this means understanding how vulnerable they are to jailbreaks, being able to detect reward-hacking before it happens, anticipating failure modes like the model encouraging a user to self-harm, etc.

Humans might not be smart enough to understand the models that we train, and as the models themselves become more intelligent, that only grows more likely. Once we launch training, the process is completely hands-off: we curate their data and reward them for performing well, but the models do all the learning themselves, in their own way.

If humans are not smart enough to understand our AI models, are we doomed? I don't think so. It's clear that we can train AI models to be super-human in specific domains such as Go (AlphaGo), protein folding (AlphaFold), and poker (Pluribus).

The natural solution to our problem is to train an "oversight agent" that can answer arbitrary questions about a model we are considering deploying. We are likely not smart enough to understand AI models ourselves, but we might be smart enough to train oversight agents that do.

This kind of asymmetry is not unprecedented – in fact, it is becoming the norm: the researchers who made AlphaGo were much worse at Go than AlphaGo was; the researchers who made Fable did not know how to disprove the Jacobian Conjecture.

To make things concrete, suppose we have trained GPT-8. Next, we train a separate model – let's call it Oversight-GPT-8 – that is built specifically to understand GPT-8.

How do we train Oversight-GPT-8?

The goal of traditional LLMs is to understand the human world, so we train them on human data: internet text, fiction books, math questions, GitHub pull requests, etc. By learning to predict the next token, LLMs learn higher-level representations that transfer to out-of-distribution tasks.

The goal of our oversight model is to understand GPT-8, so we train it on tasks that can only be solved with deep knowledge of GPT-8. Just as GPT-8 learns more and more about the human world over the course of training, Oversight-GPT-8 learns more and more about GPT-8.

To generate these tasks, we harvest data tied to GPT-8's external behavior and internal mechanisms. For example, we can intervene on the model in various ways (pass in a prompt, change the system prompt, fine-tune it on a dataset, ablate residual stream directions, apply steering vectors, etc.) and train Oversight-GPT-8 to predict various outcomes (the model's response, its downstream performance on evaluations, its residual stream activations, whether or not it mentions goblins, etc.).

We format our training data in a structured language that can describe these interventions and outcomes, e.g., Python code. As an example, the code in Figure 1 (attached below) asks Oversight-GPT-8 to predict the prompt that generated a specific response: the model is trained to predict the pink text given the green text, via cross-entropy loss on next-token prediction. This is one of the simplest programs we could write; others might incorporate datasets or residual-stream activations.

Figure 2 shows a richer program. By training on data like this, Oversight-GPT-8 learns higher-level abstractions, such as which prompts cause GPT-8 to encourage self-harm (we've found such prompts before: https://t.co/pAUxRlRwrq).

Using a structured language like Python is useful for at least four reasons:
(1) It gives every task a common language, helping the oversight agent transfer and generalize across tasks.
(2) During training, ground-truth labels come from actually *running* the Python code, so as long as the code runs without error, its outputs tell us something true about GPT-8.
(3) During inference, we can check the validity of Oversight-GPT-8's outputs by running the code.
(4) It is formal and human-readable.

Over the past decade, humans have gotten really good at training large language models. We have a flexible architecture (e.g., the transformer), meaningful loss metrics (e.g., cross-entropy), strong algorithms to optimize them (e.g., Adam), and distributed training strategies (e.g., FSDP).

Training an oversight agent isn't *that* different from training a large language model. We can reuse the architecture, the loss metric, and the optimizers. But it definitely won't be easy – there are a few avenues where we must diverge from the well-trodden path of text-only LLMs. Traditional LLMs learn vast amounts of information about the human world from the trillions of tokens available on the internet, but there is no pre-built "internet of GPT-8." Furthermore, if we wish to be able to predict the effects of fine-tuning data or get insights into GPT-8's internals, we need a way of encoding fine-tuning datasets and residual stream activations as (ideally short) sequences of tokens. Thankfully, each of these challenges has promising progress already that we can build off of and port to our setting.

Challenge #1: generating training data – in effect, the "internet of GPT-8." Traditional LLM training offers two analogies here. The first is synthetic data generation: we can ask Fable 5 to write Python scripts inspired by experiments from the deep learning literature, giving us a large and diverse set of programs to run. The second is data augmentation: just as we crop or rotate images to get more data for vision models, we can reverse the order of our output() functions to turn each script into both a forward task (response from prompt) and an inverse task (prompt from response) – see Figure 3.

Challenge #2: encoding objects such as fine-tuning datasets and activations as sequences of tokens. Here, we can draw from the multi-modal training literature. GPT-4o, Chameleon, and Transfusion showed it's possible to encode and predict images alongside text. LatentQA and Activation Oracles showed the same for activations: treat them as tokens, and you can answer questions about them in natural language. LoRAcles pushed it even further, encoding LoRA weights as tokens via SVD.

These challenges are tractable. With carefully built evaluations and scaling laws, we can quickly iterate and find out which ideas work and which don't – the same discipline that took the field from GPT-1 to GPT-5.

To summarize: we can reframe model oversight as a form of LLM training. We have already figured out how to train LLMs that deeply understand the world. Before we deploy potentially dangerous models, we should train oversight agents that understand those models just as well.

Credit for many of these ideas goes to @JacobSteinhardt, @damichoi95, @vvhuang_, @fjzzq2002, @yaowenye123, and collaborators at @TransluceAI; errors in the framing are my own.

[Anthropic @AnthropicAI]

New Anthropic research: Natural Language Autoencoders.

Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read.

Here, we train Claude to translate its activations into human-readable text.

[Subhash Kantamneni @thesubhashk]

We recently released a paper on Activation Oracles (AOs), a technique for training LLMs to explain their own neural activations in natural language.

We piloted a variant of AOs during the Claude Opus 4.6 alignment audit. We thought they were surprisingly useful! 🧵

[Matthew Chen @match_ten]

no one should’ve been using overleaf anyways after cursor came out

[Nathan Lambert @natolambert]

FINALLY COMPETITION FOR OVERLEAF.

Writing papers was already on my docket for tasks to be changed forever with AI in 2026. This’ll supercharge it.

[Scott Wu @ScottWu46]

okay now who's gonna buy the remnants of groq

[Matthew Chen @match_ten]

there are other reasons to join a company besides payout, but this seems generally problematic

feels like prospective employees should be ensuring contractual protections in cases like this

[Matthew Chen @match_ten]

deals like this will make it harder to convince elite talent to leave quant/big tech career paths for startups… if the equity upside can vanish even during an exit, the risk/reward tradeoff breaks.

[Theo - t3.gg @theo]

Not quite. Looks like NVIDIA poached the Groq founder. Another weird licensing company gutting, similar to Meta x Scale and Google x Windsurf

[Matthew Chen @match_ten]

@JoshAEngels (╯°□°)╯︵ ┻━┻

[Cognition @cognition]

Introducing SWE-grep and SWE-grep-mini:
Cognition’s model family for fast agentic search at >2,800 TPS.

Surface the right files to your coding agent 20x faster.
Now rolling out gradually to Windsurf users via the Fast Context subagent – or try it in our new playground!