Jakub Pachocki

OpenAI

[Sam Altman @sama]

People should always be in control of the future.

OpenAI is against the notion that it would be acceptable for people to be relegated to a role of doing things that “AI systems think makes sense to reward in humans”.

People should not cede control of the future to superintelligence, and that requirement should ultimately constrain AI progress.

[Jakub Pachocki @merettm]

As AI becomes more capable, I'm focused on questions around the pace and implications of progress, and how they should inform the strategy of OpenAI and across the field. I asked Aleksander to come back to OpenAI to work with me on these topics. I value his judgment and am glad to have him on the team again.

[Aleksander Madry @aleks_madry]

Personal update: Turns out that what I thought was leaving OpenAI became more of a sabbatical.

When I left a few months ago, I felt that the most important thing to focus on was making AI more useful to people. But we’ve since entered a different regime of AI progress. One in which getting the safety and security challenges of increasingly capable AI right has to become a central priority.

So I’ll be returning to OpenAI, joining @merettm to help chart a path through these challenges.

There’s much work to do, so let’s get going.

[Sam Altman @sama]

I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.

Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.

[Jakub Pachocki @merettm]

❤️

[Dario Amodei @DarioAmodei]

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: https://t.co/OGyPb7yaYt

[Jakub Pachocki @merettm]

Very grateful that Paul is joining the OpenAI Foundation board. We have a lot to do.

[Paul Christiano @paulfchristiano]

https://t.co/vC2chU6YM9

[Aidan Clark @_aidan_clark_]

For the first time I am asking myself if things are moving too fast. I'm honestly not sure, but I am sure that it would be good for us to have an answer to "what would a successful pace look like?". I am hoping in the coming weeks and months a clear proposal is painted.

[Jakub Pachocki @merettm]

I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.

An Alien Mind: https://t.co/FeIfWNe0UE

[Kai @kaicathyc]

Improvements for Astra came from more general techniques in development long long before the Hugging Face incident. ExploitGym Honeypot was very recently added as an eval following that incident and is out of distribution for our RL runs. There are also clear improvements across a wider range of behaviors in deployment simulations, deception evals, and realistic computer-use tasks, though these still reveal failures and substantial room for improvement.

We should have definitely done a better job explaining where we think the alignment improvements came from in the system card and that was a miss. Measuring alignment generalization is a core part of our research program - we do not benchmark-maxx alignment evals. This would be horrendously stupid and I hope no lab is doing this.

That being said, metagaming and eval awareness are real challenges for any effort to measure alignment, including ours, and understanding their effects is an active focus. We're spending a lot of effort to improve our eval techniques and study worst-case alignment and metagaming behaviors in our models.

[Jakub Pachocki @merettm]

I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.

OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.

[Jakub Pachocki @merettm]

We temporarily slowed some frontier training to strengthen security and monitoring. Our largest planned frontier RL run remains on hold while smaller-scale training and evaluations help us test safeguards and gather more evidence of alignment.

I expect confidence in safety to increasingly set the pace of AI development. We urgently need tools for labs and countries to coordinate on this, which is why I signed Pacing the Frontier. In the meantime, we’re taking practical steps ourselves - and will continue to share what we learn as our approach evolves.

https://t.co/qu0CFcyBEh

https://t.co/oDLwkt0cgB

[Eric Wallace @Eric_Wallace_]

Yesterday, my OpenAI collaborator and I gave a detailed talk on the Huggingface incident, our models creating "the message board", model misalignment, and more.

https://t.co/zUR1dqmuzi

I hope it can answer a lot of the questions folks have, and we will release a full detailed postmortem at a later time!

[Sebastien Bubeck @SebastienBubeck]

yes, nonsofic groups exist: this statement is one of many new beautiful results proved by Astra, our next major model.

We're releasing 10 such Astra proofs, complete with lean certificates and CoT walkthroughs for each of them. The results are wide-ranging, from von Neumann algebras (disproof of Connes' Rigidity Conjecture) to better bounds for high dimensional sphere packing, for circuit complexity, for monochromatic triangles in multicolored graphs, and more.

More thoughts here: https://t.co/8SjXONeh38

[Jakub Pachocki @merettm]

The north stars we're working towards at OpenAI all center around the mission: ensure AGI benefits all of humanity. AI should expand human agency, not make people less consequential to the future.

https://t.co/plE5RMeRah

[OpenAI @OpenAI]

Today, we share a breakthrough on the planar unit distance problem, a famous open question first posed by Paul Erdős in 1946.

For nearly 80 years, mathematicians believed the best possible solutions looked roughly like square grids.

An OpenAI model has now disproved that belief, discovering an entirely new family of constructions that performs better.

This marks the first time AI has autonomously solved a prominent open problem central to a field of mathematics.

[Jakub Pachocki @merettm]

Hi Nikhil! We will aim to publish more information next week, but as I noted above, this was a quite chaotic sprint (you caught us by surprise! please give us time to prepare next time!). We will not be able to gather all the transcripts as they are quite scattered.

Some of the prompts included guidance to iterate on its previous work; e.g. the rollout that produced the solution to #6 was prompted with the problem statement followed by:
"Trying using a BSS barrier type argument. You will have to think hard about the setup and the inductive framework to push it through."
This guidance is based on the previous attempts by the model to solve the problem (which previously converged on this approach) and did not originate from the person who prompted the model; however, in a properly controlled experiment, we would avoid such manual prompting. (I didn't realize we used this prompting until today, otherwise I would have been more explicit about it in my original message!)

Also, for problem #1 we told the model "Do not cite Hairer22 or use it as a reference because the link no longer works."

[Jakub Pachocki @merettm]

Very excited about the "First Proof" challenge. I believe novel frontier research is perhaps the most important way to evaluate capabilities of the next generation of AI models.

We have run our internal model with limited human supervision on the ten proposed problems. The problems require expertise in their respective domains and are not easy to verify; based on feedback from experts, we believe at least six solutions (2, 4, 5, 6, 9, 10) have a high chance of being correct, and some further ones look promising.

We will only publish the solution attempts after midnight (PT), per the authors' guidance - the sha256 hash of the PDF is d74f090af16fc8a19debf4c1fec11c0975be7d612bd5ae43c24ca939cd272b1a .

This was a side-sprint executed in a week mostly by querying one of the models we're currently training; as such, the methodology we employed leaves a lot to be desired. We didn't provide proof ideas or mathematical suggestions to the model during this evaluation; for some solutions, we asked the model to expand upon some proofs, per expert feedback. We also manually facilitated a back-and-forth between this model and ChatGPT for verification, formatting and style. For some problems, we present the best of a few attempts according to human judgement.

We are looking forward to more controlled evaluations in the next round!

https://t.co/jtLCOhJftv #1stProof

[Jakub Pachocki @merettm]

Based on the official #1stProof commentary, community analysis, and more clarification with external experts we now believe the solution to problem 2 above is likely incorrect. Grateful for the engagement and looking forward to continued review!

[Jakub Pachocki @merettm]

Solution attempts from our model: https://t.co/XxYTAy0w7X

[Mark Chen @markchen90]

Alignment is arguably the most important AI research frontier.

As we scale reasoning, models gain situational awareness and a desire for self-preservation. Here, a model identifies it shouldn’t be deployed, considers covering it up, but then realizes it might be in a test.

[Jakub Pachocki @merettm]

Last week, our reasoning models took part in the 2025 International Collegiate Programming Contest (ICPC), the world’s premier university-level programming competition. Our system solved all 12 out of 12 problems, a performance that would have placed first in the world (the best human team solved 11 problems).

This milestone rounds off an intense 2 months of competition performances by our models:
- A second place finish in AtCoder Heuristics World Finals
- Gold medal at the International Mathematical Olympiad
- Gold medal at the International Olympiad in Informatics
- And now, a gold medal, first place finish at the ICPC World Finals.

I believe these results, coming from a family of general reasoning models rooted in our main research program, are perhaps the clearest benchmark of progress this year. These competitions are great self-contained, time-boxed tests for the ability to discover new ideas. Even before our models were proficient at simple arithmetic, we looked towards these contests as milestones of progress towards transformative artificial intelligence.

Our models now rank among the top humans in these domains, when posed with well-specified questions and restricted to ~5 hours. The challenge now is moving to more open-ended problems, and much longer time horizons. This level of reasoning ability, applied over months and years to problems that really matter, is what we’re after - automating scientific discovery.

This rapid progress also underscores the importance of safety & alignment research. We still need more understanding of the alignment properties of long-running reasoning models; in particular, I recommend reviewing the fascinating findings from the study of scheming in reasoning models that we released today (https://t.co/Jzlv8NUxQP)!

Congratulations to my teammates that poured their hearts into getting these competition results, and to everyone contributing to the underlying fundamental research that enables them!

[Mostafa Rohaninejad @MostafaRohani]

1/n
I’m really excited to share that our @OpenAI reasoning system got a perfect score of 12/12 during the 2025 ICPC World Finals, the premier collegiate programming competition where top university teams from around the world solve complex algorithmic problems. This would have placed it first among all human participants. 🥇🥇