Rethinking pre-pretraining (PPT)
Allocating compute to synthetic PPT never outperforms pure natural-language training.
In our compute-matched settings: even when natural language is scarce, repeated epochs over a 100M-token corpus beat every synthetic curriculum. 🧵
Introducing Reinforcement Learning from Collective Rewards (RLCR).
RLCR is a recipe that trains language models as agent organisations, which share a message board and workspace. We use it to train Avalon-M1-Mini (120B) and study its behaviour.
We find a number of interesting results, including:
🛠️ Learnt organisational hierarchies and divisions of labour
🕵️ Increased collective verification, through hires of verifiers and testers
👥 The rise of collective thinking, where agents think more about other agents and how to help them
We believe RLCR can provide a laboratory for studying collective intelligence and multi-agent safety.
Blog post and technical report 👇
Agent infra is here!
Infrastructure is always a function of the workload. With agent adoption taking off since Dec'25, we are now seeing a new infra stack emerging.
Wrote down my & Tobias's thoughts on what's happening below! https://t.co/WSGbRmtAVF
Realized NVIDIA's genius today at GTC: all they do is remove bottlenecks to AI adoption to then sell more GPUs.
- AI infra build-out slowed by grid connections → DSX PODs
- Security concerns holding back enterprise agents → OpenShell
1/ Today we’re launching an open letter calling on the government to end unfair restrictions that stop Britain’s best switching jobs or starting companies.
UK AI companies that have raised $5bn agree: our best should be free to start and scale. 🧵
@downingstreet@biztradegovuk
Our CTO @kallyaleksiev is speaking at London Systems Club, sharing some infrastructure lessons from training Faraday.
Come join us on October 12 for a conversation about how to engineer scientific intuition.
1/ Today, we introduce Faraday, a 27B-parameter AI Scientist that extends the capabilities of coding agents with a layer of scientific intuition. Trained via long-horizon RL, Faraday outperforms Claude Opus 4.8 and GPT-5.5 on the task of replicating research papers. 🧵