Nathan Herr

PhD Student @AI_UCL & @UCL_DARK. ex Research Scientist @IBMResearch.

[Nathan Herr @naitherr]

Excited to introduce LLM-First Search (LFS) - a new paradigm where the language model takes the lead in reasoning and search!

LFS is a self-directed search method that empowers LLMs to guide the exploration process themselves, without relying on predefined heuristics or fixed exploration schedules.

Why it matters:
🔍🤖 Self-directed: The model strikes a balance between exploration and exploitation using its own internal scoring system.
🔄🧠Adaptive: Automatically adjusts to task difficulty with no tuning required.
📈🏆Stronger performance: Achieves higher success rates on reasoning tasks.
⏱️📉Efficient: Outperforms other LLM-augmented strategies in compute usage.
⚖️🚀Scalable: Gains amplify with stronger models and more compute.

A big thanks to both @_rockt and @robertarail for their exceptional supervision!

[Kale-ab Tessera @KaliTessera]

Can LLM agents coordinate in long-horizon, open-ended worlds?

We evaluate 13 modern LLMs in a new benchmark where agents must work together to explore, communicate, trade resources, craft tools, build structures, and fight mobs.

TL;DR: Most agents struggle, averaging only ~6% normalised return. Yet on the hardest setting, zero-shot Gemini 3.1 Pro performs comparably to the best MARL agent trained for 1 billion environment steps.

More broadly, we find coordination is a distinct bottleneck beyond long-horizon task competence, with communication having the largest effect in our harness ablations. 🧵👇

[Jakob Foerster @j_foerst]

BOLD is hiring 🚀
Right now we are looking for two brilliant individuals to help us build the machine that invents the future. If you are up for a challenge and thrive in a fast moving, collaborative, mission driven environment, this is for you..!
Links and more info👇

[British Open-ended Learning and Discovery Lab @bold_lab_ai]

We hope you’ve enjoyed a sneak peek of work from BOLD and our collaborators at #ICML2026! See below for a full summary of where you can find us this week:

▶️ (Poster) Procedural Generation of Algorithm Discovery Tasks in Machine Learning, Hall A #1803, Tuesday 10:30 - 12:15, led by @AlexDGoldie

▶️ (Poster) h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learning, Hall A #2704, led by Alesia Ivanova @sumeetrm

▶️ (Poster) Goal-Conditioned Agents that Learn Everything All at Once, Hall A #310, Tuesday 14:00 - 15:45, led by @mitrma

▶️ (Poster) Rubric Curriculum RL: Exploiting the Generation-Verification Gap in Creative Writing, Hall A #2605, Wednesday 14:30 - 14:15, led by Tejas Krishnan @sumeetrm

▶️ (Poster) Evolution Strategies at the Hyperscale, Hall A #3712, Thursday 10:30 - 12:15, led by @bidiptas13 @JuanDuquevan Mattie Fellows

▶️ (Poster) Dreaming in Code for Curriculum Learning in Open-Ended Worlds, Hall A #213, Wednesday 17:00 - 18:45, led by @k_mitsides

▶️ (Poster) Evolution Strategies at the Hyperscale, Hall A #3712, Thursday 10:30 - 12:15, led by @bidiptas13 @JuanDuquevan Mattie Fellows

▶️ (Poster) The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind, Hall A #3504, Thursday 14:30 - 16:15, led by @_andreilupu

▶️ (Poster) LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning, Hall A #1705, Thursday 14:30 - 16:15, led by @sumeetrm @DanielNichols10 @CharlieLondon02 Peggy Li Fabio Pizzati

▶️ (Talk) Superhuman Scientific Discovery, RLxF Worskhop, Friday 15:30 - 16:00, by @robertarail

▶️ (Panel) RLxF Worskhop, Friday 16:00 - 17:00, by @robertarail

▶️ (Workshop) Amortising Bayesian Experimental Design for Sequential Information Gathering in LLMs, FoGen Workshop, Friday, led by @jakobhartmann99 James Harvey Jhonathan Navott

▶️ (Workshop) Elicitation Format Drives Divergent LLM Geopolitical Forecasts, AI Forecasting Workshop, Saturday, led by @hariharansuhas @michalbravansky

▶️ (Workshop) EGGROLL-IPO: Pluralistic Alignment via Decentralised Post-Training with Population Preferences, Pluralistic Alignment Workshop, Saturday, led by @alfie_lamerton

▶️ (Workshop Spotlight) Abstraction for Offline Goal-Conditioned Reinforcement Learning, DEMO Workshop, Saturday, led by @ClarisseWibault

[British Open-ended Learning and Discovery Lab @bold_lab_ai]

Over this week, we'll be posting about papers from BOLD on show at the upcoming #ICML2026. Up first: LEO, led by @mitrma!

Typical approaches in goal-conditioned RL select a single goal that is completed later in the episode. This paper instead asks a simple question: what if *every* goal is used for labelling (and how could we do that efficiently)?

Check it out: https://t.co/zTLt3xL1Ll

[British Open-ended Learning and Discovery Lab @bold_lab_ai]

🪩 Next #ICML2026 paper: DiscoGen, led by @AlexDGoldie. DiscoGen is a procedural generator of algorithm discovery problems for AI research agents, supporting the creation of over 100 billion diverse tasks! Tasks vary across many axes, such as their field of machine learning or datasets they use. They can even support different evaluations, such as the time an algorithm takes to train (speedrunning ⚡️), its energy usage (efficiency 🌱) or its performance 💪!

Check out the paper at: https://t.co/W6YB9CwFno

Give the code a look: https://t.co/30oEbzJEgr

Or install the DiscoGen package for your research: pip install discogen

[British Open-ended Learning and Discovery Lab @bold_lab_ai]

🛌💭Our next ICML paper is Dreaming in Code, led by @k_mitsides. In this work, LLMs are used as architects of experience: writing worlds in code that scaffold an RL agent towards long-horizon skills it could not discover alone!

Check out the project page at: https://t.co/irhsoujonm

And the paper: https://t.co/t2VzsvLYUU

[Aneesh Muppidi @aneeshers]

Introducing Real-time RL.

In the real world, time isn't free. The environment keeps "moving" even when you're computing your next action.

We show how RL agents can learn to adaptively think in real-time games.

1/🧵

[Nathan Herr @naitherr]

You can find both the paper and the code here:
📄 Paper: https://t.co/e0BzZEHxgZ
💻 Code: https://t.co/7fttuSqfZE

[Nathan Herr @naitherr]

Excited to introduce LLM-First Search (LFS) - a new paradigm where the language model takes the lead in reasoning and search!

LFS is a self-directed search method that empowers LLMs to guide the exploration process themselves, without relying on predefined heuristics or fixed exploration schedules.

Why it matters:
🔍🤖 Self-directed: The model strikes a balance between exploration and exploitation using its own internal scoring system.
🔄🧠Adaptive: Automatically adjusts to task difficulty with no tuning required.
📈🏆Stronger performance: Achieves higher success rates on reasoning tasks.
⏱️📉Efficient: Outperforms other LLM-augmented strategies in compute usage.
⚖️🚀Scalable: Gains amplify with stronger models and more compute.

A big thanks to both @_rockt and @robertarail for their exceptional supervision!

[Nathan Herr @naitherr]

We evaluated LFS using three key metrics:
- Win Rate: Percentage of problems solved across 5 runs per instance.
- Efficiency: Win Rate normalised by total tokens used.
- AUP Score: Area Under Profile, an aggregate measure of performance across varying task difficulties.

Key Findings:
- LFS outperforms MCTS, the next best method, across both tasks.
- Better scalability: Performance holds up as task difficulty increases.
- Higher efficiency: Solves more problems using fewer tokens.
- Improves with scale: Benefits compound with stronger models and more compute.