Maxim Bobrin

Novelty Search & Neural Evolution
Q: "What makes someone truly creative? What is the main driving force?"
BSc & MSc in Mathematics

[Maxim Bobrin @maxsbob21]

Lots of other interesting insights were made during this work. Thanks @machinestein, see everyone interested in Seoul!

[Arip @machinestein]

ICML 2026: Zero-Shot Off-Policy Learning

Distribution ratios play a central role in off-policy reinforcement learning.

We show that using only one behavioral foundational model, you can obtain its ratio to other policies for free, without any importance sampling or min-max optimization (DICE algorithms).

We show that forward-backward representations store this ratio and can be used for better zero-shot adaptation, solving off-policy issues on the fly.

[Maxim Bobrin @maxsbob21]

@zhaisf @geoffreyhinton Even without reading slides, the most probable explanation is that mnist contains all required features for predicting 3 based on other numbers (some of which look similar to 3). It would be more interesting to take most non representative digit and check OOD

[Maxim Bobrin @maxsbob21]

@its_vayishu Am i correct that you store smth like a latent buffer that acts as a memory and based on this dynamic buffer predictive model outputs next states? And you show that there is some structure arising from memory?

[Maxim Bobrin @maxsbob21]

@willccbb How is this different from Unsipervised Environment design? In RL there are already lots of papers and i assume some folks already managed to apply ideas from UED. Seems like this paper is just the same findings albeit from other perspective

[Maxim Bobrin @maxsbob21]

@Amank1412 Most ppl would say that they are optimizing their work, but in reality will spend more free time for soom scrolling

[Maxim Bobrin @maxsbob21]

@heyrimsha can we pla stop already? There are a lot of services for those kind of things… Research community needs to heal asap

[Maxim Bobrin @maxsbob21]

@daniel_mac8 they tested this on public games. Private is much harder

[Maxim Bobrin @maxsbob21]

@agenticasdk gg

[Maxim Bobrin @maxsbob21]

@KeyTryer You can check several replays and see that gemini is the only model which tries to understand what to do through reasoning traces, while other models just return action

[Maxim Bobrin @maxsbob21]

@loveofdoing How you came up with this approach? somekind of blogpost would be beneficial

[Maxim Bobrin @maxsbob21]

@ChenTessler Seems like there is no method that can do this even in simulation without spending hours on RL only for this single level

[Maxim Bobrin @maxsbob21]

@pengzhangzhi1 @Anru_Zhang @AlexanderTong7 Where to find recordings?

[Maxim Bobrin @maxsbob21]

@itsolelehmann what is 56%? How it is measured? Agent can basically just find some adverserial solution, get improvement on metric that he chose by himself and you will never notice. dafuq?

[Maxim Bobrin @maxsbob21]

@mikeknoop But what about NetHack challenge? It measures the same skills ARC-AGI-3 aims to check

[Maxim Bobrin @maxsbob21]

@deliprao @karpathy As far as i understand, the program.md needs to be different for other domains (e.g for RL/robotics)

[Arip @machinestein]

Zero-Shot Off-Policy Learning

Behavioral foundation models are pretrained on large, reward-free transition datasets. At deployment time, they can be "prompted" to infer a policy for a new reward in a zero-shot manner, without any fine-tuning.

This falls under offline or off-policy RL: once the inferred policy is executed, its state-action visitation may diverge from the dataset, leading to distribution shift, value overestimation, and other typical off-policy issues.

The missing ingredient is a principled off-policy correction—specifically, stationary occupancy (density-ratio) correction.

In this paper, we show that by using Forward–Backward successor representations, this density-ratio correction can also be performed in a zero-shot manner!

Paper: https://t.co/6myZI8G2Ty

Code: https://t.co/JFZ3fybmBe

[Maxim Bobrin @maxsbob21]

@ChenTessler Is this a sample video replicating some motion from training set of AMASS? If so, how it was prompted as inference time? Or agent was trained with this particular option only?

[Arip @machinestein]

While we are going back to the era of research…
Introducing 𝗗𝗲𝗲𝗽 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝗺𝗲𝗻𝘁 𝗦𝘂𝗽𝗲𝗿𝘃𝗶𝘀𝗶𝗼𝗻 (𝗗𝗜𝗦) – a new learning method for recursive reasoning.

DIS builds on the elegant Tiny Recursive Model (TRM)(@jm_alexia) but makes recursion radically simpler:
- 𝟏𝟖× 𝗳𝗲𝘄𝗲𝗿 𝗳𝗼𝗿𝘄𝗮𝗿𝗱 𝗽𝗮𝘀𝘀𝗲𝘀
- 𝗡𝗼 𝗵𝗮𝗹𝘁𝗶𝗻𝗴 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝗺
- And a tiny 0.8M-parameter model reaching 24% accuracy on ARC-AGI-1 (@arcprize)

Paper: https://t.co/QM6hNFMm5M

Code: https://t.co/d4nhzvBz4G

[Ilya Zisman @suessmannn]

🔥 Zero-shot generalization is the dream: adapt instantly, no fine-tuning. It's why LLMs blew up—but it's not just a language modeling thing. It’s happening in RL too.

🚨 @maxsbob21's new paper dives deep into zero-shot RL under shifting dynamics—and why current methods break.