Timofei Gritsaev

Researcher at @bayesgroup | Amortised Sampling & Generative models

[Mishan Aliev @ne_mishan]

My course Deep Learning 2 is back on Sep 1 🚀

Updated materials, modern DL topics, and guest lectures from researchers & industry experts.

Everything is open-source, and the repo just passed 100⭐
https://t.co/r4BGUDtcfr

[Timofei Gritsaev @gritsaev]

Thank you for the kind words, and thank you for your work!

On the differences, as I see them:

1/ The two algorithms target slightly different distributions. Ours is derived for sampling from R/Z, which corresponds to entropy-regularized RL. FlowRL places the reference model inside the target, exp(βr)·π_ref/Z, which turns the entropy term into a KL toward π_ref — entropy regularization is recovered when π_ref is uniform. Adapting Ent-PPO to your setting is mostly a matter of redefining the terminal reward accordingly. Worth noting that in autoregressive generation the DAG is a tree, so P_B ≡ 1 and the intermediate rewards log P_B vanish; what remains from the soft-RL derivation is the α = 1 entropy weight and the analytic KL trust region against π_old.

2/ Ent-PPO avoids learning a partition function, while FlowRL learns one. Avoiding log Z can help, since it is a globally consistent quantity that is hard to estimate at scale, and training can be inefficient as long as the estimate is inaccurate. The cost is losing the off-policy freedom, for example, a replay buffer.

3/ As for practical trade-offs for long-horizon LLM reasoning, I honestly don't know — our papers consider different applications, and this is worth investigating.

Overall I'd say the papers address different problems: general discrete sampling versus LLM reasoning, but share the same high-level formulation and both build on GFlowNet theory.

Thanks again for reaching out!

[Timofei Gritsaev @gritsaev]

🍌🍌🍌
1/ The original GFlowNets paper tried PPO for sampling, but it failed. We figured out why and fixed it. And now PPO beats all objectives on standard problems including molecular graph generation in large spaces.

[Timofei Gritsaev @gritsaev]

@josephdviviano @pleuviei thank you very much! i am happy you have found this work useful!

[Ildus Sadrtdinov @irsadrtdinov]

💨 Did you know neural networks can behave like literal ideal gases? 💨
In our #IJCAI2026 paper, we show that at stationarity, scale-invariant NNs surprisingly obey... the ideal gas law 🤯
Thermodynamics helps us to understand how train hyperparams shape final solutions!
🧵👇1/8

[Timofei Gritsaev @gritsaev]

8/ So, first, Ent-PPO outperforms value-based approaches, and, second, we assume Ent-PPO shines in more challenging problems! See a detailed Ent-PPO derivation and discussion of important implementation choices in the paper:

📄 Paper: https://t.co/BvEDa180Kh
💻 Code: https://t.co/NKex2tyhkh

This is a joint work with my amazing coauthors, @akzykova @dtiapkin @nvimorozov!

[Timofei Gritsaev @gritsaev]

7/ We then consider molecular graph generation problems: sEH and QM9. These are large problems with over 10^18 and 10^180 states, respectively. Our results are:
1️⃣Ent-PPO achieves the best result!
2️⃣The gap in ELBO between Ent-PPO and value-based methods grows with problem difficulty!

Caption: ELBO and EUBO are common metrics measuring inner- and outer-mode coverage, respectively. A higher ELBO and a lower EUBO indicate more accurate sampling.

[Alexander Shabalin @a__shabalin]

Why Gaussian diffusion models fail on text data and how to prevent it?

☝️ We find that discrete-like latent spaces are fundamentally bad for continuous diffusions.
☝️ We explain what happens inside, and why self-conditioning and other heuristics improve generation.

🧵👇 1/7

[Timofei Gritsaev @gritsaev]

Great work by my dear colleagues!

[Nikita Morozov @nvimorozov]

(1/n) Excited to share our latest work “Learning Shortest Paths with Generative Flow Networks”!

We uncover a novel theoretical connection between flow minimization in GFlowNets and finding shortest paths, and develop a learning approach that rivals SOTA in solving Rubik's Cubes!

[Nikita Morozov @nvimorozov]

Got asked in a review for my ICML paper whether "there are realistic tasks where one needs to sample from a probability distribution given by its unnormalized density" (rephrased for anonymity). Are we cooked?

[Vedant Shah @veds_12]

LOTs of discourse lately about the correctness of the KL-regularization term used in RLVR fine-tuning of LLMs.

Which estimator to use? Whether to add it to the reward or loss? What’s even the difference? 🤔

In our new preprint, we evaluate these choices empirically. 🧵

1/n

[Daniil Tiapkin @dtiapkin]

While frontier labs are announcing their new models, we also want to be part of this parade. So, we’re happy to announce gfnx – a JAX-first library with environments and a single-file baseline implementation for GFlowNet research.

[Timofei Gritsaev @gritsaev]

@nvimorozov

[Timofei Gritsaev @gritsaev]

7/ Huge thanks to my amazing co-authors: @nvimorozov,@ktamogashev, @dtiapkin, @svsamsonov, Alexey Naumov, Dmitry Vetrov, and @FelineAutomaton! The full version: https://t.co/KOTbwuXkr4

[Timofei Gritsaev @gritsaev]

1/ Can we efficiently learn the destruction process of diffusion samplers? Can we learn not just the drift, but also the variance for all transition kernels? – We answer YES in our recent paper “Adaptive Destruction Processes for Diffusion Samplers” (Oral at NeurIPS 2025 FPI Workshop).

[Timofei Gritsaev @gritsaev]

6/ But it’s not limited to small-scale problems. We successfully use these methods to improve outsourced sampling: Bayesian posterior inference in latent spaces of generative models.