Zhe Wang

Frontier AI @GoogleDeepMind. Generative modelling, Robotics and AI4Science.

[Zhe Wang @zhexwang]

(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples?

We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by!

In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks.

Read our paper here: https://t.co/hbXWjSdVkQ

Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias

More in 🧵👇

[Lester Mackey @LesterMackey]

Qiang Liu, Chris Oates, and I are writing a monograph on Probabilistic Inference and Learning with Stein’s Method, and we’d love to get your feedback on the first draft

[Eddy Xu @ CVPR @eddybuild]

today, we’re open sourcing the largest egocentric dataset in history.

- 10,000 hours
- 2,153 factory workers
- 1,080,000,000 frames

the era of data scaling in robotics is here. (thread)

[Gatsby Computational Neuroscience Unit @GatsbyUCL]

📢 Job alert
We are looking for a Postdoctoral Fellow to work with @ArthurGretton on creating statistically efficient causal and interaction models with the aim of elucidating cellular interactions.

⏰Deadline 27-Aug-2025
ℹ️ https://t.co/4B4UIu1uh3

[Jiaxin Shi @thjashin]

Nice writeup on CTMC and discrete diffusions

[Zoubin Ghahramani @ZoubinGhahrama1]

Google DeepMind has been leading the revolution in Foundation Models for Robotics. Come join us. We are hiring!

[Arthur Gretton @ArthurGretton]

Accelerated Diffusion Models via Speculative Sampling, at #icml25 !

at 16:30 Tuesday July 15 poster E-3012

https://t.co/GGuwJxdIFf

@ValentinDeBort1 @agalashov @ArnaudDoucet1

[Arthur Gretton @ArthurGretton]

Distributional diffusion models with scoring rules at #icml25

Fewer, larger denoising steps using distributional losses!

Wednesday 11am poster E-1910

https://t.co/tIfeAWYj6V

@agalashov
@ValentinDeBort1
Guntupalli
@zhouguangyao
@sirbayes
@ArnaudDoucet1

[Zhe Wang @zhexwang]

@YuchenZhu_ZYC Thanks! Shall we catch up in person if you're also at ICML? Feel free to let me know if you have any questions.

[Zhe Wang @zhexwang]

(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples?

We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by!

In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks.

Read our paper here: https://t.co/hbXWjSdVkQ

Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias

More in 🧵👇

[Zhe Wang @zhexwang]

@GatsbyUCL @KevinHanHuang1 Congratulations!! @KevinHanHuang1

[Zhe Wang @zhexwang]

(6/6) Finally, can LO-ARMs generate "better" molecules?

Yes, they can. On both the QM9 and ZINC250k datasets, LO-ARMs match or exceed state-of-the-art (SOTA) results.

Performance was evaluated across key metrics for distribution similarity and drug-likeness.

[Zhe Wang @zhexwang]

(5/6) Can LO-ARMs learn consistent and meaningful generation orders on their own?

Yes — and the strategy/generation order they discover is surprisingly intuitive.

On two major molecular generation tasks (QM9 and ZINC250k), our models learned a consistent, two-phase process:

1. Build the Skeleton: First, generate the chemical bonds.
2. Infill the Atoms: Then, add the atoms to the skeleton.

We find that 99% of the new samples follow this generation order.

Here’s an example of this in action for generating a ZINC250k molecule. We start with a blank slate where all tokens are masked, and in each step, the model adds one new piece — either an atom or a bond, until the molecule is complete.

[Zhe Wang @zhexwang]

@ZoubinGhahrama1 @geoffreyhinton Just took the Gatsby courses last year (now they are split to two modules approximate inference and unsupervised learning), and the courses are incredibly helpful. I even read your matlab code!😁

[Gatsby Computational Neuroscience Unit @GatsbyUCL]

📢 We have an opportunity for students to join our PhD programme in Theoretical Neuroscience and Machine Learning this September.

Application deadline is 27 May 2025.

Information & how to apply 👉 https://t.co/nbdHOqwGSA

[Jiaxin Shi @thjashin]

We are hiring a student researcher at Google DeepMind to work on fundamental problems in discrete generative modeling!

Examples of our recent work:
masked diffusion: https://t.co/lLQxO6Yw5D
learning-order AR: https://t.co/t8D7xQMH07

If you find this interesting, please send an email to: {jiaxins,mtitsias} AT google DOT com

[Zhe Wang @zhexwang]

@karl_tuyls Congratulations Karl!

[Thang Luong @lmthang]

Excited to share details of AlphaGeometry2 (AG2), part of the system that achieved silver-medal standard at IMO 2024 last July! AG2 now has surpassed the average gold-medalist in solving Olympiad geometry problems, achieving a solving rate of 84% for all IMO geometry problems over the last 25 years, compared to 54% previously! Figure is a crazy problem solved elegantly by AG2 (for the first time according to our knowledge!). Will follow up with more details later in a thread.

Paper: https://t.co/ZC7ohtFMiD
Nature article: https://t.co/PBn3ojIDFg

Authors: Yuri Chervonyi, Trieu H. Trinh, Miroslav Olsak, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V. Le, Thang Luong

Special thanks to Dawsen Hwang, Edward Lockhart, and Steven Creech for contributions to the development of AlphaGeometry2. We would also like to thank Yifeng Lu, Henryk Michalewski, Ed Chi, David Silver, Pushmeet Kohli, and Demis Hassabis for their thoughtful discussions, help and support.