(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples?
We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by!
In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks.
Read our paper here: https://t.co/hbXWjSdVkQ
Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias
More in 🧵👇
Qiang Liu, Chris Oates, and I are writing a monograph on Probabilistic Inference and Learning with Stein’s Method, and we’d love to get your feedback on the first draft
today, we’re open sourcing the largest egocentric dataset in history.
- 10,000 hours
- 2,153 factory workers
- 1,080,000,000 frames
the era of data scaling in robotics is here. (thread)
📢 Job alert
We are looking for a Postdoctoral Fellow to work with @ArthurGretton on creating statistically efficient causal and interaction models with the aim of elucidating cellular interactions.
⏰Deadline 27-Aug-2025
ℹ️ https://t.co/4B4UIu1uh3
(1/6) Can Autoregressive Models (ARMs) go beyond fixed or random generation orders and learn the optimal order to generate new samples?
We believe so! We are excited to present our paper, “Learning-Order Autoregressive Models with Application to Molecular Graph Generation,” at ICML 2025 in Vancouver on July 18th. Feel free to drop by!
In our new paper, we introduce Learning-Order Autoregressive Models (LO-ARMs), which are trained to find a consistent, context-dependent generation order on their own. Our model matches and surpasses state-of-the-art results on two major molecular graph generation tasks.
Read our paper here: https://t.co/hbXWjSdVkQ
Joint work with @thjashin, Nicolas Heess, @ArthurGretton, Michalis K. Titsias
More in 🧵👇
(6/6) Finally, can LO-ARMs generate "better" molecules?
Yes, they can. On both the QM9 and ZINC250k datasets, LO-ARMs match or exceed state-of-the-art (SOTA) results.
Performance was evaluated across key metrics for distribution similarity and drug-likeness.
(5/6) Can LO-ARMs learn consistent and meaningful generation orders on their own?
Yes — and the strategy/generation order they discover is surprisingly intuitive.
On two major molecular generation tasks (QM9 and ZINC250k), our models learned a consistent, two-phase process:
1. Build the Skeleton: First, generate the chemical bonds.
2. Infill the Atoms: Then, add the atoms to the skeleton.
We find that 99% of the new samples follow this generation order.
Here’s an example of this in action for generating a ZINC250k molecule. We start with a blank slate where all tokens are masked, and in each step, the model adds one new piece — either an atom or a bond, until the molecule is complete.
@ZoubinGhahrama1@geoffreyhinton Just took the Gatsby courses last year (now they are split to two modules approximate inference and unsupervised learning), and the courses are incredibly helpful. I even read your matlab code!😁
📢 We have an opportunity for students to join our PhD programme in Theoretical Neuroscience and Machine Learning this September.
Application deadline is 27 May 2025.
Information & how to apply 👉 https://t.co/nbdHOqwGSA
We are hiring a student researcher at Google DeepMind to work on fundamental problems in discrete generative modeling!
Examples of our recent work:
masked diffusion: https://t.co/lLQxO6Yw5D
learning-order AR: https://t.co/t8D7xQMH07
If you find this interesting, please send an email to: {jiaxins,mtitsias} AT google DOT com
Excited to share details of AlphaGeometry2 (AG2), part of the system that achieved silver-medal standard at IMO 2024 last July! AG2 now has surpassed the average gold-medalist in solving Olympiad geometry problems, achieving a solving rate of 84% for all IMO geometry problems over the last 25 years, compared to 54% previously! Figure is a crazy problem solved elegantly by AG2 (for the first time according to our knowledge!). Will follow up with more details later in a thread.
Paper: https://t.co/ZC7ohtFMiD
Nature article: https://t.co/PBn3ojIDFg
Authors: Yuri Chervonyi, Trieu H. Trinh, Miroslav Olsak, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V. Le, Thang Luong
Special thanks to Dawsen Hwang, Edward Lockhart, and Steven Creech for contributions to the development of AlphaGeometry2. We would also like to thank Yifeng Lu, Henryk Michalewski, Ed Chi, David Silver, Pushmeet Kohli, and Demis Hassabis for their thoughtful discussions, help and support.