Arif Ahmad ✈️ CVPR 🔜 SF

We are in the world model era now. Currently IFM. Prev. @GoogleDeepMind and @Nvidia

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

We need more senior researchers camping out at their posters like this. Managed to catch 10 minutes of Alyosha turning @anand_bhattad’s poster into a pop-up mini lecture. Extra spark after he spotted @jathushan. Other folks in the audience: @HaoLi81 @konpatp @GurushaJuneja.

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@joey072shx I think both our accounts were compromised in a similar way. We also discussed you in our thread here. https://t.co/yby9EtQGl8

[Arif Ahmad @arif_ahmad_py]

Thanks @tombukic. The X folks are a beast. @singhai responded a few minutes ago and I now have full access to my old account now. Wouldn't have been possible without you tagging the right people. If there's any possible way I could honour your kindness I'd be glad to. Please feel free to reach out to me in future for anything!

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@joey072shx Congrats Joey!

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@Yurochkin_M Awesome work, @Yurochkin_M !

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@prajdabre @JeffDean Lmao. I hope I can get fresh produce from your farm!

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@simi_97k Great to see this. Now I have a public talk to point to when I tell someone to check out your work!

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@tw_killian @BYU Congratulations again @tw_killian! @BYU and it's students sure are lucky to have you.

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@sumanthd17 Gotta check some keyboard places. Thanks for the Intel!

[Gurusha Juneja @GurushaJuneja]

Poster session at 10:30 AM Hall A. Please come by poster #203 on training Process Reward Models. @icmlconf

[Arul Murugan @_arulm_]

Who gets access to frontier AI is now a geopolitical question. But another question matters too: which economies are most exposed to it?
Our new paper answers this by measuring national exposure to frontier AI across 141 countries.
(1/8)

[Gurusha Juneja @GurushaJuneja]

Automated discovery is a search problem over combinatorially large spaces. It requires reasoning to explore promising ideas while handling long contexts.

Introducing ARTS, a search algorithm that (i) reasons to guide exploration and (ii) test-time trains to handle long context

[Lakshya A Agrawal @LakshyAAAgrawal]

Learning from rich textual feedback (errors, traces, partial reasoning) beats scalar reward alone for LLM optimization. GEPA demonstrated this for context-space optimization (prompts and agent harnesses), delivering frontier results at a fraction of the cost of RL.

But context-only optimization is bounded by the base model's capability ceiling; weight updates can reach further.

Very excited about this new line of work on Fast-Slow Training (FST), which interleaves context and model weight optimization!

The idea is a clean division of labor between two interleaved loops:
🔹 Fast loop (context): GEPA reads rich rollout feedback updating the context layer. The context becomes a fast-updating scratchpad of what the model needs to know about this task, right now.
🔹 Slow loop (model parameters): RL updates the model's parameters conditioned on the evolving context. Because the prompt already carries task-specific nuances, the model parameters are freed from absorbing them and focus on what actually generalizes across tasks and pushes the frontier.

⦁ 3× more sample-efficient than RL on math, code, and physics reasoning
⦁ ~70% lower KL divergence from base at matched accuracy
⦁ Plasticity preserved: FST checkpoints respond better to additional RL on new tasks than RL-only ones
⦁ Continual learning across changing tasks (HoVer → CodeIO → Physics) where RL stalls the moment the task switches

FST is a direction towards:
⦁ Addressing RL's pain points: entropy collapse, sparse rewards, long-horizon exploration
⦁ Providing a clean channel for rich feedback into weight updates
⦁ Demonstrating model-harness co-evolution
⦁ Discovery: Using fast context updates for broad exploration, while leveraging a continually improving model.

Check out the full thread below:

[Rishabh Tiwari @rish2k1]

Very excited about this line of research of fast-slow learning,

1) potential to solve a lot of issues with current RL (eg. entropy collapse, sparse rewards)

2) an intuitive way of incorporating rich feedback with RL

3) provides a way to transfer knowledge of text-only based learning into the model

4) a great candidate for model-harness co-evolution, seeing a lot discussion on X lately about future models developing their own harness.

5) most importantly, can imagine these kinds of algorithms to be more suitable candidates for discovery that requires both extreme exploration but at the same time improving the underlying model capabilities.

and much more ...

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@tw_killian > I don’t have any expectations that my writing will catch much attention.
You have my attention. 🫡

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

@teortaxesTex Thanks for the kind words @teortaxesTex . Many more projects cooking in the World Model team at IFM (MBZUAI).

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

Great to see @seungwookh's recent work on pre-pre-training LMs! Always refreshing to see people still working on good ol' representation learning.

[Seungwook Han @seungwookh]

Can language models learn useful priors without ever seeing language?

We pre-pre-train transformers on neural cellular automata — fully synthetic, zero language. This improves language modeling by up to 6%, speeds up convergence by 40%, and strengthens downstream reasoning.

Surprisingly, it even beats pre-pre-training on natural text!

Blog: https://t.co/Pni0RsIcxL

(1/n)

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

Also the way it just zooms into the device / object it's interacting with and makes it the center focus. This is too good!

[Arif Ahmad ✈️ CVPR 🔜 SF @tensor_mechanic]

The interactions here are definitely something I haven't seen in any previous models before. Genie 3 kinda hints it tries to do it. But the interactions here feel more natural than the ones by Genie. This is definitely something more people need to see. Also, it would lead to amazing dynamics models if the interaction capacity and diversity alone is scaled. I would like to try how good it really is.

[Moonlake @moonlake]

Introducing a world built by the Moonlake's world model. 🏙️

Most world models only allow for a limited action space.

Moonlake maintains multimodal states across physics, appearance, geometry, and casual effects and predict how they evolve under different actions. 👇