Jesse Allardice

Building Interactive World Models @Odysseyml | Previously Apple & Physicist

[Odyssey @odysseyml]

World models can now create imagined experiences for AI—environments where agents continuously learn, adapt, and improve.

We suspect multi-agent interaction may be a critical ingredient for recursive AI and general intelligence.

https://t.co/rIUKZZVl9f

[Jesse Allardice @JRAllardice]

Going to be with the @odysseyml crew at CVPR this week. DM if you’d like to chat World Models!

[Jeff Hawke @jeffrey_hawke]

@odysseyml is at CVPR this week in Denver! If you want to chat about our world model research, DM me or email [email protected]

[Odyssey @odysseyml]

Introducing Agora-1, a multi-agent world model.

Multiple participants—human or AI—can now interact inside the same world simulation, all in real-time.

Try our playable research preview today, with Agora-1 simulating a multiplayer GoldenEye deathmatch!

[Jesse Allardice @JRAllardice]

Multimodal World Models here we come!!!

[Oliver Cameron @olivercameron]

Introducing Starchild-1 from @odysseyml, the first ever real-time multimodal world model.

This a model that can generate interactive simulations of the world that you can—for the first time ever—hear.

Starchild-1 represents a big step towards a general-purpose world simulator.

[Oliver Cameron @olivercameron]

Introducing Starchild-1 from @odysseyml, the first ever real-time multimodal world model.

This a model that can generate interactive simulations of the world that you can—for the first time ever—hear.

Starchild-1 represents a big step towards a general-purpose world simulator.

[A. Hamdi Guzel @ahguzelUK]

1/ Really proud of what the team pulled together on this one. Here's what we developed to make PROWL work 👇

[Odyssey @odysseyml]

Introducing PROWL!

We’ve built RL agents that explore game environments, tasked with discovering failures in world models across physics, visuals, and actions.

Those failures then become training data in an automated loop that advances world model performance.

[Jesse Allardice @JRAllardice]

Check out the amazing real-time World Model we’ve been cooking at @odysseyml!

The scale of compute we hit with this model really shines through here. We’ve totally moved beyond just Language Models and are now in the era of World Model scaling.

[Odyssey @odysseyml]

It’s time to go beyond language models.

Introducing Odyssey-2 Max, our most powerful world model yet. It materially advances the SOTA in physical accuracy.

This is a big step toward models that simulate and interact with the world in real time.

A new intelligence entirely!

[Jesse Allardice @JRAllardice]

@odysseyml So awesome to see scale can make all the difference between physics that can pass for looking reasonable to one that is super realistic!

[Jesse Allardice @JRAllardice]

Check out this awesome work on video tokenization and generation led by @andrew_atanov during his internship at Apple. I’m super excited by the progress and results we made here. Really great to see non-grid tokenizers applied to video!

[Andrei Atanov @andrew_atanov]

Are all videos worth the same number of tokens? Whether rich in motion or visually minimal, standard 3D-grid tokenizers treat them equally. We present VideoFlexTok, which represents videos using a flexible-length, coarse-to-fine sequence of tokens.

Page: https://t.co/aDbvsz2Arw
Demo: https://t.co/aM0BrPzfSq
Paper: https://t.co/e8g7nXrLCn

1/n

[Jesse Allardice @JRAllardice]

🚀 Huge shout-out to the team for open-sourcing l3m! We’ve been using it in our own image generation research and love how seamlessly everything clicks together. Excited to see what others build with it!

[Victor Turrisi @victorturrisi]

Super excited to share l3m 🚀, a library for training large multimodal models, which we used to build AIM and AIMv2. Massive thanks to @alaa_nouby @DonkeyShot21 Michal Klein @MustafaShukor1 @jmsusskind and many others.

[Oğuzhan Fatih Kar @oguzhanthefatih]

🚨 Research Internship opportunity at Apple 

We’re looking for interns to push the limits of multimodal AI agents!

📍 Santa Clara Valley 🇺🇸 & Zurich 🇨🇭
🗓️ Start: asap

Send CV + representative work to [email protected]

Also apply: https://t.co/AS7Nlfx3Pl

[Jesse Allardice @JRAllardice]

@CSProfKGD Hope you enjoyed it!

[Jesse Allardice @JRAllardice]

Woo our latest paper made it into @TheAITimeline week of AI wrap up!

[The AI Timeline @TheAITimeline]

🚨This week's top AI/ML research papers:

- Mixture-of-Recursions
- Scaling Laws for Optimal Data Mixtures
- Training Transformers with Enforced Lipschitz Constants
- Reasoning or Memorization?
- How Many Instructions Can LLMs Follow at Once?
- Chain of Thought Monitorability
- Aime
- Cameras as Relative Positional Encoding
- π^3
- Language Models Improve When Pretraining Data Matches Target Tasks
- SpatialTrackerV2
- Your LLM Knows the Future
- MindJourney
- REST
- The Devil behind the mask
- Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation

overview for each + authors' explanations
read this in thread mode for the best experience

[Afshin Dehghan @afshin_dn]

Yesterday we shared our latest work on pretraining data curation. What if we stop guessing which data is “good” and directly match pretraining data to the benchmarks we care about?

📄 https://t.co/Mvea0rJ8vc

#AIResearch #llm #DataCuration #Pretraining #ScalingLaws

[Simo Ryu @cloneofsimo]

"Aggressive Filtering aint good for larger training"
Similar find also at

[𝚐𝔪𝟾𝚡𝚡𝟾 @gm8xx8]

Language Models Improve When Pretraining Data Matches Target Tasks

BETR (Benchmark-Targeted Ranking) picks training data that closely matches benchmark tasks, instead of relying on vague quality heuristics:

- Embed benchmark train examples + a small document sample
- Score by similarity
- Train FastText to rank full corpus

Result: 2.1× compute multiplier over DCLM-Baseline (4.7× over unfiltered). Outperforms on 9/10 Core tasks, scales cleanly from 10¹⁹ to 10²² FLOPs.

Two use modes:
- Target-Core: SOTA on eval benchmarks, but weak on unseen
- Target-Noncore: strong generalization, even to held-out tasks

Also:
- Scaling law insight: optimal filtering rate grows with model size → small models = strict filters; large models = broader data
- BETR shifts model capabilities with precision: you get what you target

[Rosinality @rosinality]

https://t.co/V2mehMWqbn

Language Models Improve When Pretraining Data Matches Target Tasks

Filtering pretraining data by finding texts similar to benchmarks. This goes beyond simple benchmark hacking and is much more nuanced. The paper reports that when filtering using only popular benchmarks, hacking occurs, but when using more diverse benchmarks, the approach appears to be competitive. Given that benchmark-targeted data filtering is common practice, this demonstrates the risks of pursuing it too much.

The analysis also shows that lighter filtering tends to be better as model sizes increase. This trend is weaker for smaller models, suggesting this could be another factor that distinguishes larger models from smaller ones.

[Jesse Allardice @JRAllardice]

Forget vague heuristics, match your pretraining data to your benchmarks!

Check out our latest work, BETR, which does exactly that and delivers a 2× compute multiplier. We also use scaling laws to reveal that when you scale up model size and compute, milder filtering is optimal.

[David Mizrahi @dmizrahi_]

Excited to share our new work: “Language Models Improve When Pretraining Data Matches Target Tasks”

Yes, it sounds obvious (and it is!), but typically this only happens implicitly and indirectly: intuitively select data → benchmark → refine → repeat.

We wondered: what happens if we explicitly match pretraining data to benchmarks? The result is a dead simple approach that yields 2x+ compute multipliers over strong baselines and gives us a principled way to study how benchmark choices shape (and constrain!) model capabilities.

Bonus: extensive scaling laws from training 500+ models that reveal how optimal data selection evolves as models scale.
🧵 (1/14)

[Roman Bachmann @roman__bachmann]

We will present FlexTok at #ICML2025 on Tuesday! Drop by to chat with @JRAllardice and me if you're interested in tokenization, flexible ways to encode images, and generative modeling.

📆 Tue, Jul 15, 16:30 PDT
📍 East Exhibition Hall, Poster E-3010
🌐 https://t.co/17oJKymhPl