Omead Musa

trying to work on cool things

[Omead Musa @OmeadMoses]

@romir_jain Why does this happen?

Does this happen with World Models that are latent space only? So no visualisation

[Omead Musa @OmeadMoses]

How would one go about experimenting with this?

[Lucas Maes @lucasmaes_]

JEPA are finally easy to train end-to-end without any tricks!

Excited to introduce LeWorldModel: a stable, end-to-end JEPA that learns world models directly from pixels, no heuristics.

15M params, 1 GPU, and full planning <1 second.

📑: https://t.co/cpTzgvbTS0

[Omead Musa @OmeadMoses]

@drawais_ai What's the mechanism that makes repeating runs on the same scenario more efficient?

[Omead Musa @OmeadMoses]

@connerruhl Whys that?

[Omead Musa @OmeadMoses]

@__yogesh007 @eccvconf @PooyanFazli @gameschoolasu @SCAI_ASU @ASUEngineering When you say verify, is that a one off process upfront or is it constantly reviewing and verifying?

[Omead Musa @OmeadMoses]

@Junwon__Seo So to sum it up, train on the IMPORTANT scenarios instead of the average of all scenarios? Which all else being equal is probably not that important?

[Omead Musa @OmeadMoses]

@Junwon__Seo Is this basically adding another level of evaluation to the scenario?

[Omead Musa @OmeadMoses]

@JieWang_ZJUI @sundayrobotics @GeneralistAI @DynaRobotics @physical_int Holy acronym overload

[Omead Musa @OmeadMoses]

@Deeplearner2 @chris_j_paxton Solid proposition but what about all the people who like folding laundry 😂

I'm personally too new to robotics to justify this expense but definitely down the line I'd love to experiment.

[Omead Musa @OmeadMoses]

A conversation with Claude: discovering world models from first principles 😄

0:00:00 video is redundant
0:04:13 why language is special and why pixels break
0:13:13 world models (V-JEPA)
0:17:18 LLMs as a dead end?
0:24:36 human-scale vs particle-level
0:31:02 benchmarks
0:34:20 what it unlocks
0:37:27 robotics & multi-agent finale 38:27

[Omead Musa @OmeadMoses]

@DanielMika_ @dariusfdi What do you think are the criteria/conditions needed for the right model to scale? What is bottleneck that needs to be solved for?

E.g. for LLMs it was moving away from sequentially looking at words

[Omead Musa @OmeadMoses]

@dariusfdi This is the LeCun camp no?

[Omead Musa @OmeadMoses]

RL is reinforcement learning?

So is the stage:

small data size, high efficiency via effective reinforcement

large data size, throw efficiency out the window to just brute force skill learning

If we go back to human sample efficiency, we have a mix of ingredients that robots don't have:
- other fully developed minds and skillsets to observe and learn from (parents)
- embedded genetics for certain skillsets
- im sure there are other things but I can't think of them rn

So should we be trying to replicate this set up for robots?

[Omead Musa @OmeadMoses]

@0xDeFiTH @teortaxesTex Read about that yeah. I see where he is coming from on that point though, even if it is contra majority. Not sure that counts as cooky?

[Omead Musa @OmeadMoses]

@_wenlixiao @junyi42 Hmmmm wasn't this one of the central premise of the 2018 paper, "World Models" by David Ha and Jürgen Schmidhuber?

How does it differ if not?

[Omead Musa @OmeadMoses]

@sunfanyun Oh very cool it's Claude Fable plus a World Model.

What's so different about Diffusion World Models?

[Omead Musa @OmeadMoses]

@richard41814 What are the implications of this though?

[Omead Musa @OmeadMoses]

@FkSg16KN @richard41814 So just adding another variable telling the model to consider its own relative position and view?

[Omead Musa @OmeadMoses]

@francoisfleuret An LLM isn't failing in understanding the human world.

It just never was built to actually live in it.

Everything it ingests is text. So how can you expect it to understand anything else?