lm

research at nvidia

[lm @loicmagne_]

@wuyang_zhou I think mostly yes, it's able to parse UI/game data really well

[lm @loicmagne_]

so it seems like you can let Astra play games by looking at the screen and it'll do pretty well

here it completed Celeste chapter 1 with only a few retries

(this is non-realtime, game is paused when the model thinks)

[Wuyang Zhou @wuyang_zhou]

I asked GPT-6 Astra to mine a diamond in Minecraft using computer use, then went to sleep.

Woke up to a diamond in its inventory 🤯🤯🤯

Setup in the replies.

[lm @loicmagne_]

@wuyang_zhou timings in general, I tried it on Elden Ring, it understands that the boss is about to attack but it struggles to time the roll correctly

[lm @loicmagne_]

@jfFxjfgLrmKRr pure computer use, the prompt specifies what each button does (though Astra probably already knows it for a popular game like Celeste)

[lm @loicmagne_]

@imjustnewatai medium

[lm @loicmagne_]

@WeihaoTan64 yeah close to 1hr in realtime just for this clip haha

[lm @loicmagne_]

@WeihaoTan64 no limit, the model decides which buttons to press and for how long, so in theory it could do frame by frame

in practice it plays at ~5Hz but it varies

[Ilya Sutskever @ilyasut]

if you value intelligence above all other human qualities, you’re gonna have a bad time

[lm @loicmagne_]

@ayaannmalik @amilabs @ylecun Congrats Ayaan!

[Jan Kautz @jankautz]

Excited that NitroGen won a Best Paper Honorable Mention at #CVPR. @NVIDIAAI

[Jim Fan @DrJimFan]

NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations.

It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!

[#CVPR2026 @CVPR]

Honorable Mentions👏

[NVIDIA @nvidia]

This week at #CVPR2026, NVIDIA Research is presenting three papers across physical ai that offer groundbreaking solutions for training at scale across diverse applications:

→ GraspGen-X: the first foundation model for zero-shot grasping, trained on billions of simulated grasps

→ LCDrive: a model that replaces expensive text-based reasoning with compact latent representations

→ NitroGen: a generalized gameplay AI foundation model that harnesses NVIDIA Isaac GR00T to help train embodied agents

Learn more: https://t.co/H748YkAWS9

[lm @loicmagne_]

@alys_key the DeepMob

[lm @loicmagne_]

@shengranhu @Recursive_SI Congrats Shengran!

[Vlad Golyanik @VGolyanik]

🚀 Neural expressivity meets 3DGS efficiency at #ICLR2026! Our approach matches the rendering quality and speed of 3DGS with
✅ 10x fewer primitives
✅ 6x fewer parameters

Jointly led by X. Zhou and @huy_nguyen_bao.
🔗https://t.co/0cnEGeeNu2
📝https://t.co/XDVt2YgEZb

#3DGS #VR

[lm @loicmagne_]

@_rockt @NetHack_LE Same for VideoGameBench https://t.co/eOymUqZxHe

[Ruijie Zheng @ruijie_zheng12]

Really like the idea of “effect-driven retargeting.” Most recent works (incl. EgoScale) focus on imitating human motion. But moving forward to 0shot regime, the real challenge is: given human intent, how do we translate it into robot actions that achieve the exact same outcome? Bridging embodiment gaps + robot control stack constraints makes this much harder than a pure IDM problem and more interesting.

[lm @loicmagne_]

@egrefen @gabriberton @woj_zaremba I thought it had been stopped at one point https://t.co/33UUIogsxW

[lm @loicmagne_]

@MrCatid Isn't it supposed to be the time humans take to complete the task?