Yichen Sheng

Research scientist in NVIDIA. Working in creating a world you cannot distinguish from reality. DLSS5 is my first step. Opinions are my own.

[Yichen Sheng @Coding_Black]

Have to correct your misunderstanding of my sentence.

My original sentence: PT *faithfully* simulate the scene. The bottleneck is that the assets are a real-world approximation. This approximation leads to fake feeling.

There does exist many games with great assets, PT makes them look incredible.

[Yunze Man @yunzeman]

Introducing Flora, an open model suite for scaling robotics data engines.

It watches robot and human video and annotate what happened (timestamped instructions) and how well it went (rewards and failure reasons).

🔗 https://t.co/yZUS5pHvv9

We use it to train NVIDIA GR00T. 🧵

[Yunhao Ge @GeYunhao]

Today, Dream Labs is officially out of stealth! 🚀

What an incredible journey these past six months have been.

I’ve always believed in two things: scaling and generalization. That belief led us to introduce World Action Models (WAMs), which learn from videos of the world and use that understanding to act. With enough scale, the right architecture, and the right data, I believe we can build a general-purpose robot brain that works across different robots, tasks, and environments.

Yet despite incredible progress in robotics, most breakthroughs have remained in research labs, out of reach for the broader public.

Six months ago, @jang_yoel , @scott_e_reed , @youliangtan , and I co-founded Dream Labs with a simple dream: What if anyone could give their robot a general-purpose brain, just through an API call?

Today, we’re taking that first step. Dream-1 is now live for everyone

Dream-1 controls bimanual robots with parallel grippers it has never seen before, zero-shot, on unseen tasks in unseen environments. But what excites me most isn’t just what Dream-1 can do today. It’s what we might discover together.

With our public robot brain API, we want to put general-purpose robot intelligence into everyone’s hands. Try it on new robots, tasks, and environments we’ve never imagined. We want to empower everyone to discover their own “wow” moments with robots and explore what’s truly possible.

ChatGPT wasn’t perfect when it first launched, but it opened up AI to the world. I hope today marks a similar beginning for robotics.

Dream-1 is still a research beta. It will make mistakes, and there’s a long way to go. But someone has to take the first step.

None of this would have been possible without our amazing team. I’m incredibly proud of what we’ve accomplished in just six months and grateful to be building alongside such talented people. ❤️

And we’re just getting started. If you share our dream, come build the future of robotics with us! 🚀

Everyday capabilities for everyday robots @dreamlabs_ai

[Yichen Sheng @Coding_Black]

Cheaper experiments → more hypotheses tested. That's the part that actually compounds.

[Physion Labs Official @Physion_Labs]

The AI industry has a massive compute efficiency problem. At Physion Labs, we think it's time to challenge it.

Today, we're sharing the technology behind NexTron, our world-intelligence training engine.

Our benchmarks on matched hardware:
⚡ Up to 20.9× SFT throughput vs. DeepSpeed (H100)
⚡ Up to 6.6× SFT throughput vs. Megatron (H200)
⚡ Up to 7.2× SFT throughput vs. Megatron (B200)
⚡ Up to 3.1× end-to-end RL throughput vs. Miles–Megatron
🔥 180B model fine-tuning on just 8 H100 GPUs.

Same hardware. Dramatically higher throughput. Comparable model quality.

We believe the next frontier isn't just building bigger GPU clusters.

It's unlocking more intelligence from every GPU, every watt, and every dollar spent on training.

#PhysionLabs #NexTron #WorldIntelligence #AIInfrastructure

[Arash Vahdat @ArashVahdat]

Please help spread the word!

We’re hiring Research Scientists and Research Interns at NVIDIA Research to work on some of the most exciting problems in generative AI:

🌎 World Models & Video Diffusion
🧠 Diffusion Language Models
🧬 Biomolecule Design

If your research aligns with any of these areas, please apply directly through the links below.

Open positions + application links:
https://t.co/MUAt4mcewx

[Bryan Catanzaro @ctnzr]

“5 years of graphics progress in one toggle”

Now that people have had a chance to play with DLSS5, we can see that neural rendering has redefined real-time graphics.

Ten years of dedicated research and development paying off.

https://t.co/jPjXJ5H8Kd

[Lu Ling @LuLing26466911]

Seeing two female scientists Dr.@LisaSu and Dr.@drfeifei shaping the future of AI together moved me deeply. Hoping more female see a place for themselves in that future and find the courage to follow their curiosity, pursue their own North Star, and take on the hardest scientific questions.

[Bryan Catanzaro @ctnzr]

Can't wait to see some of you in Berlin!

[Johan Edstedt @Parskatt]

Hello everyone!

Me and @davnords will be presenting RoMa v2 and LoMa back-to-back at 9AM in the Geometry, Localization, Matching & Motion spotlight session at #ECCV2026

[Lu Ling @LuLing26466911]

Impressed by GPT-Astra’s CoT ! This Kung Fu demo is built from scrach in Blender and takes the reference video + simple prompt as input. The CoT reasoning really show a grasp of kinematics, joint coordination, and body motion !

Reference video comes from PhysAvater @qingqing_zhao_ @yang_zheng18

[Lu Ling @LuLing26466911]

GPT-6 Astra feels deeply personal. In summer 2024, during my internship with #NVIDIACosmos team, we started Scenethesis(https://t.co/D6MTlODqMs) with GPT-4o: an agent that could imagine a 3D scene, ground it through promt/vision, align it with physics, judge the result before replaning. Those were good old days.

Two years later, the dream has grown far beyond a single scene. Astra brings much stronger multimodal reasoning and long-horizon tool use; Blender and Unreal are becoming canvases for intelligence. Watching the base model advance this quickly is astonishing—and incredibly exciting.

The mission ahead excites me most: agents that create, simulate, and explore worlds; accumulate knowledge from experience; and keep evolving—for games, robotics, and embodied AI. Huge congratulations to the brilliant @OpenAI team. What a beautiful moment for the agentic world!

[Wei Xiong @WeiX1762273]

🚀🚀 We officially release DLSS 5, a real-time, 3D-guided neural rendering stage that transforms rendered frames into vivid, realistic scenes while preserving developer-authored content and design intent. To achieve both enhanced realism and real-time rendering at up to 4K resolution, we develop a one-step pixel-space diffusion model.

Compared with offline foundation models such as GPT Image 2 and Gemini 3 Pro, DLSS 5 delivers strong realism while much more faithfully preserving the input content, identity, and scene structure.

Looking ahead, we’re continuing to push on realism, controllability, temporal stability, and efficiency. Stay tuned!

Check out our technical report for more on how we think about generative modeling for gaming:
https://t.co/2skNDIW3bD

I feel excited to finally see DLSS 5 released! As a researcher working on visual generative modeling, it’s especially meaningful to see our fundamental research make its way into a real product. This has actually been a very special journey for me—from exploring research ideas, to building our first working prototype, and then working closely with the team to turn the research, step by step, into DLSS 5. Proud of what our team has accomplished!

We’d love to hear your thoughts on how we can continue improving the technology and where generative rendering can take the future of gaming. Discussions are very welcome!

[Edward Liu @edliu1105]

There we go 🚀 https://t.co/TA6kNoBkJn

[Yichen Sheng @Coding_Black]

Have to correct your misunderstanding of my sentence.

My original sentence: PT *faithfully* simulate the scene. The bottleneck is that the assets are a real-world approximation. This approximation leads to fake feeling.

There does exist many games with great assets, PT makes them look incredible.

[Yichen Sheng @Coding_Black]

@Timebringer @edliu1105 DLSS5 is deterministic in inference time. Generative methods can still be deterministic in inference.

[Yichen Sheng @Coding_Black]

I feel incredibly lucky that my first project after finishing my PhD became #DLSS5.

In my first 1:1 with @edliu1105, I created this slide to show what I believed next-generation rendering could become. For me, this was the beginning of the journey.

My North Star for rendering has always been a world indistinguishable from footage captured by a real camera. The realism upper bound for existing #DLSS is path tracing (PT). PT can faithfully simulate the scene we describe—but that scene remains an approximation of the real world. As a result, it still looks fake, plastic, far from visually realistic in most of the current gaming.

The rise of generative models opened a complementary path. Early results gave us enough signal to pursue a belief: learned real-world appearance priors could work with physically based rendering—not replace it—to move beyond this ceiling.

DLSS 5 is our first step: a generative video model that operates in real time at up to 4K while remaining grounded in the authored scene.

Getting here took years. Skepticism pushed us to clarify the idea, strengthen the system, and make the evidence more rigorous. I’m deeply grateful for the leadership of @edliu1105, @drewtao and @ctnzr, and for our team’s extraordinary execution.

I’m proud that we have demonstrated a new research and design space in graphics rendering—one that deserves much more exploration. If you work in graphics, vision, or generative models, I hope you’ll read our technical report and join us in exploring it.

[Edward Liu @edliu1105]

We’re publishing a deeper technical report on why generation is needed for the next leap toward photorealism.

VRAM and compute limit both the scene abstraction and the number of rays we can afford. Previous DLSS reconstructs from that representation—so even perfect reconstruction is still bounded by what the scene contains.

DLSS 5 is the first DLSS to generate beyond that abstraction and break through that ceiling, while staying grounded in the game and controlled by the artist.

We also compare it with offline generative models and renderers at similar scaling cost.

https://t.co/uk03ksIFmq

[Yichen Sheng @Coding_Black]

People are worrying about AI takes over human's jobs.

After vibe coding for months, I confirmed(a lot of times) I am the bottleneck.

So no need to scale agents, we need to scale human labors to maximize productivity actually 😅😅😅

[Yichen Sheng @Coding_Black]

A super cool world model that you can control the action as well.

[Ruili Feng @feng_ruili_frl]

We made an Elden Ring boss fight playable inside a real-time video world model.

Text is not a scene prompt. It is the controller.

Play Incantation:
https://t.co/07lRhtLRQQ

[Yichen Sheng @Coding_Black]

Remember to join this keynote to know more about DLSS5.

[Edward Liu @edliu1105]

https://t.co/edZlh1qy7C

[Edward Liu @edliu1105]

Look forward to it!

[ACM SIGGRAPH @siggraph]

A new era of computer graphics is emerging through AI, with new speaker Edward Liu joining NVIDIA Research and Engineering leaders Jan Kautz and Ming-Yu Liu.

Join the 'SIGGRAPH 2026 Sponsored Keynote, Next Era of Graphics — Neural Rendering, World Models, and Simulation'

This session will explore the latest advances in neural rendering, world foundation models, and AI-driven simulation through breakthrough research and live demonstrations. https://t.co/J8BynNbUli