Zhenyu Pan

2nd year cs PhD student at Northwestern University.

[Zhenyu Pan @SeerePan]

@hibb_bb 😁

[Zhenyu Pan @SeerePan]

Our project, "Interactive Spatial Reasoning and 3D Scene Generation with RL-Enhanced VLM" has been selected for the NVIDIA Academic Grant Program!
https://t.co/pvXGJsSm31

Our research project, led by Professor Han Liu at Northwestern University, has received generous support from NVIDIA.
We are deeply grateful for the donation of:
🔹 32,000 A100 GPU-Hours on Saturn Cloud
🔹 1× RTX PRO 6000 Blackwell Max-Q Workstation Edition
🔹 1× Jetson AGX Orin Developer Kit
These state-of-the-art resources will significantly accelerate our efforts to combine reinforcement learning and vision-language models, enabling more coherent and spatially grounded 3D scene generation.
Our first milestone is MetaSpatial — “Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse”:
🔗 https://t.co/qq1T2VmliS
Huge thanks to the NVIDIA Academic Grant Program for supporting academic innovation. We’re excited to push the boundaries of interactive AI and look forward to sharing more impactful results with the community!
#NVIDIAGrant #RL #VLM #3DSceneGeneration #SpatialReasoning #AcademicResearch #AI #Northwestern @NVIDIAAIDev @NorthwesternU @northwesterncs

[Zhenyu Pan @SeerePan]

@andwhynut69633 @Scobleizer Thanks for your interest!

[Zhenyu Pan @SeerePan]

🚀 Excited to introduce MetaSpatial! 🚀
MetaSpatial is the first RL-based framework designed to enhance 3D spatial reasoning in Vision-Language Models (VLMs). By leveraging reinforcement learning with physics-aware constraints, our model enables structured, realistic, and adaptive scene generation, unlocking new possibilities for the metaverse, AR/VR, and game development.

How It Works
🔹 Trains VLMs to reason about 3D space without relying on hard-coded rules
🔹 Optimizes spatial layouts through reinforcement learning
🔹 Physics-aware reward function:
✅ Positive rewards for layout coherence, physical consistency, and aesthetics
❌ Negative penalties for object collisions, overlaps, and unrealistic configurations

We're excited to keep improving MetaSpatial by exploring different RL algorithms, faster rendering alternatives, and multi-turn reasoning, where VLMs iteratively refine their outputs based on rendered feedback.
🔗 Check it out here: https://t.co/qq1T2VmliS

[Zhenyu Pan @SeerePan]

@RobbieLemos @Scobleizer Thanks for your interest!

[Zhenyu Pan @SeerePan]

Learn more by exploring the complete introduction to Codev-Bench on Arxiv! https://t.co/ySQIqSEdNz
I'm currently seeking a research internship for Summer 2025! If there’s a good fit, feel free to reach out.

[Zhenyu Pan @SeerePan]

Excited to announce Codev-Bench! 🚀 Designed for developers, advances code completion tools by Codev-Agent—simulating real-world coding. It reduces manual annotation costs, supports complex completions, and sets a standard built on industrial product feedback!
@northwesterncs