Our project, "Interactive Spatial Reasoning and 3D Scene Generation with RL-Enhanced VLM" has been selected for the NVIDIA Academic Grant Program!
https://t.co/pvXGJsSm31
Our research project, led by Professor Han Liu at Northwestern University, has received generous support from NVIDIA.
We are deeply grateful for the donation of:
๐น 32,000 A100 GPU-Hours on Saturn Cloud
๐น 1ร RTX PRO 6000 Blackwell Max-Q Workstation Edition
๐น 1ร Jetson AGX Orin Developer Kit
These state-of-the-art resources will significantly accelerate our efforts to combine reinforcement learning and vision-language models, enabling more coherent and spatially grounded 3D scene generation.
Our first milestone is MetaSpatial โ โReinforcing 3D Spatial Reasoning in VLMs for the Metaverseโ:
๐ https://t.co/qq1T2VmliS
Huge thanks to the NVIDIA Academic Grant Program for supporting academic innovation. Weโre excited to push the boundaries of interactive AI and look forward to sharing more impactful results with the community!
#NVIDIAGrant #RL #VLM #3DSceneGeneration #SpatialReasoning #AcademicResearch #AI #Northwestern @NVIDIAAIDev@NorthwesternU@northwesterncs
๐ Excited to introduce MetaSpatial! ๐
MetaSpatial is the first RL-based framework designed to enhance 3D spatial reasoning in Vision-Language Models (VLMs). By leveraging reinforcement learning with physics-aware constraints, our model enables structured, realistic, and adaptive scene generation, unlocking new possibilities for the metaverse, AR/VR, and game development.
How It Works
๐น Trains VLMs to reason about 3D space without relying on hard-coded rules
๐น Optimizes spatial layouts through reinforcement learning
๐น Physics-aware reward function:
โ Positive rewards for layout coherence, physical consistency, and aesthetics
โ Negative penalties for object collisions, overlaps, and unrealistic configurations
We're excited to keep improving MetaSpatial by exploring different RL algorithms, faster rendering alternatives, and multi-turn reasoning, where VLMs iteratively refine their outputs based on rendered feedback.
๐ Check it out here: https://t.co/qq1T2VmliS
Learn more by exploring the complete introduction to Codev-Bench on Arxiv! https://t.co/ySQIqSEdNz
I'm currently seeking a research internship for Summer 2025! If thereโs a good fit, feel free to reach out.
Excited to announce Codev-Bench! ๐ Designed for developers, advances code completion tools by Codev-Agentโsimulating real-world coding. It reduces manual annotation costs, supports complex completions, and sets a standard built on industrial product feedback!
@northwesterncs