PhD student @PrincetonVL. Previously at @Berkeley_AI, '21 Math/CS @Caltech. Interested in all things 3D computer vision and graphics.
When in doubt, --force
1/ We've released Infinigen 2.0! Currently in preview. It creates indoor 3D scene files in 1min CPU time, and includes new and better materials --- all still fully procedural. Our new 2.0 design is highly efficient and allows easy control and recombination via Python APIs.
InFlux++: Real and Synthetic Data for Estimating Dynamic Camera Intrinsics
@ErichLiang, Caleb Kha-Uong, Chinmaya Saran, Sreemanti Dey, David W. Liu, Junhan Ouyang, Benjamin Zhou, @jiadeng
tl;dr: synthetic video dataset + real-world benchmar
https://t.co/e0EaRWDWe9
Meet WAFT (Warping-Alone Field Transforms), our new optical-flow estimator. #1 on public benchmarks (Sintel & Spring), 1.3-4.1x faster than leading methods, and 2x lower memory. Key idea: replace cost volumes with high-res feature-space warping. Code and paper:👇
New #NVIDIA Paper
We introduce Motive, a motion-centric, gradient-based data attribution method that traces which training videos help or hurt video generation.
By isolating temporal dynamics from static appearance, Motive identifies which training videos shape motion in video generation.
🔗 https://t.co/TbKXjQMN3H
1/10
Estimating camera intrinsics from video is key to 3D reconstruction, but most methods assume they’re fixed per video. What if the camera keeps zooming and refocusing?
Meet InFlux, the first benchmark with per-frame ground truth for videos with dynamic intrinsics. 🧵1/5
Estimating camera intrinsics from video is key to 3D reconstruction, but most methods assume they’re fixed per video. What if the camera keeps zooming and refocusing?
Meet InFlux, the first benchmark with per-frame ground truth for videos with dynamic intrinsics. 🧵1/5
Princeton365: A Diverse Dataset with Accurate Camera Pose
Karhan Kayan, Stamatis Alexandropoulos, Rishabh Jain, Yiming Zuo, @ErichLiang, @jiadeng
tl;dr: dataset with gt camera pose and 6 DoF camera motion; new optical-flow based metric
https://t.co/zv1ifi4f2L
Data drives progress in computer vision. We introduce Infinigen: a generator of unlimited high-quality 3D data. 100% procedural, no external assets, no AI. Free and open source.
Intro video: https://t.co/ER9zz1flxm
Code: https://t.co/WpeGIoPeo1
CVPR23: Wed 4:30PM, Poster 27
Infinigen v1.0.1 is out, with license switched to BSD, line credits, expanded docs for ground truth, and miscellaneous fixes. Please do a fresh git clone if possible as we have reset the commit history to show line credits (sorry for any inconvenience from this one-off reset).
A roadmap page for Infinigen is now up (short term for now)
https://t.co/oTdrTrGER4
Find out what is coming up and send us requests to help us prioritize!
Infinigen v1.0.3 is released! https://t.co/WpeGIoOGyt
You can now generate water and fire: https://t.co/uvfbkzpgmO
Also check out how to implement your own assets with python: https://t.co/BtHwKj1BPJ
Infinigen v1.1.1 is out: updated to Blender3.6 and revamped installation to be more robust. See https://t.co/cYZioJEgoQ for more, including our "minimal" install variant which lets you generate non-terrain assets in almost any python environment (no local compilation required).
How good is the compositional generation capability of current Text-to-Image models? https://t.co/cWki6OUyjJ
Introducing ConceptMix, our new benchmark that evaluates how well models can generate images that accurately combine multiple visual concepts, pushing beyond simple, single-object generation.
This work is done in collaboration with @dingli_yu, @YangsiboHuang, @orussakovsky & @prfsanjeevarora