This enables a number of exciting applications such as the composition of different dynamic scene elements, first-person view synthesis and temporally consistent 4D scene editing.
It’s also FAST! These render at 850 FPS, and only take around 2 hours to train on a single GPU.
If you find dynamic NeRFs cool, check out this work where we exploit some of the very interesting properties of the 3D Gaussian Splatting for persistent tracking through time.
We learned a lot working with @JonathonLuiten that helped us understand better the power of 3DGS.
3D Gaussian Splatting for Real-Time Radiance Field Rendering
paper page: https://t.co/q9P2DmfJyP
Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.
I was just playing around with the @SnapAR#AI Machine Learning 'eraser' filter, combining it with regular LensStudio effects
But then I discovered that I accidentally created a "live #Banksy generator" #Snapchat lens!
Give it a try here: https://t.co/TlcbYx5M6f
Introducing the Luma✨Unreal Engine alpha! Fully volumetric Luma NeRFs running realtime on Windows in UE 5 for incredible cinematic shots and experiences, starting today!
Try now: https://t.co/mdyGTWhhgN
This is how the real Metaverse will happen.
The web has evolved to be the most powerful computing platform in the world. It is not just the natural choice for enabling the Metaverse, it is guaranteed that it will become the de facto winner.
Why?
I brought my childhood imaginary friend back to life using A.I. (#GPT3) and it was one of the scariest and most transformative experiences of my life.
A thread 🤖 (1/23)
Passthrough is really nice, but it'd be a lot nicer if it had some colors. It's a good thing that my #iPadPro can help with that. #OculusQuest2#MadeWithUnity