Motion control is crucial for generating expressive and compelling video content; however, most existing video generation models rely mainly on text prompts for control, which struggle to capture the nuances of dynamic actions and temporal compositions.
To this end, this work trains a video generation model conditioned on spatio-temporally sparse or dense motion trajectories. In contrast to prior motion conditioning work, this flexible representation can encode any number of trajectories, object-specific or global scene motion, and temporally sparse motion; due to its flexibility we refer to this conditioning as motion prompts
Paper Title: Motion Prompting: Controlling Video Generation with Motion Trajectories
Project: https://t.co/EQYELjIRvc
Link: https://t.co/jMWShbelhG
#AI #motionGraphics #animation #GenerativeAI
#A 56-qubit quantum computer has achieved certified #Randomness, marking a significant advancement in quantum computing. This breakthrough holds promise for cryptography and privacy applications. @utaustin@nature https://t.co/yDZVuRUdJR
From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting
Contributions:
1. We propose a novel sparse-to-dense camera relocalization pipeline that leverages Feature Gaussian as the scene representation. This localization method uses sparse features for the initial pose and dense features for pose refinement, enabling accurate camera relocalization. Instead of performing image retrieval followed by feature matching, this pipeline introduces a new coarse-to-fine localization paradigm.
2. We introduce a novel matching-oriented sampling strategy to address the challenge of selecting landmarks from millions of Gaussians. This strategy significantly reduces the number of Gaussians, selecting only a small subset while ensuring they are multi-view consistent and evenly distributed.
3. Directly matching the dense feature map with the sampled landmarks results in an unacceptable computational load. Therefore, we introduce a scene-specific detector that effectively detects landmarks from the extracted dense feature map. This detector can be trained in a self-supervised manner.
4. Based on these landmarks, the camera pose can be easily estimated by feature matching and the PnP algorithm, then refined by aligning the dense feature map with the feature field. We conducted extensive experiments to validate the pipeline's effectiveness. The results indicate our approach surpasses state-of-the-art methods in localization accuracy and recall.
Introducing GAIA-2 🌎Generative world modeling just stepped up a gear.
GAIA-2 is the latest development of Wayve’s video-generative world model tailored for driving. GAIA-2 offers richer, more realistic, and highly controllable synthetic driving scenarios, accelerating Wayve’s path to safe driver assistance and automated driving at scale.
Learn more about GAIA-2 in our Blog: https://t.co/1Rfi5ppiZQ
#GAIA2 #GAIA #EmbodiedAI