check out RAEv2 led by Jas. through extensive exps, we found some really intriguing behaviors showing why strong representation encoders are key for pixel decoders.
spoiler: it’s not about hillclimbing fid; new metrics like ep@fid-k/fdr^k show there’s a lot more left to explore!
To a large extent - yes, we wanted the model to use as much context from the original video for consistency of subjects/movement. Logically, it's easy to train one which can fix issues in the original video too, but for the purposes of this work: subject consistency with source > aesthetics , in terms of importance.
@hckinz Hi! Vista4D was concurrent work to ours, and takes some different arch decisions, which are quite interesting in itself. Our method builds on Wan2.2, and more so tackles this pressing issue : can you scale a vid2vid by using raw footage, using very scarce synthetic data
vision🍌 is here https://t.co/Ued6GGk4Et
if you got into computer vision the way I did, starting with pixel-level labeling tasks like segmentation, edges, depth, or surface normals, you’ll probably feel the same seeing these results -- something big has quietly shifted, and it’s going to change how we approach these problems for good 🧵
i’m joining forces with @ylecun and an incredible group of people to start AMI Labs @amilabs.
AMI isn’t a conventional lab. we don’t intend to become one.
a lot to say about why this moment matters, but for now we’re heads down building.
join us: https://t.co/zXj1IyBYDc
🚀 Unlocking Standard Diffusion Transformers on Representation Encoders
Why do standard DiTs fail to converge on high-dimensional features like DINOv2? 📉 We found the answer isn't just "more parameters"—it's Geometry.
Introducing Riemannian Flow Matching with Jacobi Regularization (RJF)
📄 Paper: https://t.co/yMutIh2mh5
Hiring a Content & Narrative Operator to work with me at @morphic.
- Someone young, obsessed with AI and startups
- Uses ChatGPT / Claude for writing tasks DAILY
You could be working remotely, but it’s very crucial that you live on Twitter. 😺
Just DM me. Or send this tweet to someone you know.
What if Odyssey was reimagined as an anime?
Inspired by the timeless epic, this is our anime interpretation of Odysseus’ journey.
Same myth. New visual language. Made entirely with @morphic.