We can reach 20 FPS for real-time video editing on a single 4090.
1.3B self-forcing model, 4-step denoising, fp16.
Here's a screen recording of it running live on 4090. Repo in the comments.
Tweet 1
We achieve real-time video editing at 19.3 FPS on a single RTX 4090 (using just 16GBVRAM) π
Introducing LIPAR: We exploit temporal redundancy in latent space to boost Self-forcing throughput by 1.53Γ with ZERO additional training with no drop in performance!
@_akhaliq