Thom

[Thom @ThomShutt]

@jojobreeeeze @madisontayt_ You forgot the magic word: Metamask

[RyanOnTheInside @RyanOnTheInside]

YO,

I adapted VACE to work with real-time autoregressive video generation.

Here's what it can do right now in real time:
- Depth, pose, optical flow, scribble, edge maps — all the v2v control stuff
- First frame animation / last frame lead-in / keyframe interpolation
- Inpainting with static or dynamic masks
- Stacking stuff together (e.g. depth + LoRA, inpainting + reference images)
- Reference-to-video is in there too but quality isn't great yet compared to batch

Getting ~20 fps for most control modes on a 5090 at 368x640 with the 1.3B models. Image-to-video hits ~28 fps. Works with 14b models as well, but doesnt fit on 5090 with VACE.

This is all part of Daydream Scope, an open source tool for running real-time interactive video generation pipelines. The demos were created in/with Scope, and is a combination of Longlive, VACE, and Custom LoRA.

There's also a very early WIP ComfyUI node pack wrapping Scope.

But how is a real-time, autoregressive model relevant to @ComfyUI ? Ultra long video generation. You can use these models distilled from Wan to do V2V tasks on thousands of frames at once, technically infinite length. I havent experimented much more than validating the concept on a couple thousand frames gen. It works!

Full technical details on real-time VACE + more examples here (link in comments)

Curious what people think. Happy to answer questions.

Video + Custom LoRA links also in comments.

Love,
Ryan

p.s. I will be back with a sick update on ACEStep implementation tomorrow

(links in first comment)

[Daydream @DaydreamLiveAI]

We adapted VACE for real-time autoregressive video generation.

Depth, pose, inpainting, temporal extension, reference-to-video, all composable. 20+ FPS. No retraining needed.

[Daydream @DaydreamLiveAI]

Daydream Cohort kicked off today.

The #2 Interactive AI Video Program is rolling.

Keep an 👁️ out for their creations.

[Thom @ThomShutt]

Reading things in 2026

[Daydream @DaydreamLiveAI]

Feb 9–20.
One curated cohort.
Dedicated mentorship.

🪂 Who's ready?

[Thom @ThomShutt]

This is so interesting - a world model performing better if a fake game HUD is overlayed on top of the source images https://t.co/Z0SNF7anGh

[Thom @ThomShutt]

We just released support for Waypoint-1 in Scope! Create a world model with WASD controls from a single image: https://t.co/TJHGrqdBfQ

[Thom @ThomShutt]

@andrianita02 @keyvan_dr @Holland23064113 Here you go!

[Daydream @DaydreamLiveAI]

In this new post from @MarcoTundo, read how we tried to teach StreamDiffusion V2 to do image-to-video (I2V). Two “promising” ideas imploded before a third finally worked.

https://t.co/Fg3FwP5ziQ

[Thom @ThomShutt]

🚢🚢🚢 https://t.co/TUaHzYtb9O

[Amjad Masad @amasad]

NPM worm “Shai Hulud” is back and you can see a list of people getting pwned in realtime by refreshing this GitHub search page.

When a user is infected the worm collects API keys and secrets and pushes them to a new repo on GitHub.

[Thom @ThomShutt]

I'm going to be here to talk about the work we've been doing on @DaydreamLiveAI tomorrow - come and say hi!

[Encode Club @encodeclub]

Join @StoryProtocol & @Livepeer this Friday at Encode Hub for coworking + a hands-on creative video AI workshop!

꩜ 12–3pm: Coworking, feedback, 1:1s w/ Nick from Story
꩜ 3–5pm: Workshop w/ Jamie (Sound of Fractures)

Learn to use AI to craft intentional content + videos - not generic ones.
You’ll map your story, find themes, and turn ideas into real outputs.

RSVP now ↓

[Thom @ThomShutt]

The @DaydreamLiveAI eng team has been derailed this morning by an anime style transfer and an argument about who's the cutest. Try it yourself at https://t.co/zXqq4XzxSx

#stablediffusion #genai

[Thom @ThomShutt]

Sad to not be making it to @demuxed this year, but everyone should check out @RafalLeszko talking about the work we've been doing with realtime generative AI!

[Yondon Fu @YondonFu]

Excited to support StreamDiffusionV2 in the latest release of Scope today!

As a reminder, Scope is tool for running and customizing real-time, interactive generative AI pipelines and models.

Here is a walkthrough of it in action in Scope (see next post if you just want the demo) running on a 4090 at ~9 fps and 512x512 resolution:

[Chenfeng_X @Chenfeng_X]

🥳We’re releasing StreamDiffusionV2 for the live-stream community—from individual creators with one GPU to enterprise platforms with many. StreamDiffusionV2 is our follow-up to StreamDiffusion: #StreamDiffusion powered real products, but temporal consistency still bugged us. We’re fixing that for the community:

🤔Efficiency ≠ live-readiness: many “real-time” video diffusion models are fast, but live streaming demands per-frame deadlines with low jitter. StreamDiffusionV2 balances temporal stability with strict per-frame deadlines, achieves fast interaction, and is ready for your AI Youtuber Stream Live👇

📒Project page: https://t.co/ItuO5zbysl
💻Code: https://t.co/KD53E4f8s6

[Daydream @DaydreamLiveAI]

🇫🇷Paris, we’re bringing real-time video AI to the stage!

Thom Shutt (@TLivepeer) our director of engineering is presenting at @ParisVideoTech meetup on September 18th.

Come check out live video transformed frame-by-frame, in real time!

[Thom @ThomShutt]

We put Cobol into our SRE job description to filter out AI applications

[Thom @ThomShutt]

Today in "Tripping up AI job applications with a few simple questions". This person is in a constant state of bewilderment.