Zippy

Software engineer working on ML image generation techniques & inference. Obsessed with programming and life.

Research Engineer @ Luma AI

[Luma @LumaLabsAI]

Uni-1 is here! A new kind of model that thinks and generates pixels simultaneously. Less artificial. More intelligent.

[Zippy @AlexanderRedde3]

@hisham_artz You can use the code in the official trt repo to build a trt engine, and run it from there. Mine is different- took the Engine class from repo & modified it to behave like a diffusers unet so i can run it in diffusers pipeline. It's under /demo/Diffusion/ https://t.co/MsyThRUp2l

[Zippy @AlexanderRedde3]

Now we're real time! 😎 - with SDXL turbo & tensorrt optimized further.

[Zippy @AlexanderRedde3]

Made an fp8 implementation of Flux which gets ~3.5 it/s 1024x1024 on 4090, >2x faster than other methods. https://t.co/H3DkO0sZYK #flux1

[Zippy @AlexanderRedde3]

@oleg__chomp I think you could probably just compile it normally- following the examples from the tensorrt repo. Most of the speedups from tensorrt are a result of it's own internal autotuning. The api for using that autotuning remains the same for most models.

[Zippy @AlexanderRedde3]

@vibeke_udart Also make sure that the scheduler is using "timestep_spacing": "trailing". And depending on the actual trt impl you are using. If you're just using the code in the trt repo as I did, you'll want to make a diffusers compatible unet wrapper which can be used in normal diffusers.

[Zippy @AlexanderRedde3]

@vibeke_udart Well sdxl-turbo should use a maximum of guidance scale ~ 1.5, looks like it is using more. I've also found that sometimes the diffusers LCMScheduler works best for details, though can make the image a bit smudgey. Either LCMScheduler or EulerAncestralDiscreteScheduler.

[Zippy @AlexanderRedde3]

@rudzinskimaciej Yes I am 😊 I have a serious case of programming addiction haha

[Zippy @AlexanderRedde3]

Finally figured out how to speed up my #sdxlturbo
frontend! It's so fast that the only way to show the actual speed is to delete the prompt, since I can't type fast enough 😆 .. built with next.js frontend & tensorrt backend.

[Zippy @AlexanderRedde3]

@koltregaskes Essentially, yes. Though the lack of inter-frame coherency even with the same seed would make it seem very jittery, similar to the video.

[Zippy @AlexanderRedde3]

@rudzinskimaciej It's an amd 7950x cpu + 2x4090 machine, though this demo is only using 1 4090. Also thanks! 😊

[Zippy @AlexanderRedde3]

@ZealotDKD Right now there isn't a way. It's just a web ui / api running on my workstation. 🥹

[Zippy @AlexanderRedde3]

@vibeke_udart There's a demo on their github https://t.co/yHqhQZRwgp. It uses diffusers, so can be pulled automatically though huggingface, though getting tensorrt to the state it's in-in my demo requires a lot of tweaks. Also I recommend using docker, tensorrt can be kina finnicky.

[Zippy @AlexanderRedde3]

@JonathanSolder3 Not at the moment, it's just a little app running on my workstation at home. Not sure when or if it'll ever be a part of a public thing.

[Zippy @AlexanderRedde3]

Made a neat GUI in react for fun & hooked it up to an optimized sdxl-turbo tensorrt api backend I built for image autocomplete! It generates so fast that my browser cant keep up 🥲 - so fun! #sdxlturbo #sdxl #AIart #stablediffusion

[Zippy @AlexanderRedde3]

@EMostaque Your prediction of real time image generation came true 😄. It took a little while, but we're here!

[Zippy @AlexanderRedde3]

For anyone who may think that this is a fast-forwarded video, or using image caching, I can guarantee it is not- the images are all base64 and it's using a websocket, the terminal below is the output from the tensorrt rest-api server in real time as I ctrl+y / ctrl+z

[Zippy @AlexanderRedde3]

@XWAVEart You are so nice friend <3

[Zippy @AlexanderRedde3]

@ScottieFoxTTV But my workstation is on the other side of my room. It's too far. 🥴

[Zippy @AlexanderRedde3]

Just broke 20 fps!

[Zippy @AlexanderRedde3]

Optimized further meaning I'm getting 17 fps for full clip encode -> unet pass -> vae decode -> serialize to jpeg bytes.