@hisham_artz You can use the code in the official trt repo to build a trt engine, and run it from there. Mine is different- took the Engine class from repo & modified it to behave like a diffusers unet so i can run it in diffusers pipeline. It's under /demo/Diffusion/ https://t.co/MsyThRUp2l
@oleg__chomp I think you could probably just compile it normally- following the examples from the tensorrt repo. Most of the speedups from tensorrt are a result of it's own internal autotuning. The api for using that autotuning remains the same for most models.
@vibeke_udart Also make sure that the scheduler is using "timestep_spacing": "trailing". And depending on the actual trt impl you are using. If you're just using the code in the trt repo as I did, you'll want to make a diffusers compatible unet wrapper which can be used in normal diffusers.
@vibeke_udart Well sdxl-turbo should use a maximum of guidance scale ~ 1.5, looks like it is using more. I've also found that sometimes the diffusers LCMScheduler works best for details, though can make the image a bit smudgey. Either LCMScheduler or EulerAncestralDiscreteScheduler.
Finally figured out how to speed up my #sdxlturbo
frontend! It's so fast that the only way to show the actual speed is to delete the prompt, since I can't type fast enough ๐ .. built with next.js frontend & tensorrt backend.
@koltregaskes Essentially, yes. Though the lack of inter-frame coherency even with the same seed would make it seem very jittery, similar to the video.
@vibeke_udart There's a demo on their github https://t.co/yHqhQZRwgp. It uses diffusers, so can be pulled automatically though huggingface, though getting tensorrt to the state it's in-in my demo requires a lot of tweaks. Also I recommend using docker, tensorrt can be kina finnicky.
@JonathanSolder3 Not at the moment, it's just a little app running on my workstation at home. Not sure when or if it'll ever be a part of a public thing.
Made a neat GUI in react for fun & hooked it up to an optimized sdxl-turbo tensorrt api backend I built for image autocomplete! It generates so fast that my browser cant keep up ๐ฅฒ - so fun! #sdxlturbo#sdxl#AIart#stablediffusion
For anyone who may think that this is a fast-forwarded video, or using image caching, I can guarantee it is not- the images are all base64 and it's using a websocket, the terminal below is the output from the tensorrt rest-api server in real time as I ctrl+y / ctrl+z