Sayak Paul

ML at Hugging Face 🤗

[Sayak Paul @RisingSayak]

Had the honor to present diffusion transformers at CS25, Stanford. The place is truly magical.

Slides: https://t.co/Npm3NCmbtm
Recording: https://t.co/8Q9Mcrjxlq

Thanks to @stevenyfeng for making it happen!

[Sayak Paul @RisingSayak]

Want to know if a kernel is compatible with your system configuration?

We shipped `get_kernel_variants()` to make it easier!

Query every kernel available on the Hub and know if it'd run on your machine 🫡

[Sayak Paul @RisingSayak]

Having tons of fun with the DGX Spark running img and vid generation models locally (finally!)

Thanks to @NVIDIAAI for the gift 🫡 It's quite the beast!

[Sayak Paul @RisingSayak]

@neovectormind But great value for return.

[Sayak Paul @RisingSayak]

I am sold on the fact that there's no better build system than Nix when it comes to absolute reproducibility, system definition, and interoperability.

There's no match.

[Sayak Paul @RisingSayak]

@liuliu I would argue that Nix's enforcement on sandboxing is a special plus.

[Sayak Paul @RisingSayak]

@liangsu325045 @ned714 @NVIDIAAI Just sent the invites. Let's ship great kernels.

[Sayak Paul @RisingSayak]

Having tons of fun with the DGX Spark running img and vid generation models locally (finally!)

Thanks to @NVIDIAAI for the gift 🫡 It's quite the beast!

[Sayak Paul @RisingSayak]

@AtlasInference @NVIDIAAI Perhaps for LTX2.3 or Wan2.2? Those two are quite used.

* https://t.co/F5RelqtJ6E
* https://t.co/yed70hEyyr

[Sayak Paul @RisingSayak]

@AtlasInference @NVIDIAAI My interests are in {img,vid}-gen x kernels. So, those kernels won't help much yet.

[Sayak Paul @RisingSayak]

@liangsu325045 @ned714 @NVIDIAAI I am happy to create a shared Slack channel with you on the HF workspace and bring in other members of our team there. Is the email on your GitHub the right one for this?

[Sayak Paul @RisingSayak]

@Lndsp12 @NVIDIAAI Yeah it's a monkey

[Sayak Paul @RisingSayak]

Learned it the hard way at HF and it was an extremely valuable lesson.

[Jeremy Howard @jeremyphoward]

@badlogicgames When that's happened to me, I've spent a couple of hours trying to build enough familiarity to instead file a slop-ish issue, including a "PoC fix".

My possibly-old-fart view is that a PR should generally be code you're willing to stand behind, or at least spend time polishing.

[Sayak Paul @RisingSayak]

@zhyncs42 @lightseekorg @LeiLMx Agreed. This is why we are developing the kernels project at @huggingface.

This is our recent GPU Mode:
https://t.co/MRH4AQI4IP

Would there be any interest in a collaboration?

[Sayak Paul @RisingSayak]

@BlastBurn27 @NVIDIAAI Priceless

[Sayak Paul @RisingSayak]

@0xnicname @ned714 @NVIDIAAI Kya pata bhau

[Sayak Paul @RisingSayak]

@liangsu325045 @ned714 @NVIDIAAI Yeah same experience for NVFP4 here:

https://t.co/QmtN8enO8g

You have to be selective.

Very cool work with the kernels, would you be interested in building and packaging them with

https://t.co/5jhJ1OQ6M5 ?

[Sayak Paul @RisingSayak]

@liangsu325045 @ned714 @NVIDIAAI And when you are turning the entire pipeline into kernel, there are still some component boundaries, no, for the separate components (text encoder, denoiser, VAE, sampler). Maybe I misunderstood.

[Sayak Paul @RisingSayak]

@liangsu325045 @ned714 @NVIDIAAI Sweet. Are you referring to memory bandwidth? In that case I don’t think Spark is bandwidth constrained either.

[Sayak Paul @RisingSayak]

@TonyDaDesigner @ned714 @NVIDIAAI Comparing this to closed source stuff isn’t fair.

[Sayak Paul @RisingSayak]

@liangsu325045 @ned714 @NVIDIAAI Haven’t looked into fully optimizing yet. General impression is that the workloads I have tried so far are not utilizing the SMs much so the latency is often that shiny.

I will first try out the usual. FA4, NVFP4, etc.

Do share your recipes!

[Sayak Paul @RisingSayak]

@ned714 @NVIDIAAI You are terribly mistaken.