@rasbt Hi Sebastian, thank you for this post! In your readme, you listed the RoPE θ as increasing from 10,000 to 50,000. I believe the paper says they increased it to 500,000. Is that correct?
@TKyaman Hi there, I'm glad you found this code useful. It was such a small snippet that I didn't bother licensing it :). Feel free to modify as you like.
Kudos to @benfrederickson for py-spy. Just tracked down a nasty problem with a CPU tensor that should have been on GPU in PyTorch using the sampling profiler, which brought my GPU utilization back up to 100%.
@GoodGood014 Nice work on the library! When I went looking I was surprised something like this didn't already exist, so it's good to see a library for calculating FAD in PyTorch.
@neonbjb At least for speech, bandwidth extension has been a popular technique lately for getting higher sample rates out of audio models. I have had some success working with the HiFi-GAN+ model https://t.co/lM7rFvQRUa. BTW, Tortoise is an amazing achievement.
I just finished Deep Generative Modeling by @jmtomczak. Covering topics from VAEs to GANs to EBMs, it has changed the way I think about ML. This book also demonstrates the reasoning behind the pesky sqrt(2) voodoo that shows up in so many neural networks.
I published an open source implementation and pretrained models for the HiFi-GAN+ architecture, from the paper "Bandwidth Extension is All You Need" https://t.co/AlXBCsp35d
Do you (really) know PyTorch 🔥?
Try out my Tensor Puzzles 🧩. 16 mini-puzzles for those ready to take off the stackoverflow training wheels.
https://t.co/B5L2bja1DN