I'm excited to say my paper, Audio-reactive Latent Interpolations with StyleGAN, has been accepted to the Workshop on Machine Learning for Creativity & Design @ NeurIPS 2020!
Blog: https://t.co/YTxGg4B4rM
Paper: https://t.co/fsPQjqxRfM
Code: https://t.co/Fa2MRFJ0TO
Video:
Jamming with @cerspense on Sentinel. Built a 3D scene of points and lines, fed it through SDXL to turn it into jellyfish, layered a particle ripple system on top, finished with film grain. Everything vibe coded with Claude in real time. We described what we wanted as we listened to the music and just watched it come together.
@kettukaa@dnbt777 Yeah, I looked at a lot of higher-level spectral/musical features (onset envelopes, chromagrams, etc) and tried to map as many of those as possible to the different bits of styleGAN as I could
@cerspense has been developing a tool (Sentinel) that allows live diffusion.
I have been playing with it in conjunction with TouchDesigner (for audio reactivity) and my Teenage Engineering OP-XY (for additional effects trigger).
Being able to make animations on the fly based on a song feels like playing an instrument that makes visuals instead of sounds. It is awesome!
This has been made possible thanks to my Dell Pro Max T2 tower equipped with an NVIDIA RTX PRO GPU (you can check out the spec here: https://t.co/w5xWEcaOun).
#DellProPrecision #DellTech #NVIDIA