In this micro-experiment, I try to extract playable instrument models from a single audio recording. You can play with a handful of the models in your browser!
https://t.co/DY3FtAIr6Y
I'm also continuing my work to show that the strong inductive bias, or "resonance prior" as I've been calling it, is helpful in producing sparse, compressive and interpretable audio encodings, at least for "natural" signals.
Text-to-music is amazing, but it sucks all the fun out for me, as a musician. I'm using machine learning to "extract" models that can be controlled in real-time. My goal is to give them all the subtlety and richness that you'd get from a real acoustic instrument.
@bohannon_bot This is a great question! I've been working toward a model that does not use "tracks" or any strict hierarchy, but instead allows you to select and mutate subsets of events, sort of like CSS selectors. Very early days, but it feels like a promising direction!
I wrote a paper summarizing the work I've been pursuing toward a sparse audio codec!
I'd like to post this to the CS -> Sound section of arxiv, if possible, so if anyone would be willing to "endorse" me, shoot me a message and I'll share the code!
https://t.co/tagRkziK4S
I think a lot about the "tyranny of the grid" and the connections between transformers and NERF-like, "neural operators" or Gaussian splatting approaches that learn a discretization-independent representation of a signal. I think many of our struggles with perceptually-relevant losses stem from grids, vs. comparing two sets or graphs.