Simon Rouard

PhD @kyutai_labs and @Ircam. Deep Learning and music

[Simon Rouard @simonrouard]

Very happy to announce that my paper “Audio Conditioning for Music Generation via Discrete Bottleneck Features“ done with @honualx @adiyossLC @jadecopet and Axel Roebel has been accepted at ISMIR24.
Paper: https://t.co/2KwG6Bk1jH
Sample: https://t.co/Dkom70Eoie
Code: soon

[Simon Rouard @simonrouard]

Today at 10:30 at @iclr_conf I’ll be presenting CALM (Continuous Audio Language Models), the architecture behind Pocket TTS, @kyutai_labs’s 100M params TTS that runs on CPU.

Come chat with me if you want to build an audio LM without tokens

Paper: https://t.co/IR2vaZ0wxH

[Simon Rouard @simonrouard]

@kyutai_labs Thank you @nmboffi, as we noticed that Lagrangian self-distillation worked much better than consistency for our TTS task.

[Simon Rouard @simonrouard]

Super happy that our work on Continuous Audio Language Models (https://t.co/8KmdWlymUB) led us to build an outstanding 100M TTS with voice cloning ability that runs on any laptop CPU.

[kyutai @kyutai_labs]

We’re excited to introduce Pocket TTS: a 100M-parameter text-to-speech model with high-quality voice cloning that runs on your laptop—no GPU required.
Open-source, lightweight, and incredibly fast. 🧵👇

[Gradium @GradiumAI]

Gradium is out of stealth to solve voice. We raised $70M and after only 3 months we’re releasing our transcription and synthesis products to power the next generation of voice AI.

[kyutai @kyutai_labs]

1/2 We’re releasing an in-depth tutorial on neural audio codecs, the secret sauce that makes it possible for audio LLMs to not sound like a horror movie:

[arXiv Sound @ArxivSound]

Rouard Simon, Orsini Manu, Roebel Axel, Zeghidour Neil, D\'efossez Alexandre, "Continuous Audio Language Models," https://t.co/LatjRsbPgt

[kyutai @kyutai_labs]

Kyutai Speech-To-Text is now open-source! It’s streaming, supports batched inference, and runs blazingly fast: perfect for interactive applications.
Check out the details here: https://t.co/bQMP56XaKC

[kyutai @kyutai_labs]

Talk to https://t.co/1ZcGtCwvgx 🔊, the most modular voice AI around. Empower any text LLM with voice, instantly, by wrapping it with our new speech-to-text and text-to-speech. Any personality, any voice. Interruptible, smart turn-taking. We’ll open-source everything within the next few weeks.

[kyutai @kyutai_labs]

Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧.
Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the source speech.
Based on objective and human evaluations, Hibiki outperforms previous systems for quality, naturalness and speaker similarity and approaches human interpreters. 🧵

[arXiv Sound @ArxivSound]

``MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling,'' Simon Rouard, Robin San Roman, Yossi Adi, Axel Roebel, https://t.co/tGwJyDyRIH

[kyutai @kyutai_labs]

Meet Helium-1 preview, our 2B multi-lingual LLM, targeting edge and mobile devices, released under a CC-BY license. Start building with it today!
https://t.co/X4Dbx2T1cJ

[Simon Rouard @simonrouard]

I am presenting our paper MusicGen-Style “Audio Conditioning for Music Generation via Discrete Bottleneck Features” at @ISMIRConf this afternoon. The code as well as the weights of the model are available on https://t.co/tSvrr446v3. You can now play with it!

[Nicolas DUFOUR @nico_dufour]

It start now at poster 227!

[Simon Rouard @simonrouard]

The code and weights of the model will be released soon. Stay tuned!

[Simon Rouard @simonrouard]

Very happy to announce that my paper “Audio Conditioning for Music Generation via Discrete Bottleneck Features“ done with @honualx @adiyossLC @jadecopet and Axel Roebel has been accepted at ISMIR24.
Paper: https://t.co/2KwG6Bk1jH
Sample: https://t.co/Dkom70Eoie
Code: soon

[Simon Rouard @simonrouard]

Then we can as well use text and style conditioning to generate music, but we noticed that the model tends to ignore the text prompt. We then introduce a double classifier free guidance. This guidance could be applied to other multi-conditioned generative models.

[arXiv Sound @ArxivSound]

``Audio Conditioning for Music Generation via Discrete Bottleneck Features,'' Simon Rouard, Yossi Adi, Jade Copet, Axel Roebel, Alexandre D\'efossez, https://t.co/Z01vzcESpi

[Jean-Marie Lemercier @jm_lemercier]

#ICML2024 paper “An Independence-promoting Loss for Music Generation with Language Models”

We promote independence between EnCodec codebooks using a kernel trick and improve music generation quality 🎶

Paper 📜 https://t.co/Uyb1sIusze
Audio/Code 🔊 https://t.co/MFmdsOIrxo