Flux

PhD Student at the University of Stuttgart, Germany

Working with Speech Synthesis mostly

[evan loves worf @esjesjesj]

“It was an accident!! He’s autistic!!”

[François Chollet @fchollet]

Today OpenAI announced o3, its next-gen reasoning model. We've worked with OpenAI to test it on ARC-AGI, and we believe it represents a significant breakthrough in getting AI to adapt to novel tasks.

It scores 75.7% on the semi-private eval in low-compute mode (for $20 per task in compute ) and 87.5% in high-compute mode (thousands of $ per task). It's very expensive, but it's not just brute -- these capabilities are new territory and they demand serious scientific attention.

[Flux @Flux9665]

@reach_vb And such an excellent name! Also with the release by black forest happening perfectly timed while I‘m in the black forest.

[Vaibhav (VB) Srivastav @reach_vb]

Demo: https://t.co/jT84widr21
Dataset: https://t.co/Ngk3otiS6k
Library: https://t.co/wyqNttqNVD

[Vaibhav (VB) Srivastav @reach_vb]

Toucan TTS: MIT licensed Text to Speech in 7000 languages! 🔥
The most multilingual open-source TTS model out there ⚡

Step 1: They built a text frontend that can turn text in any language from the ISO-639-3 list into language-agnostic articulatory features.

Step 2: Then, pre-trained a speech synthesis to convert those features into speech for 462 languages.

Step 3: Finally, they applied meta-learning to fill in the remaining languages for which they had no data.

Bonus: They also open-source their dataset too! 🤗

Kudos to @Flux9665 and everyone involved in the project!

[Flux @Flux9665]

@erogol I'm really sorry to hear that! Not only did you guys build great open models, you also shared so much of the technical details on your substack that other companies may have guarded like a dragon its gold. Very awesome from a research perspective!

[Flux @Flux9665]

@ylecun Can we get access to the MMS-lab data as well, like with MultiLingualLibriSpeech?
It's amazing that the models are free to use, but having access to the data would allow for multilingual experiments on a totally new scale.

[IMS at University of Stuttgart @ims_stuttgart]

We're happy to announce that IMS has 5 papers accepted at #ACL2023 #NLProc :)

https://t.co/KFyFTer0Hh

[Hendrik Schuff @HendrikSchuff]

Check out our new ACL'23 findings paper! (w/ @alon_jacovi, Heike Adel, @ThangVu2014, @yoavgo)

Neighboring Words Affect Human Interpretation of Saliency Explanations
https://t.co/aUUh6V1Xo8

We find that a word's perception is biased by its neighbors in heatmap explanations.
1/7

[AI at Meta @AIatMeta]

🧠 Modeling human-level intelligence is still a far-off goal, but the findings from this work suggest we may be on the right path.

Congratulations on the #NeurIPS2022 acceptance, @c_caucheteux, Juliette Millet, @JeanRemiKing, @agramfort & team!

[Flux @Flux9665]

@onk78368343 @_akhaliq @ak92501 @RiversHaveWings @Miles_Brundage @Ted_Underwood Yes, finding more and better ways to give context-information to the Model is on the agenda :)

[Flux @Flux9665]

@khursani8 @_akhaliq @ak92501 @RiversHaveWings @Miles_Brundage @Ted_Underwood If you only change the text and nothing else, the speaker should stay exactly the same. However the random seed for the speakers change everytime the space is restarted, which huggingface does from time to time.

[Flux @Flux9665]

@_akhaliq @ak92501 @RiversHaveWings @Miles_Brundage @Ted_Underwood It's been a while, but here it is: https://t.co/AKHSF8fEYd

Using the GANSpace approach to generate speaker embeddings with desired properties. Will be updated with better models soon™

[Flux @Flux9665]

@ak92501 @RiversHaveWings @Miles_Brundage @Ted_Underwood That‘s something I‘m working on! It‘s working pretty well so far. Still trying out new ways of controlling speaker characteristics.

[Flux @Flux9665]

@csteinmetz1 @miblue119 adding another layer of postprocessing by normalizing the loudness with a small block size using pyloudnorm helped with that pretty well. But if there was a tool that would just enhance my 48kHz synthesized speech with no downsides, I'd be very glad.

[Flux @Flux9665]

@csteinmetz1 @miblue119 I just discovered this one today as well. I use it to post-process synthesized speech. While it does get rid of some hums and buzzes pretty well, it also diminishes sibillance in speech a lot, which is a problem.

[arXiv Sound @ArxivSound]

``Exact Prosody Cloning in Zero-Shot Multispeaker Text-to-Speech. (arXiv:2206.12229v2 [https://t.co/mPAjnto8C8] UPDATED),'' Florian Lux, Julia Koch, Ngoc Thang Vu, https://t.co/JQAcMzhfKh

[arXiv Sound @ArxivSound]

``Low-Resource Multilingual and Zero-Shot Multispeaker TTS. (arXiv:2210.12223v1 [https://t.co/JosjykNw75]),'' Florian Lux, Julia Koch, Ngoc Thang Vu, https://t.co/w7dzWaJ5Hn

[Agnieszka Faleńska @afalenska]

Interested in working on biases in AI? There is an open PhD position in my new group on Diversity-Aware NLP Models at @Stuttgart_IRIS @Uni_Stuttgart. More info: https://t.co/aEWR2QFZTa . Please RT.

[Rivers Have Wings @RiversHaveWings]

Stable Diffusion weights are out for academic research purposes upon request: https://t.co/evjhBKYAF0! Good job, everyone! :)

[arXiv Sound @ArxivSound]

``Speaker Anonymization with Phonetic Intermediate Representations. (arXiv:2207.04834v1 [https://t.co/mPAjnto8C8]),'' Sarina Meyer, Florian Lux, Pavel Denisov, Julia Koch, Pascal Tilli, Ngoc Thang Vu, https://t.co/8kqoEOwJXu