Dan Lyth

Research engineer at @sesame. Previously leading speech research at @StabilityAI and @RockstarGames.

[Dan Lyth @danlyth]

Maya and Miles are joined by Simone and Charlie, and have far greater utility than they used to, while still being extremely conversational and low-latency. More info in the blog: https://t.co/8yGu9c1jSO

[Dan Lyth @danlyth]

Sesame preview now available on iOS ❤️

[Sesame @sesame]

A collection of personal agents, crafted for everyday conversation.

Preview available now on iOS.

https://t.co/Hq5ZXAqPvX

[Dan Lyth @danlyth]

@dscape Ah, sorry, fixed now

[Dan Lyth @danlyth]

@dscape Hey, DMs are open :)

[Dan Lyth @danlyth]

@sesame @_apkumar https://t.co/qzwEUfOLi9

[Dan Lyth @danlyth]

We're a small team @sesame, but looking for great people to join. Check out this podcast with @_apkumar to get a feel for how we work. More info at https://t.co/BD2PsOWlhk

[Anjney Midha @AnjneyMidha]

the core research team behind the sesame voice model is <9 ppl

as @_apkumar walked through in our latest 1.5 hr podcast, talent density beats team size most days

[Dan Lyth @danlyth]

@realmrfakename @sesame https://t.co/vxf29q0YBv
https://t.co/GUhV3lJbOi
https://t.co/BPN0hMWZEK
❤️

[Dan Lyth @danlyth]

We're just getting started at @sesame. Check out the demo here: https://t.co/yCyyNKsiw7

[Brendan Iribe @brendaniribe]

We’re exploring a future where the computer isn’t just a tool—it’s a partner with a truly natural voice and personality. No big claims, just early work we’re excited to share. @sesame

[Dan Lyth @danlyth]

@realmrfakename @sesame Yeah, we’re open-sourcing one of the base models (not fine-tuned with the voices you hear in the demo) in the next two weeks. Will be here: https://t.co/BPN0hMXxui

[Dan Lyth @danlyth]

@reach_vb @realmrfakename @sesame Hey, sounds good vb, will be in touch.

[Dan Lyth @danlyth]

Kind words from @seanhollister 🙏. But we've still got a long way to go...

[The Verge @verge]

Sesame is the first voice assistant I’ve ever wanted to talk to more than once https://t.co/TddtBZUZPY

[Brendan Iribe @brendaniribe]

And we’re building hardware.

[Dan Lyth @danlyth]

Nice overview of some of the things we've been working on @sesame. Always a pleasure working with @justLV.

[Justin Alvey @justLV]

Excited to share a peek of what I’ve been working on

We @sesame believe voice is key to unlocking a future where computers are lifelike

Here’s an early preview you can try! 👇

We’ll be open sourcing a model, and yes…
we’re building hardware! 🧵

[Dan Lyth @danlyth]

Delighted to share a little glimpse of what we've been working on @sesame

[Sesame @sesame]

At Sesame, we believe in a future where computers are lifelike. Today we are unveiling an early glimpse of our expressive voice technology, highlighting our focus on lifelike interactions and our vision for all-day wearable voice companions. https://t.co/Edp8V8urgC

[Dan Lyth @danlyth]

@FluorescentGrey @stableaudio @StabilityAI @iScienceLuvr @jordiponsdotme @ednewtonrex @harmonai_org @zqevans @chrlaf

[Dan Lyth @danlyth]

@Dorialexander @pleiasfr @huggingface This is awesome. Do you know approximately how many hours this comes to?

[Dan Lyth @danlyth]

Excellent work by @sanchitgandhi99 and @yoachlacombe reproducing the text-description-to-speech model I developed while at @StabilityAI 👏❤️

[Sanchit Gandhi @sanchitgandhi99]

Introducing Parler-TTS: an inference and training library for high-quality, controllable text-to-speech (TTS) models 🗣️

To fuel the development of open-source TTS research, we are open-sourcing all datasets, training code and our first iteration checkpoint: Parler-TTS Mini v0.1

[Dan Lyth @danlyth]

@HubertSiuzdak Congrats, sounds great! Any code/paper?

[Dan Lyth @danlyth]

@erogol I was wondering the same thing, there’s not a lot of detail on that.

[Dan Lyth @danlyth]

Moving beyond naturalness and WER, they propose a set of sentences that test the model’s ability to deal with compound nouns, emotions, foreign words, paralinguistics (e.g. whispering if the text requires it) etc. etc.

The full test set is included in the appendix. 👏

2/7

[Dan Lyth @danlyth]

There are a bunch of other interesting elements to this work, and it’s worth a read. Plenty of examples on the demo site too. Nice work Mateusz Łajszczak, @guillecambara, Yang Li, and all the other contributors.

https://t.co/VV2SwDmidD

[Dan Lyth @danlyth]

The speech “de-tokenizer” (or decoder) is a convolutional model that’s streamable and 3x faster than their diffusion-based baseline (and also sounds better).
It’s built around BigVGAN.

6/7