Dev Patel

Member of Technical Staff @BoardyAI | Prev. Founding Engineer @ Bluejay (YC X25) & AI @Fidelity | Voice AI, LLMs, Finance

[Dev Patel @_devp]

@akshay_pachaar 0.5 GB on a light weight model is completely fixated on fine tuning to your own needs, this is what we need in the open source!

[Dev Patel @_devp]

@jack @wholemars 👀

[Dev Patel @_devp]

@owengretzinger @boardyai @hf0 🫡

[Dev Patel @_devp]

@iamjohndaniels +1

[Dev Patel @_devp]

@Quan made for this since I was born

[Dev Patel @_devp]

@BradAI +1 on the domain expertise especially for cascaded systems

[Dev Patel @_devp]

honestly speaking most times one is enough. 2 is just a safe guard I have since prefix caching lives on one servers GPU's anyway. the only thing is that if it lands on replica 1 and the first turn routes to replica 2 it doesn't help.

so when I was benchmarking this and testing just kept 2. cost is minimal anyways but yes you can easily just do 1! good q

[Dev Patel @_devp]

ai engineers ship great models and awful first impressions.

cache warming is the easiest, highest-leverage way to cut latency, yet almost everyone ignores it, even in voice ai. that obsession with the first second is one reason why @boardyai stands out! https://t.co/AUiJJBE5PO

[Dev Patel @_devp]

@owengretzinger @boardyai lots of improvements soon too 🙌

[Dev Patel @_devp]

@TheJacobKelly me

[Dev Patel @_devp]

@0xAfroTechBoss @boardyai 🫡

[Dev Patel @_devp]

@PH_BJT @boardyai yup! good cascaded voice agents is just all these optimizations to try to replicate the behaviour of s2s lol

[Dev Patel @_devp]

honestly i think it still depends, but right now s2s is much less controllable for developers, especially for custom voices. Most s2s systems support cloning, but it’s harder to synthesize well bc you’re going audio-to-audio instead of using a dedicated TTS model.

The model has to disentangle content, prosody, and speaker identity from the source, which creates trade-offs in timbre (vocal diff) leakage and prosody that a cascaded TTS pipeline avoids. so it has its pros and cons

[Dev Patel @_devp]

@francedot Congrats! Huge 🙌

[Dev Patel @_devp]

@composio nicee! this is crazy

[Dev Patel @_devp]

@bick_nyers completely agree, literally wrote about this yesterday! prefill + prewarming cache.

the text-output problem I mention above is more so about the paradigm shift of using an audio <> action model over a LLM intermediary for reasoning

https://t.co/y7x9Np1bg6

[Dev Patel @_devp]

I do think that voice ai pipelines still route almost everything through full ASR to generative decoding. A lot of latency and cost sits in that text-output step so reformulating the problem as constrained classification (audio to action) instead of open generation is def going to be the new paradigm shift

[Caine Builds 🐧 @CaineArdayfio]

Say hello to Thud: the first Audio Action Model.

It listens, chooses an action, and executes it in 200ms. 10× faster and 20× cheaper than Gemini Live.

Open an app. Navigate a menu. Send a message. Just say it.

Watch me navigate my Mira smart glasses, just by talking to Thud.

[Dev Patel @_devp]

@_josh_meyer_ that’s why you run background noise evals on your voice-agent 🫡

[Dev Patel @_devp]

@GadiBorovich @AntigravitySF hugee gadi 🫡

[Dev Patel @_devp]

If you're building agent loops, this isn't just important for context management, it also drives latency and cost down heavily over time. def recommend reading...

[Ahmad Allabadi @AhmadAllabadi6]

been meaning to write this one for a while https://t.co/JGy8uiZkf9

[Dev Patel @_devp]

@jamielevinson here before 1k likes