74.8% VS 7.8%. THAT'S HOW OFTEN A VOICE AGENT CUTS YOU OFF WHEN IT READS THE TRANSCRIPT VS WHEN IT HEARS THE AUDIO.
speech-to-text keeps your words and throws away everything else.
what the agent never gets:
> the falling pitch that says you're done talking
> the pause that means you're still thinking
> the tone behind a polite "fine"
> who you're talking to. "yes, go ahead" to the agent and to your partner look the same on paper
> the TV or the kids in the background
his fix is a small model that hears the call and makes the quick calls: speak, wait, hand over. the big LLM only writes the reply.
apple got up to 39% fewer errors when it added the audio next to the transcript. meta and cmu cut tool latency 20% by starting the lookup mid-sentence.
I think the biggest gap in voice AI is that agents make decisions from a transcript, not from the audio. I wrote down my thoughts.
I’d love to hear where you disagree. https://t.co/hHmfjlbFKG