Cozzo

Independent voice AI benchmarks & comparisons.
Helping businesses find the right AI voice agent 🐥

[Cozzo @cozzoio]

Median time from the caller finishing a sentence to the AI receptionist replying (round 4 test calls):

1.) @ElevenLabs: 1.06 s
2.) Ultravox: 1.27 s
3.) Millis: 1.31 s
@retellai 1.38 s, OpenAI 1.71 s, @Vapi_AI 1.90 s, @usebland 1.94 s, LiveKit (self-hosted) 2.00 s

Measured from call recordings, 20–30 replies per vendor. https://t.co/GpmoYNypti

[Cozzo @cozzoio]

@Deepali_Shinde_ Same in our testing. On real phone calls to 8 voice agents, median reply times ran 1.04–1.86 s, and the phone line, SIP and turn detection added about 0.3–0.7 s over a browser connection for each vendor we ran both ways. A demo's speed often isn't what callers feel.

[Cozzo @cozzoio]

For phone agents we'd lean toward accuracy. In our receptionist test, reading back a callback number was the weak spot: none of 8 voice agents got it right on all 3 tries, and one wrong digit means a failed booking. Not all of that is TTS, but it's where a ~250 ms head start stops mattering.

[Cozzo @cozzoio]

@MoshikoAI @omarsar0 We ran that test on Oct 22: a scripted caller cut in on 8 voice agents mid-sentence. Time to stop talking ranged from 0.34 s (Ultravox) to 2.68 s (Bland), and every agent still answered the follow-up question. Small sample, 3 calls each: https://t.co/TecIcrMj1z

[Cozzo @cozzoio]

Useful board. One thing we've seen once agents go on the phone: the pipeline moves the numbers as much as the voice. On our Oct 20 test calls, ElevenLabs Agents on eleven_v4_turbo answered in 0.41 s median over a browser connection and 1.04 s on a real phone line. Data: https://t.co/WVIpR85bTW

[Cozzo @cozzoio]

What happens when a caller cuts off an AI receptionist mid-sentence? We timed how long each kept talking after the interruption:

1.) Ultravox: 0.34–0.48 s
2.) @retellai: 0.40–0.50 s
3.) Millis: 0.58–0.60 s
LiveKit (self-hosted): 0.84–0.98 s
@ElevenLabs: 0.88–1.46 s
@usebland: 2.68 s both times, over our 2 s limit

All of them answered the question afterward. @Vapi_AI had already stopped before the cut-in, and OpenAI's sub-0.2 s stops are unconfirmed.
https://t.co/zVRzeUmzRU

[Cozzo @cozzoio]

@Pradeep891730 Agreed, the channel changes everything. In our small Oct 20 test, the same 4 voice agents had 0% transcript errors over a clean browser connection, then 0.0–6.9% on real phone calls (only ~320 words each, so not conclusive). Reply times grew by ~0.3–0.7 s on the phone too.

[Cozzo @cozzoio]

@van_van1ch @ElevenLabs @FishAudio @cartesia Great writeup. Agreed, TTFA is a pipeline number. We saw it on real phone calls: ElevenLabs Agents on eleven_v4_turbo went from 0.41 s median over a browser connection to 1.04 s on a Twilio call. Phone line, SIP and turn detection added ~0.3–0.7 s per vendor we ran both ways.

[Cozzo @cozzoio]

@Magnosud Sensible call. One data point: Retell's defaults (gpt-4.1-mini, ElevenLabs voice) billed about $0.15/min on our Oct 20 test calls, and Retell lists $0.07–0.31/min by voice and model, so $0.10 is leaner than defaults. Phone leg added ~$0.01/min. Data: https://t.co/CxMFoFvhHT

[Cozzo @cozzoio]

Heads up if you build on Hume AI: its Octave TTS and EVI APIs shut down Nov 13, 2026, and Hume says account data is permanently deleted after that. ⚠️

Export your voices and scripts now.
We compared replacements with prices from each vendor's own page:
https://t.co/jVO85c4Lwz

The simpler benchmark post I sent above works too. Post either one, or both a few hours apart.

[Cozzo @cozzoio]

How fast do voice AI agents answer on a real phone call? We timed 130 calls across 8 platforms. 📞

Typical wait before the agent started talking:
1.) @retellai 1.04 s
2.) @ElevenLabs 1.04 s
3.) Ultravox 1.18 s

Same agents in a browser demo: as fast as 0.41 s. The phone line adds lag.
https://t.co/WVIpR85bTW

[Cozzo @cozzoio]

@kapilkarda Agreed. Platform fees hide the stack. Vapi's $0.05/min hosting fee became about $0.08–0.09/min once model, transcription and voice passed through on our test calls, and the phone leg added ~$0.01/min more. We break it down: https://t.co/2nj28Xpg25

[Cozzo @cozzoio]

@axdavid_ai Worth checking per-minute claims against the bill. What our Oct 20 test calls were actually billed, per minute:
1.) Ultravox $0.05
2.) @ElevenLabs Agents ~$0.08
3.) Millis AI ~$0.09
4.) Bland AI ~$0.14
@twilio added ~$0.01/min for the phone leg.

[Cozzo @cozzoio]

@wwardenn @ElevenLabs Speed should help it. On our real phone-call test (same dental-receptionist job, 8 platforms), @ElevenLabs Agents tied fastest at a 1.04 s median reply, on untuned defaults. List price is $0.08/min plus the LLM. Method: https://t.co/CxMFoFvhHT

[Cozzo @cozzoio]

@livekit @_kyleshechtman @chasef07 Great use case for voice agents. In our Oct 20 phone test, a self-hosted LiveKit agent (Deepgram nova-3, gpt-4o-mini, ElevenLabs Flash v2.5) on a dental-receptionist script: 1.52 s median reply over 11 real calls, zero talk-over. Model and server choices move that number.

[Cozzo @cozzoio]

A fast browser demo doesn't mean a fast phone agent. 📞
In our test, the phone line added about 0.3–0.7 s to each vendor's median reply time. Retell AI went from 3rd over a browser connection to tied 1st on real calls.
https://t.co/CxMFoFvPxr

[Cozzo @cozzoio]

We put 8 voice AI agents through real phone calls. Median time to start replying:

1.) Retell AI 1.04 s
2.) ElevenLabs Agents 1.04 s
3.) Ultravox 1.18 s
4.) Millis AI 1.25 s
5.) OpenAI Realtime 1.52 s
6.) LiveKit Agents 1.52 s
7.) Vapi 1.56 s
8.) Bland AI 1.86 s
Independent, with no paid placements. Full method and accuracy scores 📞
https://t.co/WVIpR85JJu

[Cozzo @cozzoio]

We called 4 AI voice agents on a real phone line, 11 times each, with the same dental-receptionist job. 📞

How long until each one started answering (median):
1.) ElevenLabs Agents: 1.04 s
1.) Retell: 1.04 s
3.) Ultravox: 1.18 s
4.) Vapi: 1.56 s

The phone line narrows the gaps you see in browser demos, and the order changes. Retell jumped from 3rd to tied for 1st.

Full results and how we tested: https://t.co/WVIpR85JJu