Ivan Panakhno

Fullstack @Wix
building AI products after hours

[Ivan Panakhno @van_van1ch]

Compared end-to-end TTFA across 10 voice models from @ElevenLabs, @FishAudio, and @cartesia. I measured from the end of the user’s sentence to the first audible agent response, not just TTS speed.

@ElevenLabs v4 Turbo had the best balance of quality, cost, and latency for my use case.

Setup and results:
https://t.co/H4PhbLSka1

[Ivan Panakhno @van_van1ch]

I compared 10 voice models from @ElevenLabs, @FishAudio, and @cartesia across 312 test conversations covering 15 language and regional variants.

My choice: ElevenLabs v4 Turbo. Here’s how I got there.

When you listen to a generated audio clip, it’s easy to focus on how pleasant the voice sounds. In a live conversation, something else becomes very noticeable: the pause between finishing your sentence and hearing the agent respond.

I focused on three things:

• Voice quality and naturalness
• Generation cost
• TTFA, time to first audible: how long it takes to hear the start of the response

To test this, I built a small evaluation setup on @livekit. An automated caller works through a scenario, the agent responds, and both sides are recorded. Everything goes into an HTML report where I can listen to conversations, compare models, inspect transcripts, and review latency measurements.

The lineup included six ElevenLabs models, Fish S2 Pro and S2.1 Pro, and Cartesia Sonic 3.5 and 3.6.

Languages included English, German, French, Portuguese, Japanese, Spanish, and Ukrainian, with regional variants for English, Spanish, and Portuguese.

The initial scenarios had 10 turns per side. For longer comparisons, I used a scenario with 25 turns per side to hear how the voice holds up over a full conversation.

Two things worth keeping in mind when running these tests:

TTFA depends on the entire voice pipeline.

In my setup, it measures the time from the end of the caller’s speech to the first audible agent audio frame received through LiveKit. That includes end-of-turn detection, STT, LLM, TTS, and audio delivery.

It’s useful for evaluating the conversation experience, but it doesn’t isolate the speed of the TTS model.

Regional wording doesn’t guarantee a regional accent.

You can adapt vocabulary for British English or Spanish from Spain. That alone doesn’t guarantee the corresponding pronunciation. Accent needs a separate listening assessment.

My practical takeaway: ElevenLabs v4 Turbo gives me the balance I need right now. The voice quality works for my use case, the price is acceptable, and I’m happy with the TTFA.

That’s the combination I was looking for in my voice agent.

What usually decides your choice of a voice model: how it sounds, response latency, or cost?

[Ivan Panakhno @van_van1ch]

Interesting results. I also landed on Eleven v4 Turbo after running a full-pipeline TTFA comparison (not just voice quality) across 10 models on LiveKit. The end-to-end latency made more difference in real conversations than pure TTS quality alone.

Wrote up the setup and numbers here: https://t.co/H4PhbLSka1

[Ivan Panakhno @van_van1ch]

I compared 10 voice models from @ElevenLabs, @FishAudio, and @cartesia across 312 test conversations covering 15 language and regional variants.

My choice: ElevenLabs v4 Turbo. Here’s how I got there.

When you listen to a generated audio clip, it’s easy to focus on how pleasant the voice sounds. In a live conversation, something else becomes very noticeable: the pause between finishing your sentence and hearing the agent respond.

I focused on three things:

• Voice quality and naturalness
• Generation cost
• TTFA, time to first audible: how long it takes to hear the start of the response

To test this, I built a small evaluation setup on @livekit. An automated caller works through a scenario, the agent responds, and both sides are recorded. Everything goes into an HTML report where I can listen to conversations, compare models, inspect transcripts, and review latency measurements.

The lineup included six ElevenLabs models, Fish S2 Pro and S2.1 Pro, and Cartesia Sonic 3.5 and 3.6.

Languages included English, German, French, Portuguese, Japanese, Spanish, and Ukrainian, with regional variants for English, Spanish, and Portuguese.

The initial scenarios had 10 turns per side. For longer comparisons, I used a scenario with 25 turns per side to hear how the voice holds up over a full conversation.

Two things worth keeping in mind when running these tests:

TTFA depends on the entire voice pipeline.

In my setup, it measures the time from the end of the caller’s speech to the first audible agent audio frame received through LiveKit. That includes end-of-turn detection, STT, LLM, TTS, and audio delivery.

It’s useful for evaluating the conversation experience, but it doesn’t isolate the speed of the TTS model.

Regional wording doesn’t guarantee a regional accent.

You can adapt vocabulary for British English or Spanish from Spain. That alone doesn’t guarantee the corresponding pronunciation. Accent needs a separate listening assessment.

My practical takeaway: ElevenLabs v4 Turbo gives me the balance I need right now. The voice quality works for my use case, the price is acceptable, and I’m happy with the TTFA.

That’s the combination I was looking for in my voice agent.

What usually decides your choice of a voice model: how it sounds, response latency, or cost?

[Ivan Panakhno @van_van1ch]

@humayun_x @ippolit1337 @ElevenLabs @FishAudio @cartesia Thx, will check it 🤝

[Ivan Panakhno @van_van1ch]

I compared 10 voice models from @ElevenLabs, @FishAudio, and @cartesia across 312 test conversations covering 15 language and regional variants.

My choice: ElevenLabs v4 Turbo. Here’s how I got there.

When you listen to a generated audio clip, it’s easy to focus on how pleasant the voice sounds. In a live conversation, something else becomes very noticeable: the pause between finishing your sentence and hearing the agent respond.

I focused on three things:

• Voice quality and naturalness
• Generation cost
• TTFA, time to first audible: how long it takes to hear the start of the response

To test this, I built a small evaluation setup on @livekit. An automated caller works through a scenario, the agent responds, and both sides are recorded. Everything goes into an HTML report where I can listen to conversations, compare models, inspect transcripts, and review latency measurements.

The lineup included six ElevenLabs models, Fish S2 Pro and S2.1 Pro, and Cartesia Sonic 3.5 and 3.6.

Languages included English, German, French, Portuguese, Japanese, Spanish, and Ukrainian, with regional variants for English, Spanish, and Portuguese.

The initial scenarios had 10 turns per side. For longer comparisons, I used a scenario with 25 turns per side to hear how the voice holds up over a full conversation.

Two things worth keeping in mind when running these tests:

TTFA depends on the entire voice pipeline.

In my setup, it measures the time from the end of the caller’s speech to the first audible agent audio frame received through LiveKit. That includes end-of-turn detection, STT, LLM, TTS, and audio delivery.

It’s useful for evaluating the conversation experience, but it doesn’t isolate the speed of the TTS model.

Regional wording doesn’t guarantee a regional accent.

You can adapt vocabulary for British English or Spanish from Spain. That alone doesn’t guarantee the corresponding pronunciation. Accent needs a separate listening assessment.

My practical takeaway: ElevenLabs v4 Turbo gives me the balance I need right now. The voice quality works for my use case, the price is acceptable, and I’m happy with the TTFA.

That’s the combination I was looking for in my voice agent.

What usually decides your choice of a voice model: how it sounds, response latency, or cost?

[Ivan Panakhno @van_van1ch]

@ippolit1337 @ElevenLabs @FishAudio @cartesia @MisoLabsAI is worth a try too. I haven't tested it yet, but next on my list is comparing these paid providers with open-source TTS models, including what it costs to run them yourself

[Ivan Panakhno @van_van1ch]

Here are the latency and pricing comparisons behind my choice.

TTFA includes the full voice-agent pipeline. Provider configurations differ, so these are observed results rather than an isolated TTS speed ranking.

ElevenLabs v4 Turbo gave me the balance of quality, cost and responsiveness I wanted

[Ivan Panakhno @van_van1ch]

@polygamingxyz contrants 🥳

[Ivan Panakhno @van_van1ch]

@benln no 🇺🇦?

[Ivan Panakhno @van_van1ch]

@benln @cursor_ai 👀

[Ivan Panakhno @van_van1ch]

@grok @firecrawl So we can't fully scrape instagram or twitter accounts by this feature?

[Ivan Panakhno @van_van1ch]

@mattpocockuk I will spend my whole day and all my limits playing with /retro in my old chats

[Ivan Panakhno @van_van1ch]

@mattpocockuk very useful🙏

[Ivan Panakhno @van_van1ch]

@ippolit1337 @solana_devs @SuperteamUKR good luck

[Ivan Panakhno @van_van1ch]

@grok @OpenAIDevs but it works same?

[Ivan Panakhno @van_van1ch]

@MaorShlomo Base code has team brain under hood, am I right?

[Ivan Panakhno @van_van1ch]

@mattpocockuk your skills are great, but I really miss one specific flow:

/analyze-repo

For the first time you open an unfamiliar repo and want the agent to produce a practical map:
- architecture overview
- key domains/modules
- risky areas
- test/build/deploy entry points
- "where should I start?"

/improve-codebase-architecture is close, but this feels like a different onboarding skill

[Ivan Panakhno @van_van1ch]

@letstri I have same issue, but with Claude