coval

Simulation and evaluation for AI agents (chat & voice agents)

[Brooke Hopkins @bnicholehopkins]

I’m excited to announce @covaldev raised a $28M Series A, led by @NorwestVP with participation from @Base10Partners, @twilio Ventures and @ycombinator, among others.

I started Coval with the belief that the simulation and evaluation approaches that made autonomous vehicles a reality could unlock the same for voice AI. That parallel has only gotten sharper as enterprises deploy autonomous conversational agents at scale.

Voice and chat are the new interface in the AI era. Talking is the most natural way to interact with complex autonomous systems. The same way web and mobile transformed every enterprise, every company will have a voice agent as the front door to their product. This platform shift is happening now.

From Y Combinator to working with companies like @DeepgramAI, @perplexity_ai , and @Zoom, it's been a wild ride. We watched voice AI go from an emerging modality to every Fortune 500 enterprise building a conversational interface.

Autonomous agents are the future. But you can't ship and hope for the best when agents are taking on real work. The stakes are too high.

Coval will be what enables us to actually trust those agents. The platform to help teams reliably scale voice and chat agents.

We wouldn’t be here without Coval’s founding engineers @rheacoval and Kobi Hudson. Rhea took a chance on Coval before we had a single customer, a line of code, or an office that wasn't my couch. Kobi left @Waymo after 10 years building foundational simulation systems to help us build the frontier of voice simulation.

To the full Coval team - you all are insane engineers and builders who inspire me every day. Rob Young Alejandra Vergara Henry Finkelstein Callum Reid Dakota Mallen Dana Dzik Mallory McLoughlin Christopher Kuester Jake Levi Brooke Hartley Kappi Patterson

To all our investors - @NorwestVP, @Base10Partners Partners, @twilio Ventures, @ycombinator, @alumniventures, Swift Ventures, Fortitude VC and so many more. You are what made Coval possible.

To @koh_terai who brought my video vision to life.

And last but certainly not least, thank you to everyone who's been building with us.

We're just getting started.

[Deepgram @DeepgramAI]

Flux Multilingual is live.

Real-time conversational speech-to-text for voice agents in 10 languages, with monolingual-grade accuracy, turn detection, and code-switching.

Deploy once and launch globally.

Learn more → https://t.co/lhhUZ2maTo

[coval @covaldev]

There’s a persistent belief in startups that you have to choose between moving fast and building stable systems.

Early on, it feels true. Infrastructure looks like overhead. Monitoring, evals, rollback logic, routing discipline — none of it directly ships a new feature. When you’re racing to find product-market fit, stability can feel like something you earn later.

But in systems like voice AI, the tradeoff turns out to be mostly imaginary.

What actually slows teams down isn’t infrastructure. It’s the absence of it. It’s spending days debugging a regression because no one captured per-turn latency. It’s pushing a prompt update and not knowing whether it improved resolution or just shifted failures somewhere else. It’s being afraid to deploy because you can’t predict the blast radius.

When you build observability and structured evaluation early, releases get less dramatic. You know what changed. You know what broke. You can attribute performance movement to a specific component instead of debating guesses in Slack. Rollbacks become procedural instead of emotional.

Speed without feedback is chaotic. Stability without iteration is stagnation. The teams that compound fastest figure out how to build both at the same time — tight feedback loops, clear instrumentation, and the confidence to ship because measurement is built in.

Over time, the supposed tradeoff disappears.

What remains is a system that improves quickly precisely because it’s stable enough to trust.

[coval @covaldev]

🎉 Exciting News! Please join us in welcoming Rob Young to the Coval team!

Before joining Coval, Rob designed products at Google and Apple, and founded and led the product design studio OM. He brings a wealth of experience crafting elegant, user-centered experiences at some of the world's most innovative companies.

Outside of work, you'll find Rob spending time with his family, skateboarding, doing yoga, and painting.

Welcome to the team, Rob! 🚀

[coval @covaldev]

🧵 This week in conversational AI:

🎤 @OpenAI Realtime takes a big step forward — sub-500ms latency, sharper function calling, non-verbal cue detection, seamless language switching, and SIP (telephony) support. Already benchmarked at https://t.co/bBpsqqjnaa, and the gains on instruction following are a real unlock for production-grade voice agents.

🧪 @cartesia demos custom evals for voice agents — defining, validating, and reusing LLM-powered metrics on real calls. A strong signal that observability is becoming core infrastructure for scaled agents.

🔗 @DeepgramAI × Cloudflare : Deepgram’s STT + TTS now run directly on Cloudflare’s global edge. Low-latency voice agents deployed serverlessly — making conversational AI faster and more accessible worldwide.

💡 Together AI named an IA40 Winner in the “Enabler” category — powering over 800K developers building AI-native apps. Great to see Cartesia , Arcee AI , and ElevenLabs also recognized, showing how fast the real-time voice ecosystem is maturing.

📣 Coming up (Sept 3, SF): 𝗩𝗼𝗶𝗰𝗲 𝗔𝗜 𝗣𝗿𝗼𝗻𝘂𝗻𝗰𝗶𝗮𝘁𝗶𝗼𝗻 𝗖𝗵𝗮𝗹𝗹𝗲𝗻𝗴𝗲𝘀, hosted by @Covaldev , @rimelabs, and @ConverseNowAI. We’ll dig into why models stumble on words like Chipotle, tradeoffs between emotional range and consistency, scaling TTS in production, and war stories from the trenches. If you’re building voice AI, this is for you.
P.S: We'll be sharing more behind-the-scenes of our Coval Benchmarks 2.0, with a special focus on TTS! Make sure to sign up for the event! https://t.co/hgGww7kbwF

What did I miss?

[coval @covaldev]

🎉 Exciting News! Please join us in welcoming Loren Phillips to the Coval team!

Before joining Coval, Loren worked on autonomous vehicle safety validation at Zoox, Amazon’s self-driving subsidiary. He also brings a diverse research background spanning nanophotonics, soft robotics, and medical devices.

When he’s not massively advancing how AI agents are tested and deployed, Loren spends his free time road cycling, running his Etsy store of 3D-printed art, and developing Voice AI applications — from cloning sites to dating platforms.

Welcome to the team, Loren! 🚀

[Heavybit @heavybit]

🎙️ Testing non-deterministic AI requires a new approach. Forget exact outputs; it's about probabilistic success. @bnicholehopkins of @covaldev breaks down the shift on the latest Generationship episode. Tune in! 🎧
https://t.co/s9xQOpKsXh

[swyx @swyx]

don't miss that OAI also published a prompting guide WITH RECEIPTS for GPT 4.1 specifically for those building agents... with a new recommendation for:

- telling the model to be persistent (+20%)
- dont self-inject/parse toolcalls (+2%)
- prompted planning (+4%)
- JSON BAD - use XML or arxiv 2406.13121 (GDM format)
- put instructions + user query at TOP -and- BOTTOM - bottom-only is VERY BAD
- no evidence that ALL CAPS or Bribes or Tips or threats to grandma work

[coval @covaldev]

🎉 Exciting News! Please join us in welcoming Kobi Hudson to the Coval team!

Prior to Coval, Kobi spent 8 years at Waymo working alongside our own Brooke, where he played a pivotal role in building Waymo's early simulation infrastructure - some of the foundational systems powering autonomous vehicle testing today. Now, he’s bringing that simulation superpower to Coval, helping us build the future of how AI agents are tested, evaluated, and deployed.

When he’s not leveling up infrastructure, you’ll find him at the beach with his wife and dogs or hitting up skateparks around the Bay 🛹

Welcome to the team, Kobi! 🚀

[Jeff Harris @jeffintime]

very detailed and fair evals and impressions on the new STT+TTS models. thx @covaldev

[Peter Bakkum @pbbakkum]

The folks at @covaldev did a deep dive with our audio models, some great examples of pronunciation with difficult words

[Brooke Hopkins @bnicholehopkins]

🎙 OpenAI’s dropped new voice AI Models- get Coval's scoop and see the benchmark results 🫢

@OpenAI just dropped a new text-to-speech model, We’ve been testing gpt-4o-mini-tts and the prosody, pronunciation, and controllability are next-level.

💭 Prosody that feels real: The intonation, rhythm, and expressiveness are way ahead of previous versions.
🔢 Impressive number pronunciation: No more awkward digit reading—finally, a TTS that understands numbers in context.
🧑‍💻 Tags for fine-tuned control: You can tweak tone, style, and delivery for different use cases.
📊 Higher latency compared to other providers, but better pronunciation: There’s a tradeoff, but for many applications, the quality boost is worth it.

🚀 They also launched
- New transcription models that are way better than Whisper
- Better noise cancellation for transcription
- Voice agent SDK & better tooling
- Semantic voice activity detection

At @covaldev, we’re diving deep into testing these models for our customers! The potential for conversational AI is massive.

If you’re working with voice AI, these new models deserve your attention. What’s your take on the latest OpenAI TTS updates? Let’s discuss! ⬇

[coval @covaldev]

Coval + @rimelabs: Bringing Lifelike Voices to AI Simulation

Excited to announce our new integration with Rime! 🎉

Coval's mission is to provide the most comprehensive voice simulation platform for testing AI agents, and today we're adding Rime's incredibly lifelike voices to our toolkit to simulate your end-users.

What makes Rime special? Their voices sound remarkably natural because they train on real conversations recorded in their own studio.

With Coval, you can now simulate an even wider variety of user interactions without manually calling your agent.
Because your customers are diverse, your testing should be too.

Want to see how Coval + Rime can improve your AI agent testing? Sign up for Coval today.

[Brooke Hopkins @bnicholehopkins]

@cartesia's latest model Sonic 2.0 just dropped and the @covaldev team couldn't wait to try it out, so we already ran some benchmarks. Some of our takeaways:

⚡ Sonic 2.0 is lightning fast

🔢 Cartesia has gotten way better at alphanumeric sequences ($52,000, [email protected], ) and

💬 Same with disfluencies (um, yah, ...)

The best part? We run these continuously, so you can check in anytime to see the latest and greatest benchmarks on the performance of these models.

Checkout some of the industry examples to try for yourself!

Coval is the first and only provider to be running continuous benchmarks. What other metrics do you want us to add?

[coval @covaldev]

🎉 Excited to announce that @covaldev and @langfuse are officially integrated - so you can test & debug Voice AI with confidence.

Voice agents require both conversation-level testing + message-level observability to be production-ready. Our integration bridges this gap, giving developers complete visibility across their voice applications - from testing full conversations to monitoring individual messages.

✅ How it works:

•Simulate conversations & track agent performance metrics using @covaldev
•One-click access to @langfuse traces to debug on a message-level
•Want to verify improvements? Re-run your Coval simulation

🤝 Why we’re partnering:

Voice AI developers need insights at every level of their stack. While Coval excels at end-to-end conversation testing, Langfuse's granular message tracing completes the picture.

We are very excited to partner partner with the amazing team at Langfuse and can’t wait to further enhance voice agent debugging with @MarcKlingen, @rawert, @maxdeichmann & team 🚀

Set up your Coval 🤝 Langfuse integration in just a couple of minutes and share your feedback!

[coval @covaldev]

🔒 Proud to announce that @coval is now HIPAA compliant!
Healthcare stands at the forefront of the voice AI revolution. From improving patient care to streamlining clinical workflows, voice AI is fundamentally reshaping the healthcare industry. This makes security and compliance not just requirements, but essential foundations for innovation.

That's why we're proud to announce our HIPAA compliance – a testament to our commitment to building secure, reliable voice AI solutions for one of the world's most critical industries.

Special thanks to the @oneleet team for their partnership in achieving this milestone 🚀

[coval @covaldev]

When you want to launch your new voice agent but you haven’t run any evals yet...

[coval @covaldev]

NYT article:
https://t.co/KD43Ki4U9o

The Brutalist voice post-production Controversy:
https://t.co/Wn03RRAhym

[coval @covaldev]

🧵 This week in conversational AI 🧵

• Conversational AI in healthcare is set for massive growth, according to this week’s report by Coherent Market Insights! Exciting to see use cases booming across the market, from to Assort Health, Counsel Health and more.careCycle (YC W25)

• Retell AI I released Listen to User Keypad Input, allowing callers to enter numbers directly for a seamless, error-free experience.

• The OG voice AI use case - in-car voice assistants – keeps evolving. Stellantis is doubling down with Mistral AI to enhance their voice AI capabilities. Very excited to continue seeing voice AI improvements in mobility!

• The biggest question this week: How good vs. bad is the impact of voice clones? I know we’re all excited about voice cloning, but The NYT just explored its impact on art and entertainment - specifically - The Simpsons. Check out their interactive piece. Another interesting controversy is the use of Voice AI for post-production in the newly released movie "The Brutalist", where Brady Corbet used voice AI to refine actors’ Hungarian accents.

Curious to get your thoughts in the comments!

[coval @covaldev]

🚀 Optimizing Voice AI with Coval + Retell AI 🚀

Are you building your voice agents with @retellai? Then you'll be excited about this partnership! Coval & Retell AI join forces to enable infrastructure for end-to-end voice agent reliability.

Why should you use Coval for testing your voice agents?

💎 Coval's foundation is based on years of developing and scaling testing infrastructure in self-driving.
💎Automated Simulation at Scale – Generate diverse test cases from transcripts, prompts, or audio, simulating conversations without manual effort.
💎 Custom Metrics & Insights – Define key performance indicators, run thousands of scenarios, and integrate seamlessly with CI/CD pipelines.
💎 End-to-End Observability – Monitor conversations in production to ensure your AI agents perform at their best.

🔥 Exclusive for Retell AI Users: Get 50% off your first 2 months with Coval!

Learn more about our partnership & start optimizing today: https://t.co/Uipqfk8BPn