ICYMI, a new speech-to-speech model from @phonic_co is topping benchmarks
✅ VAmoS Bench, @veris_ai
✅ s2s index, @ArtificialAnlys
✅ T^2-Voice, @SierraPlatform
✅ s2s, @cekuraAi
today we report the turn latency from VAmoS Bench
phonic-1 mogs the Pareto 🔥
deep dive 👇🔗
Agentic speech to speech models are not easy to train. Congrats to @phonic_co for leading the VAmoS bench. We benchmarked this model pre release and it’s a strong model.
Interviewed an engineer friend who works at a traditional major bank.
Very interesting observations:
- Junior engineers are not allowed to push code with AI. Claude Code & Codex are banned from
- Senior engineers can freely use Claude Code & Codex, but only to check their and others work.
- If you push AI-generated PRs, you will get a bad rep within the organization.
- They still have a QA department that does all the reliability on their software.
- He only uses Claude for his personal projects, which there is a lack of, because he works 9 to 7.
Sincerely asking what happens to these people.
If you are not using the frontier technology, it's either that the institution gets replaced, or that eventually, these people get laid off to realize suddenly that the world has moved on, and they just stopped in the pre-Claude Code era.
How to kill a voice agent? Just turn on the TV ☠️
We benchmarked 14 voice agents with a TV playing in the background.
Task completion fell from 38.7% to 8.6%
🧵/
Is Jev the best decision model for voice agents?
We benchmarked 8 "system one" models, letting them control when a voice agent can talk.
We got an answer. But there's a catch 👀
🧵/
Today we are releasing VAmoS Pro, the most realistic benchmark of frontier voice agents for customer service tasks. We ran more than 5,000 calls across 14 voice stacks, and even the best agent passed fewer than half of its calls. 📊 🧵
new voice agent on VAmoS Bench 🔥
built on the Voice Agent API from @DeepgramAI
we ran it through hundreds of simulated phone calls
🔗👇
cc @deepgramscott@dustinshubert_
Gold: MedKit by Bedirhan Keskin from Türkiye
A voice-based clinical simulator where medical students take histories, order labs, diagnose, and get scored on their reasoning against the latest published guidelines.
https://t.co/XcDinJi56w
@rrrvalliani@umarsaif I'd love your opinion on this as you've been supporting these people and very vocal that these will bring a huge change in economy of Pakistan.
Please enlighten me how?