Ayush Bhattacharya

Building for people who struggle to operate apps and websites.

[Ayush Bhattacharya @beayushb]

The most common way a voice agent fails isn't mishearing a word. It's talking at the wrong moment.

In English that's mostly a solved-enough engineering problem. Switch to Hindi, or the Hindi-English mix people actually speak on a phone, and it comes apart.

[Ayush Bhattacharya @beayushb]

Just tested @boson_ai's Higgs TTS 3 at @aika_labs. First open-weights model we've tried that convincingly clones Aika's voice. Self-hosted on an H100, first audio streamed in 137-224ms. Bengali pronunciation still needs work but it does sound like her.

[Ayush Bhattacharya @beayushb]

@vineerpasam Sub-500ms at 10,000 concurrent calls, casually as an interview question. We at @aika_labs believe it's a tradeoff between being over eager and being quick. Would love to see the interviewer's solution.

[Ayush Bhattacharya @beayushb]

Retested Svara-TTS for cloning Aika's voice at @aika_labs, this time with the official inference server. Preset voices sounded okay but the clones were not even close. 215-226 ms to first audio is nice, but we cant use it if it doesnt sound like her.

[Ayush Bhattacharya @beayushb]

@arthurmensch This looks promising. We at @aika_labs found STT endpointing often fires before the caller has actually finished. Have you tested Voxtral on long mid-sentence pauses in Indic and code mixed conversations?

[Ayush Bhattacharya @beayushb]

@paraschopra I already feel some of this with coding agents. I have a separate implementer and reviewer for each ticket, the author never reviews its own work. But if both agree on something I cant understand, how much control do I actually have?

[Ayush Bhattacharya @beayushb]

@JoshBromberg1 Agreed, we at @aika_labs initially scoped for government schemes and exam prep. People call about health, relationships and loneliness far more. What people actually want from personal AI is quite different from what builders assume they want.

[Ayush Bhattacharya @beayushb]

@manosaie I think voice makes this even harder, you cant glance back at what was discussed. A short recap of where you left off before resuming would help. A separate app still needs to solve that, otherwise you're just asking the agent to remember harder.

[Ayush Bhattacharya @beayushb]

@imsanyi Congratulations on the launch and all the best with Millow!

[Ayush Bhattacharya @beayushb]

We did an exercise today on what an ideal conversation at @aika_labs should feel like. A lot of the rigidity comes from flows we impose ourselves. Latency and turn taking have technical constraints, but those dont require the conversation to follow a script.

[Ayush Bhattacharya @beayushb]

@CVestergaard_ @sesame @juberti Agreed on the data. We at @aika_labs fine tuned Smart Turn on our own calls and it beat our text classifier. No manual annotation either, labels came from whether the caller kept talking after a pause or let Aika reply without continuing.

[Ayush Bhattacharya @beayushb]

@typesafeai Agreed, for our browser agent comparison @aika_labs I asked for time to verified task completion, including recovery. A browser being faster in isolation doesnt tell me if the agent will actually finish the task any sooner.

[Ayush Bhattacharya @beayushb]

TIL that RMSProp was never published in a paper, it was taken from @geoffreyhinton's lecture notes. Adam or a variation of Adam is used in training almost every model out there and RMS Prop is a key aspect of Adam. Some people are just too EPIC.

[Ayush Bhattacharya @beayushb]

@ActuallyIsaak This is nice. Our turn model at @aika_labs scores audio before the transcript is ready, so speculation still has to wait for ASR. Are tool calls held until the turn is confirmed too? Throwing away the text is easy, undoing a tool call that changes underlying state isnt.

[Ayush Bhattacharya @beayushb]

@voicearena_ai Congratulations on these results. Particularly encouraging that the gains hold on unseen conversations with the same model and training recipe. Still soo much room to improve Indic ASR by investing in conversational data.

[Ayush Bhattacharya @beayushb]

@_simonsmith The voice mode bit is interesting. We at @aika_labs are building a personal agent you talk to on a normal phone call. Different audience though, people in India who screen and text first tech has largely left out.

[Ayush Bhattacharya @beayushb]

Just traced our @deepseek_ai slowdown at @aika_labs to provider rate limits triggering fallbacks. Same model, median time to first token went from 807ms to 2,588ms. We've now pinned it to @DeepInfra's FP8 priority endpoint and disabled provider fallbacks.

[Ayush Bhattacharya @beayushb]

@defoemark This looks promising. We at @aika_labs have a fine-tuned Smart Turn v3.2 model but long mid-sentence pauses in Hindi are still hard to get right. Have you tested TYGER on Indic and code mixed conversations?

[Ayush Bhattacharya @beayushb]

Just removed a subagent package from my coding agent setup. It was exposing 15 tools in every session and was only compatible with 1 harness i.e. @pidotdev . Replaced with a CLI that just spawns agents in @herdrdev giving me full cross compatibility between Claude Code and Pi.

[Ayush Bhattacharya @beayushb]

@LightspeedIndia @kumarharsha2212 @HimanshiET @EconomicTimes Agreed on the interface. We at @aika_labs are building for people in India that screen and text first tech leaves out. They just make a normal phone call. Usefulness gets the first call, trust over repeated calls brings them back.

[Ayush Bhattacharya @beayushb]

@ambassador_grim Using Pi for this. I have Claude Code delegate work to Pi subagents when I want non-Anthropic models. So I can keep Claude Code as the orchestrator without being tied to its models for implementation.