Ramya Venkat

GM, Remote HRIS | YC Alum

[Marcelo Lebre @marcelolebre]

I rarely celebrate but @Jobvo and I have been quietly building the backbone for modern global payroll and we finally get to share it.

@remote will power @Workday global payroll - beautifully crafted infrastructure and powered by next-gen AI.

The future of payroll is here.

[Remote @remote]

@Workday + Remote

Expanded partnership set to eliminate payroll complexity.

Workday GO Global Payroll. Coming very soon. ⚡
👉 https://t.co/D6DIbW2t3i

[Job @Jobvo]

I get asked about my video setup almost daily.

Here it is.

[Ramya Venkat @ramyavmani]

@aarthir @SchemaVentures Congrats @aarthir!!

[kwindla @kwindla]

Introducing Pipecat Cloud, infrastructure for open source voice AI agents.

If you're building voice AI agents with @pipecat_ai, you have lots of options for hosting your agents: anywhere you can run a Python process and terminate WebSocket or WebRTC connections.

But managing agents in production, on rock solid infrastructure, with observability, autoscaling, blue-green deployments, and everything else needed for real usage at scale is not trivial. Devops is a big category of questions and conversation threads in the Pipecat Discord.

So we built a platform specifically for voice AI.

I've been describing Pipecat Cloud as
➡️ a "Kubernetes wrapper," or
➡️ "Heroku for voice agents, or
➡️ "you just push us a Docker container and we do the rest."

If you're building voice agents with Pipecat, take a look at Pipecat Cloud and tell us what you think.

[kwindla @kwindla]

Evals for voice AI agents — Gemini 2.0 sets a new standard for reliable instruction following. The team at @covaldev does a lot of interesting work with synthetic data and evaluations for voice AI agents.

In this video @bnicholehopkins walks through an eval comparing Gemini 2.0 and GPT-4o. These two models are currently the best-performing LLMs for conversational voice AI.

The results are subtle, but the tldr is:
- Gemini follows the eval task instructions more reliably.
- GPT-4o completes all the steps in the eval tasks more often.

One thing that a test like this highlights is the need for task-specific evals. Models have different strengths. Often a slightly different approach to prompting is required to leverage a model's strengths and work around its weaknesses.

That means that if you're serious about building voice agents, you need to build your own evals. (Coval's tools can help you do that.)

For example, you might be using a larger context window than average, or defining more tools (functions) than average. So "off the shelf" evals might not actually tell you which model will perform best for you.

In general, the things you want to test in a functional eval — the building blocks for reliable voice agent performance — are:

1. Instruction following — does the LLM "remember" and correctly follow the instructions in your prompt, even as the context window gets longer during a multi-turn conversation.

2. Function calling — almost all voice agent workflows depend heavily on function calling. Does the LLM call the functions you define, when needed, with the correct arguments. Can the LLM call multiple functions at a time to perform complex operations?

3. Context awareness — can the LLM pull relevant information out of the context as the context length and the complexity of information in the context window grows? Imagine the metaphorical "needle in the haystack." Can the LLM find the needle when it needs to!

The performance of the best LLMs is improving rapidly. GPT-4o today is much better than GPT-4o was three months ago. Gemini 2.0 is a very impressive improvement over Gemini 1.5.

I'm particularly excited about new capabilities of Gemini 2.0 that we mostly haven't written evals for, yet, in the voice AI community: the ability to combine function calls flexibly, built-in code execution, built-in search.

[kwindla @kwindla]

Congratulations to the @Vapi_AI team on this well-deserved milestone. If you're interested in building conversational voice agents, you should check out Vapi. Easy to get started with, lots of features, proven ability to deliver at scale.

@jordan_dearsley was one of the first people to start building conversational voice experiments in the current era of AI. The best startups often grow out of a passion project — something an engineer just can't not work on. Vapi is one of those startups!

All this experience shows in the Vapi product, today. Their intense customer focus shows in the product and feature set, too. I've had numerous conversations at voice AI events with Vapi customers. A common theme is, "that team is so responsive." (And "cracked." The word cracked comes up a lot.)

Looking forward to seeing what's next.

[Ramya Venkat @ramyavmani]

Build real-time voice agents with Gemini, Pipecat, and Daily! Read more about our partnership with Google to build Multimodal Live API support into @pipecat_ai as part of the Gemini 2.0 launch

[Daily @trydaily]

Gemini 2.0 launched today. Amazing multimodal capabilities, long context windows, fast response times, built-in tools, and top-of-the-leaderboards reasoning capabilities.

Plus a new API — the Multimodal Live API — for conversational AI applications, like voice agents and multimodal copilots.

@Google and Daily have partnered to build Multimodal Live API support into the @pipecat_ai Open Source SDKs for Web, Android, iOS and C++.

The Pipecat SDKs come with echo cancellation and noise reduction, device management, event abstractions, React hooks, and more. They support both direct connections to the Gemini WebSocket API, and WebRTC routing on Daily's global ultra-low latency network.

Build realtime voice agents with Gemini, Pipecat, and Daily.

Links to docs and starter kits in the thread below (1/4)...

[kwindla @kwindla]

Project submissions are rolling in for the $20,000 in prize money up for grabs this afternoon at the Open Source Voice and Video AI Hackathon.

Awards presentations — 5:00 at @solarislll. 1680 Mission St, SF.

If you're in SF and want to see projects like this one, by @maxwelllwang, and talk to the people who created them, you are welcome to come by.

[kwindla @kwindla]

Customer satistfaction scores go up with Voice AI deployments.

We're seeing what @omooretweets writes about here, in real-world Voice AI deployments.

Today's customer support operations are constrained on three axes:
1. staffing availability
2. access to the right information at the right time
3. process and technology

Voice AI provides step-function improvements in 1 & 2 , and gives companies an opportunity to solve 3 in high-leverage/high-ROI ways.

It's impossible to have enough humans on call to manage peak support volumes. (Staffing availability.) So if you can only call your health insurance support line after you get off work, for example, your wait times are going to be pretty long. Voice AI agents scale in a way that a human staff can't. It's hard to over-state how big a benefit this is for quality of customer experience.

I'm down in the trenches doing these deployments, so I see the early customer satisfaction numbers that directly compare Voice AI agent experiences to human agents. Voice AI agents are already very good, beating human agents at a wide range of tasks. But even if you think you'd always rather talk to a human, or don't think a Voice AI agent is a good fit for specific support contexts, being able to deploy auto-scalable Voice AI agents that can handle common tasks will massively improve the general customer experience by reducing wait times overall.

And why are Voice AI agents performing so well in tests and early deployments?

A big reason is the ability of LLMs to make use of large amounts of semi-structured data, quickly. You can pull all of a customer's account records into a Large Language Model's context (maybe with a little bit of contextual/RAG filtering) and get immediate, accurate answers to questions that previously required a human agent to go step-by-step through complex records. (Access to the right information at the right time.)

In fact, part of the bottle-neck that makes it hard for human agents to do their jobs is that the backend systems they use often require clicking around through multiple screens and tools. In the worst case, different agents have access to different databases. (Process and technology issues.)

If this were easy to fix, it would already have been fixed. But LLMs allow these systems to be improved much more cheaply than before, and provide much more value on the back side of investments in process and tech improvement.

LLMs are very good "adapter layers" that can sit on top of older systems. This is useful for both Voice AI agents and human agents! It's often relatively simple now to create a unified knowledge tool by pulling data from multiple systems and doing prompt-engineering work to make a SOTA LLM the interface that both Voice AI agents and human agents use to query and understand the data in legacy back-end systems.

This is a big, big change in how enterprise software is built. Forward-looking companies are investing in these tools today and seeing positive initial results.

In my experience, almost all businesses — large and small — want to provide terrific customer support. It's going to seem normal pretty soon to talk in an open-ended way to a helpful AI whenever you call a customer support line or a small business.

[Daily @trydaily]

Today we’re launching Daily Bots, the ultra low latency Open Source cloud for voice, vision, and video AI.

Build voice-to-voice with any LLM, at conversational latencies as low as 500ms.

With Daily Bots, developers can:

*️⃣  build with Open Source SDKs
*️⃣  mix and match the best Generative AI models for specific use cases
*️⃣  run at scale on Daily’s real-time global infrastructure

We've partnered with @AnthropicAI, @cartesia, @DeepgramAI, and @togethercompute for this launch. Our goal is to combine the best tools, best developer ergonomics, and best infrastructure for real-time AI into a single platform.

Daily Bots is the culmination of the last 18 months of work we've done with customers and partners, pioneering real-time and voice-to-voice AI in production. The two fastest growing Open Source real-time AI projects came out of this work: @pipecat_ai and RTVI.

*️⃣  Daily Bots apps are built using the RTVI Open Source SDKs for the Web, iOS, and Android. Your Daily Bots code will run anywhere that supports the RTVI standard. (Or you can run your own infrastructure.)

*️⃣  Your Daily Bots can also answer the phone. (You can buy a phone number from us with a single curl command.)

*️⃣  Bring your own API Keys and use any inference provider that supports OpenAI-compatible APIs. Or run your own models and connect your bots to your infrastructure.

For a highly non-serious take on the kinds of things we’ve been creating for ourselves, as we’ve worked on Daily Bots, check out the video.

We'd love to hear what multi-modal, real-time AI directions are most interesting to you. We can't wait to see what you build.

[kwindla @kwindla]

this is why we computer

[Varun Singh @vr000m]

RTVI-AI Open Standard: Make an AI voice chat app in 21 lines of JavaScript https://t.co/v9rVpdvehC by @kwindla @ninacali4 @vr000m h/t @rajivayyangar for hunting!

[kwindla @kwindla]

I look extremely tired in this video, for reasons that are probably obvious. But no rest for the wicked.

I'm moderating a panel tonight at @solarissociety with an all-star lineup of people who think a lot about SOTA models, the pace of progress with LLMs, economically valuable use cases, and Open Source.

✨ @rauchg (@vercel)
✨ @craigsdennis (@Cloudflare)
✨ @0interestrates (@juliusai)
✨ @charles_irl (@modal_labs)

And @rajivayyangar is hosting a select set of founder demos, focused on products that incorporated Llama 3.1 today. If that's you, read his thread to learn how you can demo.

Come join us for pizza, conversation, and insights about where generative AI is headed.

RSVP link below ...

[Groq @GroqLLC]

We are proud to collaborate with @trydaily on real-time voice #AI. Check out enterprise voice workflows, such as this healthcare patient intake demo running on #Llama 3.1 405B by @AIatMeta! https://t.co/AXz3MWFONX

#VoiceAI #Meta #Inference #LLM #Llama3

[kwindla @kwindla]

How to build the world's fastest voice AI bot:

- Self-host speech-to-text, LLM inference, and text-to-speech all together in the same container/cluster.
- Route audio over the internet using WebRTC and edge networking.
- Configure timings for voice activity detection, phrase endpointing, and other parts of the pipeline to optimize for latency. (There are trade-offs to doing this!)

Here's a LLama 3 voice bot that has voice-to-voice response times of ~500ms.

We used @DeepgramAI's STT and TTS for this bot, and everything is hosted on @cerebriumai's serverless GPU infrastructure.

[Ramya Venkat @ramyavmani]

@chiefaioffice I hadn’t heard of Thoughtly before. Kwin, are you familiar?

[Ramya Venkat @ramyavmani]

Packed house at @solarissociety tonight! 🔥

[kwindla @kwindla]

298 registrations for the Voice + AI event tonight at @solarissociety.

Come hang out with @rajivayyangar, @stuffyokodraws,
@joshbuckley, @rememberlenny, me, and ~300 more of our closest friends.

https://t.co/3uFBePx2lP

[kwindla @kwindla]

Here's the sign-up link for our @ProductHunt / @trydaily Voice + AI meetup crossover event: https://t.co/3uFBePx2lP

[kwindla @kwindla]

Looking forward to this conversation tomorrow with @deepgramscott about voice AI.

I first met Scott during @ycombinator in 2016. We've been working together almost that long — giving @trydaily's customers integrated access to Deepgram's speech-to-text, natural language understanding, and text-to-speech capabilities.

When Scott first took me through a deep dive into his "new fangled" (my words) machine learning model architectures, I didn't get it! Along with just a few other generative AI companies — @OpenAI most prominently, of course — Deepgram was way ahead of the rest of us.

If you're interested in building with voice AI, please join us for the webinar. It's going to be fun. I build a demo to show. :-)