Andrew Hsu

@speak cofounder/CTO - AI language learning

[Andrew Hsu @adhsu]

@speak, we've been working closely with OpenAI on GPT-Live-1 for the past few months, building with an entirely new class of voice model model that continuously listens and speaks at the same time. Here's some of what we learned. https://t.co/KInfDeuRUZ

[Andrew Hsu @adhsu]

Big disagree, imo @pbbakkum is correct here and you’re making weird assumptions about his OAI work… “s2s model” is a pretty well established term meaning the model natively processes and outputs audio with no intermediate text representation. This is the model architecture that enabled the paralinguistic understanding capability. (see original google s2s paper, meta seamless papers, they all define s2s this way)

Also, all of oai’s audio model architectures are pretty obvious if you think about it for a bit 😁

[Andrew Hsu @adhsu]

@pbbakkum 🫡

[Andrew Hsu @adhsu]

https://t.co/GH9LCjpsuz

[Andrew Hsu @adhsu]

@rabi_guha @ollama Very cool! Curious if you have any thoughts about evals for generative UI systems

[Andrew Hsu @adhsu]

dots at dev day!

[Andrew Hsu @adhsu]

Come build a superhuman tutoring agent with me @speak

[Ben Lang @benln]

45+ startups hiring in this thread:

1. @turbopuffer - remote, NYC, Sf
2. @convex - SF
3. @fal - SF / remote
4. @firecrawl - SF / remote
5. @attio - SF, NYC, London + remote
6. @Linear - North America & EU remote
7. @warpdotco - NYC
8. @speak - SF / Seoul / Tokyo
9. @BrightHarborCo - Austin
10. @Vizcom - SF & US remote
11. @PostHog - NA + EMEA
12. @sievedata - SF
13. @artie_labs - SF & remote
14. @concurrencehq - NYC / SF
15. @spacexai - global
16. @EnclaveAI - NYC, SF, TLV
17. @tweetsbyport - US, TLV
18. @RevelHQ - LA / SF / NYC
19. @SnorkelAI - SF & NYC
20. @neatlogs - remote
21. @BabaHQ - NYC
22. Tremendous - remote (Americas)
23. @nucleussec - Florida / fully remote
24. Searchable - London / Salt Lake City
25. @flytbase - Pune, India
26. Indices - London / SF
27. Duna - across Europe
28. @aiunderwriting - San Francisco
29. @zenml_io - SF
30. @nexcade_ai - London
31. Social Fetch - remote
32. @gumloop - SF + Vancouver
33. @LulaConvenience - US remote
34. CLAR AI - Munich
35. @valkai - NYC / SF
36. @KeycardAI - SF
37. Latent Defense - NY
38. @DataUAcademy - Southeast Asia + global
39. @Nooqbook - remote
40. @saasflashstudio - remote
41. @SkydioHQ - US
42. @NotionHQ - global
43. @Replit - global
44. @forus - NYC
45. @harmonic_ai - NYC

[Speak @speak]

Introducing Live Tutor Lessons, powered by @OpenAI GPT‑Live-1: Interactive language lessons with a tutor that get you talking, listens in real time, and helps you find your words. Jump in with a question anytime.

Limited rollout for English and Spanish learners. See it in action ⬇️

[Andrew Hsu @adhsu]

Congrats to the @indent team and @sthupukari on launching! We've been using Indent for well over a year now @speak and it's organically become one of our few internal agents adopted by almost the entire team, and it helps us make more data-driven decisions daily. I've been really impressed with the significant progress they've made on a lot of the nitty gritty hard parts of enterprise agent uptake including auth/IAM and team memory and context. Onwards!

[Sashank @sthupukari]

Introducing @indent, one coworker for your whole company

Indent is a single agent that is built to work with every member of your team, simultaneously. Instead of every one setting up their own agents, use Indent to build one together.

Try it at https://t.co/fUTua8mUOc

[Andrew Hsu @adhsu]

@baseten @zachtratar is so handsome

[Andrew Hsu @adhsu]

@kylerblue @VynseDev I’m in for $2k

[Andrew Hsu @adhsu]

@elizalian ➕➕➕

[Andrew Hsu @adhsu]

@zachtratar Super cool! Excited to try this out

[Andrew Hsu @adhsu]

@justindross Happy customer!!!

[Andrew Hsu @adhsu]

Amazing technical writeup worth reading in full - kudos to the gpt-live team on this work! It's clear that full duplex/bidi is the future of voice and it's really cool to peek under the hood at the complex systems engineering that makes it all "just work". Some observations/notes for voice agent builders:

- The whole post is a huge flex, but especially turning the “improve startup latency” ticket into a new webrtc handshake protocol. Generational scope creep. @juberti what was the original estimate on that issue? :)
- The voice must flow. Nothing matters but continuous realtime audio flow and smoothness. Be extremely clear in your voice agent’s system design about division of responsibilities between the realtime band and async channels/sidecars, and treat the async RPC boundary as a first-class conversational & UX design principle.
- Many other details are direct consequences of optimizing for voice fluidity: prompt caching and avoiding audio-blocking prefills, live context compaction with blue-green-style cutovers, smoothly handling long-running voice sessions (that are much longer than gpt-live’s context window)
- Future models and versions of gpt-live will continue to become more intelligent and handle more tasks natively in the realtime band. As inference speed improves, the model will also be able to spend more tokens thinking and delegating within the realtime response latency budget. This means both the actual delegation and “perceived delegation” boundary will push out.
- Interesting details on the turn-transcript heuristics, e.g. “we prioritize coherence in the displayed assistant responses even when the user speaks in the middle.” The full bidirectional/duplex model arch breaks free of modeling the convo as a traditional ordered list of chat messages, but heuristic/projected semantic turn messages are still critical for most voice agent harness systems you’d want to build around the core live model.
- Realtime full-duplex sessions load your voice agent application server’s CPU (stream handlers, queues, network paths) more heavily than almost any other ordinary application type. Think about the fact that each full-duplex session processes ~100 audio frames per second across both directions. This is a very nontrivial driver of higher infra/capacity costs when scaling realtime voice experiences.
- The bar has been set extremely high for long-running voice agent infra providers, with a lot of additional complexity to support very smooth realtime voice model inference with an explicit temporal dimension. It’ll be interesting to see the industry patterns that emerge here.

[Justin Uberti @juberti]

Just posted our technical deep dive on the GPT-Live system, by @zahanm and yours truly.

[Andrew Hsu @adhsu]

@jxnlco Fall! I’ll hit you up

[Andrew Hsu @adhsu]

@alvinhsia Congrats!!!!

[Andrew Hsu @adhsu]

@blader I follow @simonlast too! 😛

JK - this has been a big unlock for me as well, and I often tell it to call the task review subagent with a different model and to try to reduce complexity

[Andrew Hsu @adhsu]

Strongly agree with this K-shaped divergence happening on software teams. It's existentially urgent to move faster here than what many consider reasonable, because a decent-sized gap today will compound into an insurmountable chasm in output and execution speed and quality by the fall.

This means your team must simultaneously buy into the vision of what is possible and will be possible soon, while also being in the trenches on a daily basis wrestling with and steering agentic systems that still often make mistakes, overcomplicate things, and don't always do what the user is envisioning. Navigating this takes strong, continuous leadership that's also in the trenches.

We're doing a lot at Speak to stay on the cutting edge here as a team - a few examples:
- Continually thinking about what we're still doing manually and following that signal to agentify our repos and processes
- Retooling our engineering hiring process to explicitly test for agentic engineering skill and mindset
- Explicitly trying to build production features while keeping manual code as close to 0% as possible

If you're working at a company that isn't all-in, you should leave before it's too late. Accelerate!

[Eric Glyman @eglyman]

https://t.co/0VslhlEfIe