Edward Zhang

Building @nuance_AI - we are hiring. Previously @Apple. PhD at @uwcse

[Nuance Labs @nuance_ai]

Our Nuplex model is a step-change for realism in AI personas. We are keenly aware of how powerful this technology can be.

Here's how we think about safety, identity, and building good products 🧵

[Edward Zhang @Kyzyx]

@Fxblaqq @YueqianL Thanks for the feedback - we totally agree! We spent a lot more time crafting Emma's persona so we're glad that you noticed that.
We'll continue to improve the personas available on the API!

[Nuance Labs @nuance_ai]

Meet Emma: https://t.co/Pw1Jhe3SuG

Our first AI persona talks to you with her face, not just her voice.

No benchmark properly measures the naturalness of a face-to-face interaction, so we measured against how real people talk to each other.

[Edward Zhang @Kyzyx]

@Charbax @Scobleizer @nuance_ai as the guy at @nuance_ai who is most annoyed by this: every single LLM is posttrained to heavily encourage this behavior 😭. It was even worse about this before our refinement and tuning...

[Edward Zhang @Kyzyx]

@mariusschober @Scobleizer @nuance_ai @mariusschober curious about your take here - we actually think being photorealistic is not the important part of what we're doing - we'd love to have a robot that you can banter with naturally!
wdyt of e.g. stylized characters?
https://t.co/h5CBjOQWg9

[Fangchang Ma @fangchangma]

We’re also releasing the Nuplex API in beta, with a few different personas.

It’s intentionally limited for now, and more personas and use cases are coming quickly.

https://t.co/HNMz2xS7O6

[Edward Zhang @Kyzyx]

@RyvenDriving @Scobleizer @nuance_ai chuckling/laughing together is something I've been begging the team to add for a month 🥲
It's hard to decide whether to hack it in with heuristics or properly learn it from data

[Edward Zhang @Kyzyx]

@DrOnwude @fangchangma @nuance_ai Expanding past the shoulders is already on our roadmap! Would love to hear more about where you're imagining seeing a full-body avatar.

You can already use Emma to teach you things - give our API a try! https://t.co/AMEr9egvD4

[Edward Zhang @Kyzyx]

@sahilkaling_ Thanks for the hype @sahilkaling_ !
btw you don't have to speculate if it's as good as it looks. You can try it now! https://t.co/aFzCUz92Fx

[Edward Zhang @Kyzyx]

@cheneypiano @DataChaz @fangchangma @nuance_ai Great question! Short answer - it's learned implicitly from an insane number of hours of real human audiovisual conversation. You should try it out - she's pretty good at not interrupting you while you're thinking!

[Edward Zhang @Kyzyx]

@pablogimenezz @fangchangma 1. start with vibes
2. break it down
3. make internal metric
4. take all metrics with a grain of salt

[Edward Zhang @Kyzyx]

@DanielSmidstrup @fangchangma @nuance_ai People literally got confused by our fundraise announcement last month, like "why do we need another video chat app"

Just gotta be extra clear 😂

[Edward Zhang @Kyzyx]

@bicro_ sorry she's still too nice to rage at you on smash

[Edward Zhang @Kyzyx]

@DanielSmidstrup @nuance_ai Try it, I wanna see!
Careful though, Emma herself might be a bit too nice for a hard sales call.
Might have to build that yourself on our API

[Edward Zhang @Kyzyx]

@jbawsa "taste" is exactly what I've been telling people. What we've focused on is so different from what others in the space are thinking about. That's what make the experience different.

[but we have the best model, too]

[Edward Zhang @Kyzyx]

@vas We've come so far, and yet there's so much more we're excited to work on!

[Edward Zhang @Kyzyx]

@albacnt tbf we're also proud of what she says
She doesn't just sound like she's just reading off of chatgpt

[Edward Zhang @Kyzyx]

@heyDhavall @nuance_ai Making the conversation feel natural isn't one layer, it's like at least four. The full-duplex, the microexpressions, the macro behavior, the personality...

[Edward Zhang @Kyzyx]

@DerwinPrieto Love that you pointed those use cases out. So many people have asked us "why the face" - it's exactly those settings like coaching or interview practice, where the value is not in the outcome, but the interaction itself

[Edward Zhang @Kyzyx]

@fangchangma Really showcases how you can banter back-and-forth with her.

So different from the "chatbot with a face pasted on top" feel of everything else out there today, you can thank @claudiavanea for that (she's the only one here with any banter game)

[Edward Zhang @Kyzyx]

@fangchangma and me, 2025: Let's do an AI research startup, we're good at ML
@claudiavanea and me, last week: yeah we gotta set aside the afternoon to debate psychology

[Fangchang Ma @fangchangma]

The first AI persona you’ll actually want to meet face-to-face.

Most AI personas stare blankly while you talk. Emma reacts while you’re still speaking.

After 18 months of research at @nuance_ai, this is Emma, built on Nuplex, our full-duplex audiovisual foundation model.