Shaya Deaton

Solving data problems.

[Shaya Deaton @shayadeaton]

We hosted a Claude hackathon at AGI House on Saturday to push the limits of what Opus 5.5 can do:

[Shaya Deaton @shayadeaton]

Leaders from OpenAI, Meta and Arm in one room, the biggest takeaway had nothing to do with models.

It's the infrastructure.

Arm's data center systems lead described her job as learning to "balance the physics with the economics." That means gigawatts of power, memory costs going through the roof, and chip cycles that take years. In her words, "supply has not caught up to the demand."

The companies that solve for both sides of that equation will shape what comes next.

[Shaya Deaton @shayadeaton]

💻 Come join us at the ​Claude Community Team this Saturday, September 26 for a full build day!

👉 What you get:

​ $100 USD in API credits per attendee to keep building after the event.
​Exclusive launch content: Anthropic's official video on what's new in the model, screened only inside the event.
​Global reach: your build and the recap go into the roll-up Anthropic sends to the entire Ambassador network.
​A direct line to product: feedback from the room reaches Anthropic's product and model teams.

In partnership with @AnthropicAI

Event Link: https://t.co/AuQ9HqYO2W

@kaius

[Shaya Deaton @shayadeaton]

Every researcher who spoke at an Audio X Robotics event agreed on one thing: we're still early in Voice AI x Robotics.

[Shaya Deaton @shayadeaton]

@Iorel_X can you do this on web? I am not seeing option.

[Shaya Deaton @shayadeaton]

Voice AI's most persistent bottleneck: knowing when someone stopped talking.

Turn detection is figuring out in real time whether a user is done speaking or just pausing to think, sneeze, or sip coffee mid-sentence. Get it wrong and the agent either cuts you off or sits in awkward silence.

The mechanics behind this:

> Voice Activity Detection separating speech from background noise
> Endpointing models deciding if a pause means "I'm done" or "I'm thinking"
> Prosodic signals like pitch and intonation hinting whether a sentence is complete
> Semantic context from the conversation, predicting whether more is coming based on what was already said

Where it gets serious is regulated industries. In healthcare, insurance, and finance, agents have to deliver specific statements in full. Disclosures, "this call is being recorded," language that cannot be interrupted. The current fix is temporarily disabling turn detection so the agent can finish what it's legally required to say.

What other workarounds are teams building around this?

[Shaya Deaton @shayadeaton]

Last night @theworldlabs broke down that we’re shifting from generating content to generating environments. Images and video are things you watch. Worlds are things you can interact with.

They framed world models in three categories:

>Renderers → creatives / media
Best for film, advertising, marketing, concept art, worldbuilding, visual ideation
Core question: “What would this look like?”

>Simulators → robotics / AV / RL
Best for robotics training, autonomous vehicles, embodied AI, reinforcement learning. Prioritizes physics, geometry, causality, and environment realism
Core question: “What would happen?”

>Planners → agents / decision-making systems
Best for robot control, navigation, embodied agents, task execution. Takes a goal + world state and outputs an action
Core question: “What should I do?”

@kaius captures real world data to support world model builders enrich the environments that can't be scraped or simulated.

[Shaya Deaton @shayadeaton]

Mati Staniszewski, CEO of @ElevenLabs , raised one of the most important unsolved problems in voice AI: person-specific transcription.

Imagine an operating room.
A surgeon gives a command mid-procedure. Multiple people are speaking. Machines are beeping. Someone is talking across the table.

The challenge isn’t simply transcribing speech. It’s identifying which speaker matters and reliably extracting that voice from everything happening around them.

Most speech systems still perform best when the target speaker is the loudest, closest, or clearest. Real-world environments rarely give you that luxury.

The signal you need is buried inside overlapping voices, reverberation, background noise, movement, and constantly changing acoustics.

And that exposes a bigger problem: you can’t solve real-world voice AI with clean, controlled audio alone.

Kaius is capturing this, the environmental and conversational data needed to train and evaluate models under the conditions where they actually have to perform.

[Shaya Deaton @shayadeaton]

I spent this weekend with Saturday Robotics in SF

TLDR; the data problem remains one of robotics biggest bottlenecks (alongside hardware)

> real data is genuinely hard to collect at scale

> more sensors ≠ better performance

> inconsistent signals make learning harder