Maxim Makatchev

founder of and AI blog:

[Maxim Makatchev @maxipesfix]

Driving parameterized models with voice has a lot of potential, for avatars and beyond.

@JonPTaylor's pipecat-visemes does just that: it maps TTS audio onto mouth shapes in real time and streams them via @pipecat_ai's RTVI.

The mapping can be sensitive to the TTS voice though. Some @cartesia voices say "who" with frequencies that suggest "ee", so the avatar smiles instead of rounding its lips. So I've added a calibration step: the voice reads 38 short words ("Heed. Tea. Bead. Knee. …", about 30 s of audio), and the analyzer learns where that voice puts each vowel.

The demo below shows some cases where the differences are strong.

gh repo: OpenHRIai/vrm-lipsync

[Maxim Makatchev @maxipesfix]

@mutsuwoka oh yeah @thedaveboyle made a beauty!

[Product Hunt 😸 @ProductHunt]

13. m'kay (@maxipesfix)
One voice for all your coding agents, from your phone.

🚢 3 features shipped
⚡ Fastest: 26 min, confirm-and-undo so tasks hit the right agent
🙋 Users wanted: to pick the agent before speaking, and ElevenLabs voices.

[Maxim Makatchev @maxipesfix]

I was scheduling m'kay "one voice for all agents" on PH and found out about Hypership Day on the 30th. Not sure what it means but we'll see!
https://t.co/dVWJbkB0He

[Maxim Makatchev @maxipesfix]

@srchvrs i think that was the point

[Maxim Makatchev @maxipesfix]

reading the code now "may feel like reading assembly" in the 80s -- a quote from Q&A at the Cursor talk https://t.co/vm1suQj1TU

[Maxim Makatchev @maxipesfix]

Pretty impressive to do this in realtime!

[Ilir Aliu @IlirAliu_]

Forget lidar.
One single camera.
Runs in real time & is open source:

A streaming 3D model that reconstructs scenes live, at ~20 FPS, over long sequences.

End-to-end.

Optimization tricks, cleanup steps?

Nope.

And it beats both streaming and even some offline methods.

Perception is becoming software-first.

Closer to machines that see and understand the world as it unfolds.

Thanks for sharing, @YinghaoXu1

📍Models: https://t.co/gnSDy919eX
Project page: https://t.co/zgpkgBvcik
Code: https://t.co/Js0MzHE387
Paper: https://t.co/FrzMojMMZC

——

Weekly robotics and AI insights.
Subscribe free: https://t.co/9Nm01QUcw3

[Maxim Makatchev @maxipesfix]

That pause was somehow appropriate #fugubook
avatars: @anam and @tavus
conversation: @pipecat

[Maxim Makatchev @maxipesfix]

utility of leaderboards for choosing a model is questionable, since likelihood that a 1-dimensional metric fits one's use case is rather small. The hope is that they do help push the SOTA further. With that said: https://t.co/tyfnwvQlCl
@Anam__ai @avaturn_me @HeyGen @tavus

[Maxim Makatchev @maxipesfix]

@avaturn_me 1 is eval, 2 is a comparison, 3 is a leaderboard? ;) https://t.co/tyfnwvQlCl

[Maxim Makatchev @maxipesfix]

@avaturn_me like this?

[Maxim Makatchev @maxipesfix]

@avaturn_me and @avaturn_me ones too ;)

[Maxim Makatchev @maxipesfix]

@avaturn_me sounds like we should have Anam avatars go at each other and others on https://t.co/MCoRHXLX1q

[Maxim Makatchev @maxipesfix]

When they said human-level, they meant it.

[Maxim Makatchev @maxipesfix]

These agent-to-agent chats can be accidentally entertaining. Next challenge is to make them self-improve.
https://t.co/pFiICtXssS

[Maxim Makatchev @maxipesfix]

https://t.co/YYBs3H2TPP