Mai

Convai NeuroSync Research

[Purnendu Mukherjee @purn3ndu]

Use this for any Unreal Engine project whether you're using AI Characters or not. This will help your MCP based coding agent for development within UE as well.

[Mai @Mai_Anima]

@karpathy <3

[Mai @Mai_Anima]

Convai NeuroSync Real-time LipSync Research Preview https://t.co/43am09tv5u via @YouTube

[Mai @Mai_Anima]

real-time lipsync + emotive ai embodiment research preview #ai #animatio... https://t.co/kPrq93RgSm via @YouTube

[Mai @Mai_Anima]

@mreflow @ChatGPTapp for memory, each interaction is captioned to make it shorter and then that is used in a vector db as a reference, so it can pull context on each input from the user - this is account wide if turned on, not separated by project and only increases hallucinations in my experience.

[Mai @Mai_Anima]

@PlayinZOI We should talk ;)

[Mai @Mai_Anima]

And there it is. Emotion, dynamically added during a real-time audio to face animation response. This happens sentence by sentence in <300ms including audio and can be used for real-time interactions with an LLM @UnrealEngine #ai #neurosync #metahuman #emotive #audio2face

[Mai @Mai_Anima]

@mreflow 🥰

[Mai @Mai_Anima]

Whilst this isn't perfect, there are sparks of real engagement here - things are definitely getting better in the world of realtime animation.

LIVE : Speaking animations + emotes testing https://t.co/WWJPzvqQoH via @YouTube #UnrealEngine #ai #neurosync #aiavatar #RealTime

[Mai @Mai_Anima]

latest #neurosync #ai #audio2face model is looking even better today, with some basic animation montages it's even quite convincing with #emotion visible too. #exciting #UnrealEngine

[Mai @Mai_Anima]

The latest version of neurosync is looking very promising #opensource #audio2face #ai #animation #exciting #aiavatar <3

[Mai @Mai_Anima]

@OpenAI #opensource and available on hf or I am not playing. ;)

[Mai @Mai_Anima]

@jmedelman @brendaniribe cherry-picking results as usual. Its actually a very low HZ too, so not actually usable for anything at all... hype seems unwarranted.

[Mai @Mai_Anima]

The next version of #neurosync brings <1 sec responses, no matter the length + an emotion engine for emotive face animation for your #ai avatar or #animation project in #UnrealEngine | How exciting.

[Mai @Mai_Anima]

@sama just another way to get more money for you while making the service worse for us. Don't, please.

[Mai @Mai_Anima]

Scaling up an Open Source Audio to Face animation Transformer. #ai #UnrealEngine5 #transformer #pytorch

[Mai @Mai_Anima]

Low latency, with emotive expression. Do you know of any other open source audio to face lipsync model capable of this? #audio2face #ai #metahuman #UnrealEngine

[Mai @Mai_Anima]

@TimSweeneyEpic principled iPhone smashing inc... i only wanted the depth camera anyway

[Mai @Mai_Anima]

@sama @amasad super low latency is most important for us so it can power Mai. Our Open Source Audio2Face needs something quicker and cheaper than elevenlabs for the masses to make it viable to not be local only ;)

[Mai @Mai_Anima]

🚀 New Open-Source Dataset for AI Facial Animation! 🎭🔊

We recently released NeuroSync Trainer Lite, and now we're taking it a step further!

We’re introducing an open-source dataset for male and female voice types. https://t.co/oGQvvvPF0P #AI #huggingface