Maria Aroca

- Data Scientist

[Maria Aroca @MariaArocaVidal]

@mike_pavlukhin what's going on? :)

[Maria Aroca @MariaArocaVidal]

@gruffwashere boy, do I wish this was an entire VR experience

[Maria Aroca @MariaArocaVidal]

@SNDN_NM It was so hard to get him right! I had to speed up his speech and still it's more understandable than the real one

[Maria Aroca @MariaArocaVidal]

That boy ain't right!
Sound on for Boomhauer

[Jameson Hodge @jamesonhodge]

I only make serious software

[Maria Aroca @MariaArocaVidal]

@jjjoemclean wow!

[Maria Aroca @MariaArocaVidal]

@FlexUtility

[Maria Aroca @MariaArocaVidal]

@golan @jamesonhodge @p5xjs @openprocessing im seeing some medical potential here

[Maria Aroca @MariaArocaVidal]

@bradmillscan i just love this

[Maria Aroca @MariaArocaVidal]

@JasonRink @bennash 😂

[Maria Aroca @MariaArocaVidal]

With speaking it is challenging because if you only give it words it doesn't know the timing of each sound, the pauses, emphasis or pitch changes. And hank has a very specific way he says things.
For hank, i first recorded myself saying those lines sort of how he'd say it. Then you can use a library called pocketsphinx to line up the script with the recording, and numPy and sciPy to measure pitch, loudness, and vowel sounds. And those metrics can guided chimera's vocal folds and mouth as it speaks up the lines because now it knows exactly where each sound or pause needs to happen.

You can also mess with chimera's settings to get vowels to be a certain way or program a controller that has chimera say the vowel, move its jaw or tongue one control at a time, and then measure how the sound changes, and the code tries to bring it closest to your recording as possible.

For me that took it to like 80% hank, but the tone was hard to get, so you can then have an equalizer boost or cut frequencies until it sounds close, since his tone of voice is so specific.

Then you can modify the heads code to build scenes around them

[Maria Aroca @MariaArocaVidal]

@AnoyingOnline the heads are already animated with Three.js in chimera's browser interface, so you can modify that code and build scenes around them

[Maria Aroca @MariaArocaVidal]

@FLIPCHAR many many hours of this to get all the vowels semi right and then for the tone I had AI write an equalizer that compared chimera's voice with a few short snippets of the real hank and it compared different moments and boosted or cut frequencies across 47 bands to match his tone

[Maria Aroca @MariaArocaVidal]

This is absolutely the best time to be alive, ever! Hands down! @NickGideo created pure beauty

[Nick @NickGideo]

"Ego dormio"
Heinrich Schütz, 1625

thanks @MariaArocaVidal for the suggestion!

[Nick @NickGideo]

"Ego dormio"
Heinrich Schütz, 1625

thanks @MariaArocaVidal for the suggestion!

[Maria Aroca @MariaArocaVidal]

@NickGideo i must be dead cuz this is heaven!!!!! you are awesome!

[Maria Aroca @MariaArocaVidal]

@NickGideo yess Cruda Amarilli, amazing choice!!!!

[Maria Aroca @MariaArocaVidal]

@matthaios_x oh my gosh! I had just posted on your other video before seeing this https://t.co/u4mS3xjG7G

[Maria Aroca @MariaArocaVidal]

@matthaios_x Love it. Wouldn't it be cool to do Epitaph of Seikilos?

[Maria Aroca @MariaArocaVidal]

@kevin_t_ngo the CRT effect looks so good, was it postproduction or blender? or did fable do it itself?!

[Maria Aroca @MariaArocaVidal]

@WrongNebula We are all very token poor these days, waiting for the resets to reset...

[Maria Aroca @MariaArocaVidal]

@matthaios_x Love it. Wouldn't it be cool to do Epitaph of Seikilos?