@krahets Great work! Eager to check out your implementation and compare w/ the training process and codec we released recently! https://t.co/SvIz4lL6Td
Performance flex: 4DGS codec streaming on Quest 3 at 25-40mbps, captured with a single iPhone, plus real-time shadows for AI puppets 🕴️
Me & @ed_rethado are open sourcing Faltwerk - compression & streaming for dynamic gaussian splats! ~5-7mb per clip vs 1.4gb for ply flipbooks
More adventures in homemade facecap rigs
It’s too heavy and too sensitive to side-to-side shaking. I can lose the clip and drill the gooseneck straight into the forehead of the strap, but I think the camera needs to connect via two goosenecks, one on each side, for better stabilization w/ a lighter gage. Almost there…
Game out Wednesday! Recording final voiceover, mocap and facecap takes today
WebGPU + WebXR seems to be rolling out for all Quest devices this week! It's been behind a flag for weeks while I worked on new mixed reality game w/ GPU sim at its core. As usual my goal was to anchor the tech in story & character - can't wait to introduce these two next week!
@w3stmacc WebGPU does not defy the physics of device performance, no, but new features like compute shaders give you more options for computation-heavy processes like simulations
Personally I would argue you don't really even need three.js anymore!
@Katsen WebGPU bindings for WebXR have been behind an experimental flag in Quest browser since April-ish, but it's rolling out for everyone by default this week w/ the new OS version. It's already on my Quest now
@rsunghun@Skibididibity2@toyxyz3@ElevenLabs@grok Agree you should be careful! The developer of RGBMocap added a commercial-safe model pipeline here. I am not doing rapid dance movements or diving stunts for my game so it performs well for my uses so far
https://t.co/1Wj9Vdi1g3
I've added support for NVIDIA GEM-X to RGBMocap! GEM-X is a model that allows commercial use. However, its motion quality is slightly lower compared to FlowHMR and ExMR. #ai#mocap https://t.co/lIk6nnwc4o
Added face & eye tracking w/ Quest Pro to the single-camera body tracking in @toyxyz3's RGBMocap, plus @ElevenLabs voice changer API, all synced
Exports to my new game with timed captions - a unified suite for giving lifelike performances as 3D characters
Game reveal next week!
Works well enough for now. Considering ditching Quest Pro’s limited set of ~8 blendshapes for…
A prosumer VR strap jury rigged w/ a harmonica holder, holding an iPhone running ARKit or MediaPipe face tracking
Comparable Rokoko headset is like $500
Distribution of weight and construction has to be right or it gets too bouncy. But has to be head mounted or you lose accuracy with any turn from camera, and face/voice need to be recorded together w/ body for realism
Added face & eye tracking w/ Quest Pro to the single-camera body tracking in @toyxyz3's RGBMocap, plus @ElevenLabs voice changer API, all synced
Exports to my new game with timed captions - a unified suite for giving lifelike performances as 3D characters
Game reveal next week!
I'm disappointed, I assumed they were assembled by *other Microducks*, self-reproducing
Jk this is awesome. I'm order #7000-something & looking forward to playing around! Considering another more DIY build from parts and actuators if anyone has any recs
We were at the @seeedstudio Shenzhen factory today with @antoinepirrone and @matth_lapeyre and seeing the Microduck production line run for the first time was something special.
We even had our own assembly stations 🙂
Performance flex: 4DGS codec streaming on Quest 3 at 25-40mbps, captured with a single iPhone, plus real-time shadows for AI puppets 🕴️
Me & @ed_rethado are open sourcing Faltwerk - compression & streaming for dynamic gaussian splats! ~5-7mb per clip vs 1.4gb for ply flipbooks
@Sophonim Thanks for noting that, we will add some licensing info. It’s fully intended as open source and reusable, so go for it! Thanks for your interest 🙏
@ed_rethado Considering it's basically a 2022-era smartphone chip that has to render every scene twice, it's kind of a miracle imo! Impressed how unintimidated you were by what I thought was a hard problem 🙏
Now the work of making actual content begins...