Michael Hoffman

incubating an XR revolution - @MixedReality @ObjectTheory @Microsoft @HoloLens, AR, VR, MR

[Matt Shumer @mattshumer_]

https://t.co/ivXRKXJvQg

[Jim Fan @DrJimFan]

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do?

I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API!

Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a “hybrid-gradient architecture”: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based).

We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame!

Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operator’s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors.

As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: https://t.co/BHiNmqPoWE
Paper: https://t.co/bdh9TYQtHm
Code: https://t.co/lqKiaM2yYJ

Deep dive with me: 🧵

[eric provencher @pvncher]

Unity quietly released a new hands package that introduces a hands subsystem enabling native hand tracking via OpenXR in vanilla Unity. It also fully supports Meta’s aim extension that adds far rays and pinch.

The sample hand mesh is getting fixed soon and xri samples are coming

[Michael Hoffman @m_the_hoff]

https://t.co/sb66bsGV1j

[hollowaya @hollowaya_x]

some progress on SunflowerOS — making the VRChat mirror real!! This is through the lens on Magic Leap 2 w/ StereoKit.

[Michael Hoffman @m_the_hoff]

https://t.co/toIizKHXqx

[Nick Klingensmith - koujaku.bsky.social @koujaku]

I've been noodling with ReadyPlayerMe avatars in StereoKit, and sometimes coding is truly delightful :D I'm... guessing I nailed the head position part!

Welcome to how I normally see the world I guess?

[Mixed Reality DevRel Team @MxdRealityDev]

ICYMI: Start your on-demand #MRDevDays viewing journey with @its_graceh and @3DBrent introducing #MRTK3. https://t.co/ytRMkMmlhM

[Finn Sinclair 🥽 @FNS_Developer]

Excited to announce the first update to the MRTK3 public preview. We're here to make MRTK better, faster, and cleaner, and chew bubble gum. And we're all out of bubble gum. Come join us! #mrtk #mrtk3

https://t.co/R74FjZQHve

[Julia Schwarz @julenka]

Wait...switching from testing my Unity app on a HoloLens 2 vs. a Quest 2 is now as easy as just plugging in a different headset and hitting play? Awesome! https://t.co/cLdqkrcNAu I used to get lost trying to remember all the config changes...

#mrtk3
#openxr
#mrdevdays

[Steve Lukas @slukas]

Welcome to the world MRTK3, & welcome to Twitter MRTK PM Grace Hsu!

Action items:
1. Download MRTK3 (linked here👇)
2. Follow Grace @its_graceh
3. Check out more MR Dev Days 🧵

https://t.co/FGHFb4lFm9

[Nick Klingensmith - koujaku.bsky.social @koujaku]

Nailing down async texture load priority, diffuse first, then the others! This is ~200mb of png decoding.

[Alex Kipman @akipman]

Happy birthday #HoloLens. Today you are 5 years old [)-)

[Michael Hoffman @m_the_hoff]

Microsoft announces HoloLens 2 https://t.co/CKvLcH6WFy

[Michael Hoffman @m_the_hoff]

Forth Dimension Displays and VividQ Collaborate on Holographic 3D Mixed Reality Headset that offers true depth perception. The article provides a good overview of why this is relevant. https://t.co/BtJutRCUD3

[Michael Hoffman @m_the_hoff]

What a perfect application of MR, truly adding precision and efficiency to a skilled task. https://t.co/APg1gpCdHo

[Michael Hoffman @m_the_hoff]

Edge compute - This Qualcomm blog post is the best description I've seen yet of why and how edge cloud/compute + 5G will revolutionize the miniaturization of XR devices. At first, these edge compute devices will be deployed in bus…https://t.co/RdG39dOXOc https://t.co/H2rxI785TV

[Michael Hoffman @m_the_hoff]

#ar #vr #mr https://t.co/P7UZJpYULE

[Neal Stephenson @nealstephenson]

My new novel, FALL, OR DODGE IN HELL, goes on sale in June 2019 in the USA. Info on tour & signed copies to follow. In the meantime, here’s the link to pre-order and learn more. https://t.co/GxDkujL68h

[Michael Hoffman @m_the_hoff]

Army orders 100,000 Microsoft HoloLens MR devices!

#microsoft #army #mixedreality #hololens #mr #xr #objecttheory https://t.co/QrZXQjCzwZ