I’m excited to share that I’ve recently completed my PhD and started a new position @Google! Thrilled and grateful to be part of a new team working with @twi_mar, Andrea Colaço, and other amazing researchers to build the future of immersive mixed reality.
We had an amazing four-day NII Shonan Meeting on "AR and AI: Everyday AR through AI-in-the-Loop", co-organized with @da_lind, @margonmora, Misha Sra. https://t.co/beEQBElaZW
Thanks for joining! Let's shape the future of AR and AI.
Over the last 2 years we have seen an acceleration on AI, but if AI is going enable humans in their day to day tasks, it most probably will be via XR. The issue is if a selection has real-world consequences, we will need great precision to interact. We envision Reality Proxy.
At @IEEEVR we present “EmBARDiment: an Embodied AI Agent for Productivity in XR" by @riccardobovoHCI et al. showing how powerful AI agents can be when they are aware of the content around us, driven by contextual inputs like gaze.
Web: https://t.co/cCmSfg64wu
Code: soon
2 papers accepted at #UIST2024!🎉
The 1st paper, XR-Oʙᴊᴇᴄᴛs w/@GoogleARVR, introduces 🪄Augmented Object Intelligence, an interaction paradigm blurring the lines between digital & physical by equipping real-world objects with the ability to interact as if they were digital:
XR-Objects is our open-source prototype that lets you ground AI interactions to real-world objects, enabling them to act as portals for relevant digital information.
Read more here: https://t.co/HLB2RDmadc
Introducing XR-Objects, a new open-source augmented reality prototype that transforms physical objects into interactive digital portals using real-time object segmentation and multimodal large language models. Learn more at: https://t.co/sD9FL5v5VP #AR#XR#SpatialComputing
👨💻In preparation, we’ve updated the open-source prototype to use Gemini 2.0 Flash + Search!
⭕️ We've also made it easier to manually register objects using a circle or long press gesture
Try it out for yourself: https://t.co/v4kyjQZ0Sa
It’s been an incredible two years at @GoogleARVR helping make #AndroidXR a reality! So proud to work with such an amazing team 💫
➡️ https://t.co/bAl1HQcVHA
#AndroidXR is officially here! 🎉 Thrilled to see the developer excitement and passion to build immersive experiences. Huge thanks to the @GoogleARVR team and our incredible partners @SamsungMobile and @Qualcomm and more. ➡️ https://t.co/0ljK0syK2F
Thrilled by the interest in Augmented Object Intelligence & XR-Objects presented by Eric @_ejgonz_! Thanks to everyone who attended & engaged with our #XR+#AI research 🙏🏻
📸 @ACMUIST#UIST24@FaruqiFaraz
Amazing work led by @mdogadogan during his time @Google on the future of world-anchored AI interfaces in XR!
Excited to present this work at @ACMUIST later this month 🎉
Until then, feel free to try out the open-source prototype at https://t.co/v4kyjQZ0Sa
Introducing XR-Objects, a new open-source augmented reality prototype that transforms physical objects into interactive digital portals using real-time object segmentation and multimodal large language models. Learn more at: https://t.co/sD9FL5v5VP #AR#XR#SpatialComputing
🔈 Our team is hiring! 🔈
If you are interested in building the next generation of XR+AI interfaces and contributing to cutting-edge HCI research at Google, consider applying to join the BIRD Lab!
https://t.co/gwZcGgbWDB
@twi_mar@realkaranahuja@adhocdown
This work was built using our open-source, cross-device toolkit XDTK (https://t.co/mc3vKQOJlF) and in collaboration with Ishan Chatterjee, @twi_mar, Andrea Colaço, and @realkaranahuja
#CHI2024 Come by the Late-Breaking Work sessions today at @acm_chi to learn about our work on multi-device interaction in XR! @GoogleARVR
paper: https://t.co/XC2AClZ4mp
video: https://t.co/w1quYBFYlA
For example, a user can seamlessly switch from browsing furniture on their phone to placing a virtual copy within their space.
Other examples we explore include automated passthrough transitions and notification routing.