For more information, check our demo paper or contact us! pdf : https://t.co/VdHyQugb9o
*Additionally, special thanks to Jihoon Shin and Jiho Kang, who helped us capture the motion sets!
We presented 'Situately', an embodied conversational agent with LLM-powered nonverbal behavior and object interaction, as a research demo at IEEE VR 2026 (Daegu, South Korea)! Thanks to all the participants who actively engaged and enjoyed conversations with Dr. Virtual!
In this project, Kwang Bin Lee implemented object interaction, Jungsoo Lee designed the environment and scenario, and Sihyun Jeong fine-tuned the language model to align nonverbal behaviors with human judgments, under the supervision of Prof. Sung-Hee Lee and Prof. Jeongmi Lee.
Attendees noted low latency and high responsiveness, but also reported limitations in turn-taking and motion quality — all valuable feedback. We totally agree, and we are working toward more fluent social interaction with the agent.
In this work, we leveraged LLM for nonverbal behavior and action planning. To support this, we built a framework consisting of verbal conversation, conversation memory, nonverbal behavior control, and object interaction control — all concurrently coordinated.
If you are more interested in this work, check out the presentation schedule!
📄 Paper: https://t.co/sYGqWtvETN
🗓 Presentation: April 30 (Wed), 2:10–2:22 PM
📍 Location: G403
We explored how turn-taking, gaze, and nodding behaviors reflect perceived social connection during dyadic VR conversations. Our findings provide a foundation for quantitatively understanding social interactions beyond subjective impressions.
Introducing Gaze-LLE, a new model for gaze target estimation built on top of a frozen visual foundation model!
Gaze-LLE achieves SOTA results on multiple benchmarks while learning minimal parameters, and shows strong generalization
paper: https://t.co/Is2NgrrurO
I'm just blown away by the amazing performance, features, and easy integration of handsfree.js by @Metamoar (built on TensorFlow & MediaPipe)—with multiple face, hand, & body tracking at 20-60fps in @p5xjs. Here's my example project for tomorrow's class: https://t.co/1EGlyoT6k9
Yay! A paper got accepted to ACM MM'21 (with Injung Lee and Hyunchul Kim). The title is "Automated Playtesting with a Cognitive Model of Sensorimotor Coordination".
새 유전자는 어떻게 생길까. 그전까지는 기존 유전자가 복제 과정에서 오류 일으켜 뒤섞이거나 파손되며 생긴다고 봤다. 사실은 새로운 유전자 생성도 허다하다. 환경 적응 위해 유전체 중 쓸모 없었던 부분에서 새로 만들어내기도 한다. 유기체의 진화는 생각 이상으로 창의적https://t.co/7zGTFsaSsJ