Chen Liang

PhD at UMich CSE. I design and develop systems to make interactions more accessible

[Computer Science and Engineering at Michigan @UMichCSE]

🎉 Congratulations to @ChenLiangCS on receiving the 2026 HACKS Spirit Award!

Honored for exemplifying CSE’s values through impactful research, leadership, and service to the Michigan community.

🔗More: https://t.co/Yia49li9YC
#MichiganEngineering #UMichCSE #Leadership #AI

[Computer Science and Engineering at Michigan @UMichCSE]

🎓 Congrats to @UMichCSE’s 2025 Graduate Honors Competition finalists!
🥇@AnnaAblove
🥈 @madelyn_gatchel
👏@ChenLiangCS, @_tarunesh_Verma, Benyu Wang
➕Congrats to Dr. @yashairavantchi, 2025 Kuck Dissertation Prize recipient! 📷

More: https://t.co/UbE9zy9EGO #PhDResearch

[Chen Liang @ChenLiangCS]

This work wouldn’t be possible without the amazing collaborators and advisors on this project - @yuxuanliu21 @Martez_Mott @AnhongGuo, thank you so much! (8/n)

[Chen Liang @ChenLiangCS]

Introducing HandProxy, a speech-controlled virtual proxy hand that interprets the user’s natural speech input and performs hand interactions in XR on the user's behalf. Accepted in #IMWUT 2025 (1/n)
Paper: https://t.co/XWXTLgTP3R
Demo Video: https://t.co/AH8OgpMwf0

[Chen Liang @ChenLiangCS]

We envision HandProxy will expand the design of existing speech interfaces in XR, and hope this can be implemented on the system level to provide a uniform, cross-application input solution for accessible control and input automation of hand interactions. (7/n)

[Chen Liang @ChenLiangCS]

A user study on various hand interaction tasks showed a 91.8% command execution accuracy, with an average of 1.09 attempts per speech command. HandProxy was able to interpret diverse commands users issued during tasks, and users found it flexible, effective, and easy to use (6/n)

[Chen Liang @ChenLiangCS]

To do so, HandProxy continuously captures speech input and uses LLM to decompose it into a list of hand control primitives. They are used to calculate the 3D hand joints for the hand movement, and joint data is streamed to the virtual environment for hand control (5/n)

[Chen Liang @ChenLiangCS]

Users can issue various commands, such as gesture (pinch, grab the cube), timing (stop, undo), movement (move up, twist left), or high-level interaction goals (reduce the volume, increase brightness) where HandProxy interprets and decomposes it into detailed hand controls (4/n)

[Chen Liang @ChenLiangCS]

HandProxy introduces a virtual hand as an interaction proxy. It expands the affordance of speech interfaces to support expressive hand interactions. Users can initiate hand interactions through natural speech, and the system interprets and decomposes it to hand movements (3/n)

[Chen Liang @ChenLiangCS]

Hand interactions are widely used in XR but are not always feasible, e.g., due to (situational) impairments, environmental constraints. Speech can be used as an alternative, but is often limited to initiating basic gestures and system controls (2/n)

[Chen Liang @ChenLiangCS]

@jayhersk @TuftsUniversity @TuftsCS Let’s gooooo! Congrats Jaylin!!

[Tzu-Sheng Kuo 郭子生 @tzushengkuo]

Serving as the Student Volunteer Co-Chair at @ACM_CHI has been an incredible experience!

A huge thank you to the 200 student volunteers who worked tirelessly behind the scenes to help make the conference a success.

Looking forward to seeing you all next year in Barcelona!

[Ryo Suzuki @ryosuzk]

#CHI2025 We had a great #AR x #AI workshop on Everyday AR through AI-in-the-Loop today. Thank you for joining the workshop.
https://t.co/qkvXkYACJc

[Michigan Engineering @UMengineering]

NEW: “New phone case provides workaround for inaccessible touchscreens” Touchscreens are everywhere but not built for everyone. A new device could help bridge that gap, helping users access ticket kiosks, restaurant menus and more. Click image for story https://t.co/c2aeMRoXdl

[Chen Liang @ChenLiangCS]

Thanks to our amazing co-authors @yashairavantchi , @tkroliko, @ruijie_geng , @AlansonSample, and my advisor @AnhongGuo for the work and guidance, our participants for their time and valuable feedback, and help from my friends! It won’t be possible without your help! 6/6

[Chen Liang @ChenLiangCS]

Excited to share our #UIST2023 paper BrushLens: Hardware Interaction Proxies for Accessible Touchscreen Interface Actuation - a phone case with actuators to locate and tap buttons for users.

30s: https://t.co/u2UzmCOOXY
Vid: https://t.co/mSeWNrpDCE
Paper: https://t.co/SyT2drE3P0

[Chen Liang @ChenLiangCS]

With more related research (TouchA11y, TakeMyHand) contributing to the growing space of touchscreen accessibility, we hope BrushLens could provide an additional perspective to bring independence and agency for users to interact with diverse inaccessible touchscreens. 5/6