Tianrui Guan

🚗 @waymo | Ex-@Amazon Lab126 | Ex-@BaiduResearch | 🎓 PhD @umdcs @gammaumd | CV/AD/Robotic/VLM 🎾🏂🔫🎸🍻

[Tianrui Guan @terryguan97]

🚗✨ I’m happy to share that I’m starting a new position as Research Engineer at @Waymo today!

🎓✨ This comes shortly after defending my PhD at @umdcs, where I had the privilege of working with and learning from amazing collaborators at @gammaumd. A heartfelt thank you to my advisor, Prof. Dinesh Manocha @dmanocha, for his constant support and guidance throughout this journey.

🥳 Excited to join the incredible team with John Kang and Khaled S. Refaat, and contribute to building the future of autonomous driving. Looking forward to the road ahead at Waymo! 🙌

#Waymo #AutonomousDriving #AI #Robotics #ComputerVision #UMD #UMDCS

[Tianrui Guan @terryguan97]

Touchdown in Miami! ✈️🏖️🚤 Ready to dive into #EMNLP2024 and catch up with everyone! See you soon, @emnlpmeeting! #Miami #NLP

[Genmo @genmoai]

Introducing Mochi 1 preview. A new SOTA in open-source video generation. Apache 2.0.

magnet:?xt=urn:btih:441da1af7a16bcaa4f556964f8028d7113d21cbb&dn=weights&tr=udp://tracker.opentrackr.org:1337/announce

[Tianrui Guan @terryguan97]

Come and join our talk on Friday, Oct 4 at 1 pm EST! Join this link🔗: https://t.co/sFzbcUiBFu

[TwelveLabs (twelvelabs.io) @twelve_labs]

✅ @terryguan97 and @kongkong from @umdcs will discuss three separate works: (1) HallusionBench -- manual hallucination benchmark creation, (2) AUTOHALLUSION -- automatic hallucination benchmark curation, and (3) Eagle -- hallucination alleviation and mitigation.

https://t.co/Ol3E8lpnh3

[Tianrui Guan @terryguan97]

@gammaumd @niallw_cs Congrats @niallw_cs! 🎉🎊🍾🍾🍾

[Tianrui Guan @terryguan97]

@AntonJeran Congrats! 🎉🎊

[Tianrui Guan @terryguan97]

Follow our work for more updates. #AutoHallusion Code, demos, more examples, and improvements are on the way! 🛠️✨ Find out more details in our paper and on the website.

🌐 Webpage & Examples: https://t.co/G0sG5sADXm
🤗 HuggingFace: https://t.co/xEg9PzPJr5
📄 Arxiv: https://t.co/hXBROPmSQh

Big shoutout to co-leading @wu_xiyang and other wonderful collaborators! @cookielee77 @ShuaiyiH @xiaoyu_liu_1231 @xijunwang_cs @RuiqiXian @abhi2610 @furongh @boydgraber @zhoutianyi @dmanocha from @umdcs @umiacs @ml_umd @gammaumd

@UMDResearch @UMDscience
#VLMs #hallucination #benchmark #gpt4v #Gemini #Claude #LLaVa #CognitiveScience #brain

[Tianrui Guan @terryguan97]

Vision-Language Models (VLMs) suffer from hallucinations👀🙈: their reasoning may ignore some objects or create non-existing ones. This is usually caused by their language model bias dominating the facts in the input visual signals, just like the human brain relying on memory🧠 and past experiences🌎 (called "schema" in cognitive science).

But building a benchmark to evaluate and diagnose VLM hallucinations need a lot of human efforts #HallusionBench.

Can we automatically generate (image, prompt) pairs that can induce hallucinations of VLMs? 🤔

The answer is Yes.

We introduce #AutoHallusion🚀, an automatic hallucination pipeline that scales up the benchmark generation process. It probes LLMs to determine image content that contradicts with the LLM prior, and then generate the image by manipulating objects in the image using image-editing models. (1/n)👇

[Tianrui Guan @terryguan97]

In addition to creating benchmarks📏, our proposed method allows us to easily scale up the number of synthetic data that induce hallucinations. This can be used for tailored hallucination mitigation in different Vision-Language Models. (7/n)👇

[Tianrui Guan @terryguan97]

@AndreaRosasco @CVPR Arch building 4th floor, Disney Research.

[Tianrui Guan @terryguan97]

Just spotted an incredible robot 🤖 from Disney Research at @CVPR! It's amazing to see robot in animation come to life in the real world. 🌟 #Robotics #CVPR2024

[Tianrui Guan @terryguan97]

@Haoyu_Xiong_ @CVPR It’s in arch building 4th floor, near Disney Research exhibit.

[Fuxiao Liu @Kongkong]

I will present HallusionBench from Thu 20 Jun 1:30 p.m. EDT — 3 p.m. EDT at Arch 4A-E Poster #457. @cvpr #CVPR #llm #gpt4.

Come and say hi !😀

[Tianrui Guan @terryguan97]

🔥🔥🔥Excited to present #HallusionBench at the @MMFMWorkshop and @CVPR! 🎉 Catch me 👀 at poster #97 on Tuesday, June 18, from 9:40-11:00 AM and at poster session #457 on Thursday, June 20, from 10:30 AM to noon.

Come by and let's chat! 🥳

[Tianrui Guan @terryguan97]

🔥🔥🔥Thrilled to announce that our #HallusionBench was accepted by #CVPR2024! See you in Seattle!
@CVPR @gammaumd @UMDResearch @UMDscience
#cvpr #VLMs #hallucination #gpt4v #Gemini #LLaVa

[GAMMA UMD @gammaumd]

Tomorrow, @wu_xiyang will present "On the Safety Concerns of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities" (https://t.co/FP7utxqkOe) at the VLADR workshop @RuiqiXian @terryguan97 @jingliangcgm @SOURADIPCHAKR18 @kongkong @dmanocha @amritsinghbedi3