Chacha Chen

MTS @MicrosoftAI / PhD @ChicagoHAI @UChicagoCS, human-centered ML.

[Chacha Chen @chachaachen]

✨ New paper at #FAccT2023 next week ✨
Machine Explanations and Human Understanding
https://t.co/ZcdvsKLyr3

Explanations are hypothesized to improve human understanding of machine learning models. However, empirical studies have found mixed and even negative results.

[Mikel Artetxe @artetxem]

Brilliant idea! Next up: Apple randomly reboots your Mac if you're building competing tech, Gmail silently edits your email if you mention rival platforms, and Tesla Autopilot swerves if it detects you're working on self-driving cars.

All in the name of safety, of course. Because malicious actors controlling the world’s operating systems, inboxes and cars would be extremely dangerous!

[Xiaoyan Bai @Elenal3ai]

🚨Why can’t Transformers learn multiplication?🧮
Even with billions of params, models struggle with multi-digit multiplication.
In our new work, we reverse-engineer two models: a standard fine-tuned (SFT), and an implicit chain-of-thought (ICoT) model to see why. Read on!
1/n 🧵

[Dang Nguyen @divingwithorcas]

1/n You may know that large language models (LLMs) can be biased in their decision-making, but ever wondered how those biases are encoded internally and whether we can surgically remove them?

[Xiaoyan Bai @Elenal3ai]

🚨 New paper alert 🚨

Ever asked an LLM-as-Marilyn Monroe who the US president was in 2000? 🤔 Should the LLM answer at all? We call these clashes Concept Incongruence. Read on! ⬇️

1/n 🧵

[Chacha Chen @chachaachen]

I’m going to present at the poster session on Friday morning. Come find me! 😃😃

[Chacha Chen @chachaachen]

Just got accepted to NAACL!

[Mourad Heddaya @mouradheddaya]

CaseSumm is publicly available on HuggingFace! We hope this dataset enables:
- Better evaluation of long-context summarization
- Research on legal language understanding
- Development of more accurate & reliable legal AI tools

Dataset: https://t.co/VddZO3WJTw

I'll be presenting this work at 2pm and will be around until Sunday. Please reach out if you're interested in this line of work - would love to connect in person or virtually!

[Haokun Liu @HaokunLiu5280]

🚀🚀🚀Excited to share our latest work: HypoBench, a systematic benchmark for evaluating LLM-based hypothesis generation methods!

[Chacha Chen @chachaachen]

Just got accepted to NAACL!

[Chacha Chen @chachaachen]

1/ ✨ Can Multimodal LLMs generate radiology reports from chest X-ray images? In our recent evaluation paper, we show: not yet.

[Chenhao Tan @ChenhaoTan]

Even ChatGPT knows. It is very hard to think of a positive spin for Rosalind here. A shocking display of ignorance and arrogance.

[Chacha Chen @chachaachen]

and I especially like the human evaluation we did. The ground truth reports are 100% clinically usable, although they slightly fall behind the fine-tuned Llama in terms of clarity and readability.

[Chacha Chen @chachaachen]

1/ ✨ Can Multimodal LLMs generate radiology reports from chest X-ray images? In our recent evaluation paper, we show: not yet.

[Chacha Chen @chachaachen]

the last row shows GPT-4v's gap with SOTA models

[Chacha Chen @chachaachen]

Joint work with @ChicagoHAI and @bmervakMD

[Chacha Chen @chachaachen]

We’re excited to present this work at @SymposiumML4H next month in Vancouver. Happy to chat and feel free to reach out 😊.

[Chacha Chen @chachaachen]

5/ We also conducted a human reader study to compare the quality and readability of AI-generated and human written reports.

Paper: https://t.co/6TiQGvEPgI
Code:https://t.co/cKAwhqIL5c
Altogether, our findings cast doubt on the viability of using GPT-4V in a radiology workflow.

[Chacha Chen @chachaachen]

4/ Our Evaluation Overview:
We started with direct report generation using different prompting strategies, but found the model failed terribly. To understand the low performance, we decomposed the task into two steps: image reasoning and report synthesis.

[Chacha Chen @chachaachen]

I will be in Rio for #FAccT2024 next week. Hit me up if you will also be there in person and would like to chat!🏖️🍻😎

[Chacha Chen @chachaachen]

@yuwen_lu_ maybe not even fun 🥹