🗣️ Excited to share our new #ACL2025 Findings paper: “Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks” with @jad_kabbara and @dkroy. Arxiv: https://t.co/FeWQLQxt5K
Read about our findings ⤵️
If you're at #CSCW2025, check out this wonderful panel on LLMs in conversation research. Having FOMO already-- but go listen to my wonderful colleagues!
One week away from our CSCW panel on applying LLMs in conversation-based research! Excited to engage in another methodological discussion with my amazing co-organizers and panelists @marianneaubin@Schropes Alejandro @stevendow@eugeniarho@shivanikapania
Excited to be at #COLM2025! Excited to be kicking off the NLP4Democracy workshop this morning. We are in 520E (behind A/B/C) - check out our amazing program! https://t.co/TMSBK7vSWF
🚨 New paper alert 🚨 Using LLMs as data annotators, you can produce any scientific result you want. We call this **LLM Hacking**.
Paper: https://t.co/24Fyb4Ik3v
We are hiring!
The @StanfordGSB is hiring in macro & micro OB! If you do sociology, psychology, OB, strategy, management, computational social science, please apply. On the macro side, mid-career folks are especially invited to apply. See here👇
https://t.co/exYVH5dgE6
My team at @AISecurityInst is hiring! This is an awesome opportunity to get involved with cutting-edge scientific research inside government on frontier AI models. I genuinely love my job and the team 🤗
Link: https://t.co/poiWqKlmgt
More Info: ⬇️
@jad_kabbara@dkroy Implications vary by task and domain. Researchers should clearly define their annotation constructs before reviewing LLM annotation. We are subject to anchoring bias that can affect our evaluations, or even our research findings!
Read more: https://t.co/FeWQLQxt5K
🗣️ Excited to share our new #ACL2025 Findings paper: “Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks” with @jad_kabbara and @dkroy. Arxiv: https://t.co/FeWQLQxt5K
Read about our findings ⤵️
@jad_kabbara@dkroy Using LLM-influenced labels, even when a crowd of humans reviews them and is aggregated into a set of crowd labels, can lead to 1) different findings when used in data analysis and 2) different results when used as a basis of evaluating LLM performance on the task.
Do you have strong programming skills but need research experience before heading off to grad school for computational social science / business / econ? Apply now to predoc with me and @DavMicRot and @jakehofman
DMs open
https://t.co/gj0ZBOFt6i
🚨🚨 Excited to share a new paper led by @Li_Haiwen_ with the @CommunityNotes team!
LLMs will reshape the information ecosystem. Community Notes offers a promising model for keeping human judgment central but it's an open question how to best integrate LLMs.
Thread👇
I loved sharing our paper talk this morning (Wednesday, 6/25) in Meta Research and Critiques, 9:12 am at New Stage A. Read the paper here! https://t.co/1DHXeOKcTN
1) Thrilled to be at #Facct2025 for the first time this week, representing a meta-research paper on positionality statements at FAccT from 2018-2024, in collaboration with Solon Barocas (@s010n) and Akshansh Pareek.
*What should FAccT do?* The conference can clarify its policies next year, engage scholars from different disciplines when considering policy, protect optionality, and engage authors in reflexive practice upstream of paper-writing, potentially through CRAFT.