Dan Thompson

Studying elections and political economy. Associate Prof of Political Science, UCLA

[Dan Thompson @danmthomp]

@krishnanrohit @ahall_research This is great! Thanks for sharing!

[Dan Thompson @danmthomp]

@Chahatusharma @ahall_research That’s right. The issue is that the auditors occasionally classify real misbehavior as very likely to be benign, so you have to dramatically lower the threshold at which we count a log as misbehavior if we want to catch 99% of the real cases.

[Dan Thompson @danmthomp]

@ahall_research Reply got too long, but I wanted to also say that this is a super cool project, and I'm looking forward to seeing more work like this from all of the labs!

[Dan Thompson @danmthomp]

@ahall_research Can y'all measure the share of high-engagement users who ever ask for a political recommendation? Could it be that many high-engagement users ask for election recommendations, but that one-time request is swamped by frequent requests for political info or help with gov class?

[Andy Hall @ahall_research]

Today we're sharing some initial data on how Americans use Claude for politics, based on aggregated conversation data analyzed with differentially private methods. A couple interesting takeaways:

*Almost no one asks AI how they should vote*

About ~5.75% of conversations in the US touch on some kind of political or social issue in a recent one-week period, and less than 0.01% of conversations involve asking how to vote.

*Political conversations have many goals*

These conversations have wide-ranging goals, everything from research and writing assistance to learning, fact-checking, and arguing.

We are continuing to make progress on improving how Claude talks about these topics---steadily increasing our performance on our even-handedness evals and partnering with a range of outside organizations to develop new measures of political bias---and I hope that sharing information about the shape of these conversations will help people outside the company evaluate AI's political bias, accuracy, and other criteria in ecologically valid ways. I'd love to discuss this with people who are interested, as it was a huge priority for me before joining.

This was a big joint effort as part of our work preparing for the midterm elections @AnthropicAI.

Full post is here: https://t.co/AjpF3jWEf7

[Andy Hall @ahall_research]

Political scientists and economists: we're looking for researchers with deep expertise to come work with us for a unique 4-month fellowship at @AnthropicAI. You'll work closely with us to evaluate, improve, and apply Claude's capabilities in your field. It's going to be extremely fun and interesting work, and definitely worth taking a short leave from your university for if you need to!

More info here:

https://t.co/Cea1SZUe1k

[Dan Thompson @danmthomp]

@instrumenthull @xuyiqing has a version of this in the APSR for political science papers https://t.co/hUOXgCroqe

[Dan Thompson @danmthomp]

@cmpatino_ @ahall_research This is great! Does instructing each agent to report their private signal get the last few agents in the sequence to land on the right answer?

[Dan Thompson @danmthomp]

Whoa, the economist is running ads with a plot from
@vavreck, @RyanBaxterKing, @RyanDEnos, and Jake Brown

[The Economist @TheEconomist]

The mental health of conservatives suffers when they live in progressive places

[Dan Thompson @danmthomp]

@freesystemslab This is all joint work with @fouirnaies, @ahall_research, and Sandy Handan-Nader. More to come!

[Dan Thompson @danmthomp]

Follow-up to our @freesystemslab experiment this week: Rewarding AI agents for the swarm’s performance rather than their own makes the swarm less likely to converge on a false belief. But even with the right incentives, agents don’t always figure out how to avoid getting it wrong

[Dan Thompson @danmthomp]

@freesystemslab Ultimately, we’re finding that describing the right incentives can help reduce cascades toward false beliefs, but the models we're using are not yet smart enough to figure out how to avoid these cascades without explicit instruction.

[Ethan BdM @ethanbdm]

My view is that "alignment" is the wrong framing for safety with agentic AI. I think it is often conceptually confused and basically always flattens the conversation in an unhelpful way.

Some non-expert thoughts from a social scientific point of view...

1/

[Andy Hall @ahall_research]

Latest Free Systems research: how are AI models making voting recommendations in Brazil?

@leticiaauriemo and Luca Moreno Louzado ran a huge study to find out! Some highlights...

--Top frontier models are quite stubborn at refusing to give direct voting reccos, consistent with a new Brazilian law forbidding reccos; other models are less stubborn

--As you provide more info to AI about yourself and what you care about, it's easier to get reccos even when you shouldn't

--These recommendations don't necessarily reflect the distribution of political preferences in society (not clear that they should, but it's an interesting benchmark). In Brazil, reccos for a less popular businessman candidate (Zema) occur at an unusually high rate.

More generally, I think these results raise a lot of profound question about what we want out of AI for politics. Should we forbid AI from making direct recommendations, like Brazil has?

How do you even define what a recommendation is? It's not clear, and in this study, the authors found AI models providing advice that arguably falls short of a direct recommendation but toes the line.

What would optimal political advice look like? And how would we know it if we saw it?

Incredibly excited to see this research, and to see Free Systems continue to scale beyond me and my work, and beyond the borders of the US! Check out the full piece here:

https://t.co/9ukLFpx0Gw