New w/ @AISecurityInst & @UniofOxford:
Frontier AI can now out-persuade expert humans in conversation - incl. world-champ debaters and professional canvassers.
This held even when humans chose their topics, prepared in advance, and competed for £1,000 prizes 🧵
Today we’re launching the AI Trials Observatory: a living systematic review of randomized trials on how AI affects people.
So far we’ve surfaced 93 trials on two outcomes: mental health and belief accuracy. More trials and outcomes coming soon!
https://t.co/bzvT0FyKSD
🧵👇
Super excited about this new observatory, masterminded by @Ben_Tappin!
It will be an incredible resource for researchers, policymakers and members of the public who want an up-to-date picture of how AI is impacting people.
Highly recommend checking out the first release!
Today we’re launching the AI Trials Observatory: a living systematic review of randomized trials on how AI affects people.
So far we’ve surfaced 93 trials on two outcomes: mental health and belief accuracy. More trials and outcomes coming soon!
https://t.co/bzvT0FyKSD
🧵👇
One of the most-viewed PNAS articles in the last week is “AI can persuade people to take political actions.” Explore the article here: https://t.co/d9qeSZARpq
For more trending articles, visit https://t.co/aQQ35461Il.
New in @PNASNews:
In two large experiments (~18k obs from ~14.8k people), we find that brief conversations with AI can get people to take real political action - incl. signing real petitions and giving money to charity.
Surprisingly, the persuasion strategy that worked best for changing people's attitudes - packing convos with info - was the least effective at changing behavior!
🧵
Some caveats:
• Participants were paid to talk to our models
• The behaviors we measured were fairly low-cost
• Donations came from a study bonus, not people's own pockets
• In the real world, attention may limit AI's influence more than how persuasive any one conversation is
True re: attention! And glad Anthropic is starting to share this data. But I struggle to interpret these numbers:
-IIUC, they're reporting share of conversations, not users. If a small % of heavy users have lots of other conversations (e.g. about coding), which seems plausible to me, then 0.51% of convos could mask a much higher share of users (the election-relevant quantity) asking about the election
-6 weeks before a midterm is still before most voters tune in
Hope a fuller release has per-user data across the full cycle!
The attention constraint on AI persuasion is fundamental, as we argued in this chapter https://t.co/p9OmT4WzZm
Just 0.51% of Claude conversations are about elections or voting; at most 0.01% about who to vote for
🚨Published in @Nature: We show how AI can simulate human behavior to predict experimental results as accurately as experts
Our work has many implications for AI safety. For example, simulators can be used to aid harmful manipulation, and current safeguards do not prevent this🧵
🚨🚨New paper in @Nature with @AshuAshok, @lukebeehewitt & @ghezae_isaias 🚨🚨
Can LLMs predict results of social science experiments?
Across 70 preregistered studies, we find strong correspondence (r=.85) between LLM-predicted and observed effects.
https://t.co/hIrsjhrxRz 🧵