Different AI models rely on different mixes of news sources to answer the same political questions -- super interesting new research from my collaborators at @freesystemslab!
As we build towards political superintelligence, there are going to be a bunch of important questions about how to incorporate news content. Before we figure all that out, we need basic facts about how things are working on the ground today. This research is super helpful for that, and I'm thrilled that @danmthomp and Sho Miyazaki are doing such great work on this topic.
📰 New research: Inside AI’s election news diet
What news does AI read to get up to speed on politics? And what kind of news does it reward?
Following interesting recent commentary on our Japan election study from @mattyglesias, @JerusalemDemsas and @TheStalwart, we ran a bigger test on the US midterms across 35 Senate races, 18 days, and 15,750 answers.
What we found:
- Each model has its own favorite news outlet for political news: AP for GPT, Yahoo for Claude, PBS for Gemini, and NYT for Grok.
- GPT tends to behave like a civic guide (reading more election office and campaign sites), whereas Claude behaves more like a news summarizer (reading national news, Wikipedia, polls).
- Grok draws on X and Gemini on YouTube for political information, far more than the other models we studied. Gemini cites YouTube in 55% of its answers and X shows up in 59% of Grok’s.
- A huge share of AI's local news is provided by large nonprofit networks that very few people will have heard of. States Newsroom, a large network of over 40 local newsrooms, appears in 60% of GPT’s answers.
- For many models, prediction markets are now a go-to source for polling information. Kalshi is cited more by Claude than any poll aggregator.
Full study here: https://t.co/AhVqULQvbI
How will high gas prices affect the midterms?
It's an important question but hard to answer empirically. I've written down a plan to test part of it, and I'll publish my first results on Thursday, when most of the vote is in. 1/🧵
https://t.co/QeaRIZsTrw
@keendawg@EventualNews@TradeandMoney@iabvek Did you have any conversations with people you spoke to about why you were there in the first place? Any interesting responses to doing this specifically for trading?
We built an AI ads agency and let it run autonomously on Meta for weeks, with the stated goal of netting new subscribers for the @freesystemslab newsletter. Our real goal: learn how AI is persuading people online, and what it might mean for the future of politics and our information environment.
The agents did really well! They learned from their mistakes, and by the end of four weeks, they'd all brought down cost-per-subscriber by more than 50% from the first week. They brought us more than 100 new subscribers, made 70+ highly amusing online ads, occasionally ran afoul of Meta's ads policies, and taught us a lot along the way.
Political campaigns are already using AI a lot in their advertising; it's only a matter of time before they embrace automated agentic systems. Check out our full piece for a ton of details on the mistakes we saw, how the agents learned, and what we think will happen next.
This was one of the most fun pieces we've done, and I think it really makes it clear how AI enables totally new kinds of research. Super excited to get it out there, and huge congrats to @eigenstuffs who led this project.
https://t.co/GCIpvrgZ2s