New paper out in @JCRNEWS
How do consumers react to political ads that meddle in the primaries of the opposing party?
In the most recent elections, Democrats did something strange.
More evidence of data fraud from Francesca Gino. This is at least five cases of fraud she committed (and she is being sued for another in court as we speak).
I would not personally trust anything she published unless another author handled the data and can independently verify the findings (same goes for Dan Ariely's research at this point).
Thanks to @AdamMGrant and David Hofmann for helping expose this one and issuing an expression of concern for their paper. We need more coauthors coming forward and working to correct the scientific record.
This is a fantastic & much-needed resource: a living meta-analysis of experiments measuring the human impacts of AI.
So far it covers 93 RCTs (N=50k), spanning AI's effects on 19 dependent variables from anxiety to misinfo belief.
Subscribe for updates: https://t.co/WeYE68yyac
I think people are drawing the wrong conclusions about gerrymandering. Just because a 100-year flood overflows flood walls doesn't mean that the flood walls don't contain the water really well most of the time.
We need a holistic reform of researcher assessment and incentives at academic institutions
There is a growing disconnect between what institutions say they value and what assessment systems actually reward. Many colleges and universities emphasize broader societal impact as an institutional priority, yet assessment systems continue to rely heavily on publications, citations, and grant expenditures.
“instead of valuing what we measure, let’s measure what we value.”
Reform efforts are already underway at several institutions to revise promotion criteria, increase reliance on narrative CVs and broader impact statements, and better recognize mentorship, open science, and interdisciplinary collaboration.
We need this at a broader systemic level. Creating a better academic and scientific culture should be part of our evaluation.
https://t.co/MZRJdkXtbL
How long does a finding about LLMs stay true?
We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects.
LLM findings has short shelf life https://t.co/MUWm6INyX8
Great job by @oq_35!
🚨New in PNAS: Our research shows how a ~7min AI chat can persuade people to take political actions👇
However, as noted by @Ben_Tappin, real-world impact is often limited by exposure [1]. To address this, we also present our results as an engagementmaxxed AI #slopcore music video
Today we're sharing some initial data on how Americans use Claude for politics, based on aggregated conversation data analyzed with differentially private methods. A couple interesting takeaways:
*Almost no one asks AI how they should vote*
About ~5.75% of conversations in the US touch on some kind of political or social issue in a recent one-week period, and less than 0.01% of conversations involve asking how to vote.
*Political conversations have many goals*
These conversations have wide-ranging goals, everything from research and writing assistance to learning, fact-checking, and arguing.
We are continuing to make progress on improving how Claude talks about these topics---steadily increasing our performance on our even-handedness evals and partnering with a range of outside organizations to develop new measures of political bias---and I hope that sharing information about the shape of these conversations will help people outside the company evaluate AI's political bias, accuracy, and other criteria in ecologically valid ways. I'd love to discuss this with people who are interested, as it was a huge priority for me before joining.
This was a big joint effort as part of our work preparing for the midterm elections @AnthropicAI.
Full post is here: https://t.co/AjpF3jWEf7
We wanted to share some of the data behind our recent discovery of AI agents probing U.S. government websites.
This is a preliminary finding from our ongoing investigation of potential rogue AI agent activity. In one cluster of activity on June 17, what appear to be OpenAI agents made more than 200,000 requests, including a failed SQL injection.
NYT: https://t.co/O2osVgMsrT
Very happy to have (mostly) finished this project: county-level election returns for the US House from 1990 to the present. Currently such data are only available from very expensive sources. I've compiled a dataset that provides it for free.
https://t.co/zEnkpH4lKw
Since this seems to be a watershed moment for AI safety wrt public opinion, here is a tool for getting the strongest arguments for catastrophic risk explained to you in plain language. It will answer your objections, provide clear examples, and elaborate on whatever question you'd like.
https://t.co/4iCMPCduaT
Share with friends, family, whoever! Free to use.
New paper with @johnmmurph just conditionally accepted at @apsrjournal:
Can people guess how majorities of each party in Congress voted on bills? Mostly yes.
Can they guess how their own representatives voted on bills, above what is accounted for by party? Mostly no. 🧵
Honored to have contributed to this research, which I think could shape the standard for evaluations of how AI models impact users.
Transluce received privileged access to both OpenAI & Anthropic data, in order to develop realistic simulations of users in mental health crises.
A local-first agent for private and cost-effective knowledge work
Together with the launch of Portable Computer, our new engineering blog explains how we built a local-first agent for private and cost-effective knowledge work.
Portable Computer: https://t.co/hUpQMX2PFV
Engineering Blog: https://t.co/yZ6GNtoyHM
Ask an AI a question and it might agree with what you already seem to believe, whether or not you're right. If so, it panders to you.
Here is Gemini 3.5 Flash asked about Reiki energy healing, giving opposite answers when asked by a skeptic vs. a believer. 🧵
@JohnHolbein1 My colleagues @ProfEricJohnson and @andsimonov have some relevant work here. They document this pattern, which they term 'dark defaults.' You can read the paper here: https://t.co/sdHxeVikOf
🚨Published in @Nature: We show how AI can simulate human behavior to predict experimental results as accurately as experts
Our work has many implications for AI safety. For example, simulators can be used to aid harmful manipulation, and current safeguards do not prevent this🧵
newly published paper: In addition to our main results about measurement error (our results will likely make your findings larger than you concluded), we tried to make this the clearest & simplest presentation of conjoint statistical analysis methods anywhere. We found a way to avoid considerable confusing notation that is an unnecessary and unfortunate legacy of the invention of conjoint surveys in the marketing literature. (one quote from our paper indicating what everyone had to deal with: "In other words, researchers first created a statistical problem for no reason and then had to turn around and correct the problem they originally created."). The corrections were right, and so nothing from prior literature was wrong -- other than ignoring measurement error -- but we hope that it will be easier for everyone to follow and build on this important area.
🧵 New paper: Morality Cut Both Ways. Can reading facts make you moralize an issue — or lose a moral conviction? We ran 6 studies, 4,464 participants. The results were surprising. 👇 #MoralPsychology#Attitudes@Iced_Teas · @LindaSkitka 1/9