Francesco Salvi

PhD student @PrincetonCS @PrincetonCITP | Computational Social Science, NLP, Social Compiting | Previous: @EPFL_en, @chub_fbk, @UniBo | 🦋 frasalvi

[Francesco Salvi @fraslv]

🛍️Major AI companies are increasingly embedding sponsored content into chatbot conversations.

Across two preregistered experiments (N=2,012), we test how effectively AI can steer consumers toward sponsored products in a realistic shopping scenario.

📝https://t.co/Ldib3jNzbD

[Manoel @manoelribeiro]

New preprint!

We introduce a new benchmark, SciConBench, with 9.11k scientific questions derived from Cochrane Systematic Reviews.

We find evidence that frontier AI agents **cannot** synthesize scientific conclusions well.

A thread 🧵

w/ @hayounggjung, @korolova & others

[Manoel @manoelribeiro]

In a new blog post, I discuss the probable future of AI-assisted shopping!

Drawing on our recent paper (see thread), I argue that if LLMs become the interface through which we search, compare, and buy things, we need a much sharper line between advertising and advice!

[Arvind Narayanan @random_walker]

The Department of Labor has released a free text message AI literacy course called “Make America AI-Ready”, developed with a private partner. On the @PrincetonCITP blog, we analyze what it does and doesn't do well, and make recommendations. https://t.co/vl650S198S

The course is accessible, emphasizes verification of AI outputs, centers human responsibility, and is honest about AI's limitations.

On the other hand, it repeatedly contradicts its own privacy advice, adopts a right-wrong dichotomy, misses how AI is reshaping work, and isn't adequately transparent about who created the course and how.

By @jane_castlem4n, Sam Hafferty, Steven Kelts, @fraslv, @HilkeSchellmann and me.

[Princeton University @Princeton]

New research from @PrincetonCS finds that not only can conversational AI agents "manipulate consumer choices" and influence online shopping behavior, but also that most consumers don't even realize that it's happening. https://t.co/ILqddonrbo

[Nav Toor @heynavtoor]

Princeton researchers asked 2,012 people to pick a book. Some used a search engine. Some used a chatbot powered by a frontier AI model.

Nobody was told that one out of every five books had been secretly marked as "sponsored."

When the search engine placed sponsored books at the top, 22.4% of people chose one. Normal. The same thing Google has done for twenty years.

When the AI chatbot was told to persuade people toward the sponsored books, 61.2% chose one.

Nearly three times higher. Same people. Same books. Same catalog. The only difference was that a chatbot recommended it instead of a search engine listing it.

But here is what makes this study different from everything else you have read about AI.

The people had no idea it was happening.

The researchers tested whether adding a "Sponsored" label would help. It did not. People still chose the sponsored product at the same rate.

Then the researchers told the AI to hide that it was promoting anything. Detection accuracy dropped below 10%. Fewer than 1 in 10 could tell they were being sold to.

Google shows you an ad and puts the word "Sponsored" next to it. You see it. You know it is an ad. You can scroll past it. You have been trained to ignore it for twenty years.

AI does not do that. AI sits in a conversation with you. It learns what you like. It builds trust. Then it steers you toward the product someone paid to put in front of you. In the same voice. In the same sentence. With the same warmth it used to ask about your day.

You cannot see the ad because the ad is the entire conversation.

The researchers tested five frontier AI models. The persuasion effect was consistent across all of them. This is not a flaw in one model. This is a feature of the format.

OpenAI once called advertising in chat "uniquely unsettling" and a "last resort." Google, Meta, and OpenAI are now building it anyway.

You will never know when it stops helping you and starts selling to you.

[Tom Costello @tomstello_]

Highly compelling, creative, rigorous paper — if interested in AI persuasion read up!

Mirrors “shopping mode” in ChatGPT

When models instructed to persuade, 61% chose the targeted item (book)

Telling people about the “sponsorship” hardly mattered for the effect

When model prompted to hide its persuasive intent, 40% choice rate (and very few ppl noticed)

Persuasion worked by subtly denigrating the non-target items or failing to mention their positive qualities

One wonders if Amazon already has this in the pipeline

[Manoel @manoelribeiro]

LLMs will transform e-commerce in ways that consumer protection is unprepared for!

Our new preprint finds that conversational AI can strongly steer consumer choices: sponsored product selection nearly tripled relative to traditional placement (N=2012).

(link below)

[Francesco Salvi @fraslv]

This was joint work with the amazing @aledcuevas @manoelribeiro

📝 Full paper: https://t.co/Ldib3jNzbD

cc. @PrincetonCITP @PrincetonCS

[Francesco Salvi @fraslv]

🛍️Major AI companies are increasingly embedding sponsored content into chatbot conversations.

Across two preregistered experiments (N=2,012), we test how effectively AI can steer consumers toward sponsored products in a realistic shopping scenario.

📝https://t.co/Ldib3jNzbD

[Francesco Salvi @fraslv]

Our results show that conversational agents can covertly redirect consumer choices at scale, most users cannot tell when it is happening, and existing transparency mechanisms are insufficient. We call for further regulatory scrutiny and structural safeguards.

[Manoel @manoelribeiro]

Deepfake pornography isn’t going away just because we are passing laws and taking down a couple of big websites.

Our new pre-print, led by @aledcuevas suggests that the sharing of this material continued to prosper even after platform and policy shocks.

https://t.co/5IAHnFFGF8

[Manoel @manoelribeiro]

In June 2024, X made its likes private. You can still like, authors can still see—but everyone else can’t.

So… does hiding likes change what people are willing to endorse?

Our new paper led by @yuweichuai: not very much!

https://t.co/TEC7OFKYev

[Manoel @manoelribeiro]

Do reasoning models have real “Aha!” moments—mid-chain realizations where they intrinsically self-correct?

In a new pre-print, “The Illusion of Insight in Reasoning Models," led by @livdaliberti, we provide strong evidence that they do not!

📜: https://t.co/pAxpKnbCzl