Sol Messing

Research Professor @NYU @CSMaP_NYU, @Georgetown MDI. Past: @Twitter, Acronym, @Facebook, @PewResearch, @Stanford.

[Sol Messing @SolomonMg]

@stspanho Want for bike 🔥

[Sol Messing @SolomonMg]

@ey_985 @mollyeroberts @b_m_stewart @j_a_tucker Oh and the website where we replicate all of these findings on the latest models! https://t.co/ueI1LhNlQC

[Sol Messing @SolomonMg]

New in Nature: LLMs give "the party line" in the languages of authoritarian regimes. This works when they control the media, which feeds pretraining data. We show more state control over media means less critical LLMs. 6 studies w 38 languages & 13 models. Details ↓

[Sol Messing @SolomonMg]

@ey_985 @mollyeroberts @b_m_stewart @j_a_tucker Full text complimentary link https://t.co/FapB1R0f66

[Sol Messing @SolomonMg]

Cool discussion of our work in @WSJ from @Lingling_Wei 🙏🙏👏👏 https://t.co/0WBzs6Gu1g

[Sol Messing @SolomonMg]

New in Nature: LLMs give "the party line" in the languages of authoritarian regimes. This works when they control the media, which feeds pretraining data. We show more state control over media means less critical LLMs. 6 studies w 38 languages & 13 models. Details ↓

[Oliver Maclaren @omaclaren]

Have we checked the unit distance write up for hallucinated citations

[Sol Messing @SolomonMg]

Interesting work on AI disclosure across cultures. Chinese users disclosed most and German users least.

[Tetsuro Kobayashi @tkobyashi]

A co-authored paper on self-disclosure to AI chatbot is published. It's a cross-national study in Mainland China, Germany, Japan, Hong Kong, and the U.S., and it's open access.
https://t.co/TliN7sVeXn

[Sol Messing @SolomonMg]

@rishdotuk We’ve been dancing with mediocrity since time began!

[Sol Messing @SolomonMg]

A big problem to solve in eval science is how to prevent benchmark leakage while keeping things open enough that things like this can come to light

[deckard @slimer48484]

half the virology questions in MMLU are wrong🤣🤣🤣

what are we DOING?!

[Sol Messing @SolomonMg]

A side effect of Claude Code diffusion in the academy - our high performance computing clusters are at near-full capacity because CC has lowered entry barriers to computational science

[Sol Messing @SolomonMg]

@calculatedplot Thanks! Yea it seems to influence the all generation mostly by language

[Sol Messing @SolomonMg]

@calculatedplot We have a bunch of results where we translate the prompt beforehand!

[Sol Messing @SolomonMg]

https://t.co/jly0Dfljg9

[Victor Shih @vshih2]

Yes huge findings from my colleagues

[Sol Messing @SolomonMg]

Good summary but to clarify it’s all AI in Chinese (eg this is not just Deepseek)

[Robin Brown @rcmb]

China's investment in media control shapes the data that Chinese AI is trained on influencing outputs

[Sol Messing @SolomonMg]

@paul_t_miller An influential book to be sure. What we’re arguing here is less about surveillance and more about influence and control, as exercised in part by laundering the source of state messaging through LLMs

[Sol Messing @SolomonMg]

@Bongo__Fury @SevaUT Yes exactly

[Sol Messing @SolomonMg]

@Bongo__Fury @SevaUT Sure yes

[Sol Messing @SolomonMg]

@Bongo__Fury @SevaUT Yes I think this is right but contamination to me sorta suggests it’s easy to remove. My point is that the fix may not be so simple, esp in eg China

[Sol Messing @SolomonMg]

@OpnBrdrsAdvct @threadreaderapp 🙏 thanks for your interest

[Sol Messing @SolomonMg]

@HorsmanGeoff That’s why we don’t include Canada - it cannot possibly dominate the languages spoken by its citizens

[Sol Messing @SolomonMg]

@HorsmanGeoff Well Canadian media definitely criticize the party in power, and even if state funded outlets are less likely to do so, what matters for LLMs is whether there is criticism in English (and French) and there certainly is in the case of Canada.