Scott Singer (宋杰)

fellow @CarnegieEndow | frontier AI policy

[Scott Singer (宋杰) @Scott_R_Singer]

President Trump travels to Beijing this week to meet President Xi — and AI is on the agenda. This is a genuinely encouraging development.

In @ForeignAffairs, @cqknight_ and I argue that while Washington and Beijing compete fiercely on AI, they must mitigate the most extreme dangers it presents to the world.

Fortunately, working together is not only necessary, but with the right approach, it is also feasible. The key is to focus on how to look for risks rather than the specifics of what they find.

[Matt Clifford @matthewclifford]

The @AISecurityInst is hiring for a Director and for a Chief Research Officer. AISI is a remarkable organisation: doing globally important work, with a world-class team, in the heart of government.

These are some of the highest impact jobs in AI security anywhere. Do consider applying and sharing widely.

[Matt Sheehan @mattsheehan88]

New piece from me in the wake of new White House EO on AI. It looks at a seeming paradox:
- For the past 4 years, 🇨🇳 has had the world's most extensive and burdensome AI regulations.
- During that same period, Chinese AI companies largely caught up w/ their 🇺🇸 peers.

Link below

[Matt Sheehan @mattsheehan88]

Jumped on @BloombergTV to talk growing concern in 🇨🇳 about AI x jobs.

One interesting bit: labor displacement likely comes down to scale/speed of transition. Chinese policy on not firing workers are targeting that transition period.

Starts at 5:30 mark
https://t.co/TCdNwoYoyp

[Matt Sheehan @mattsheehan88]

Really enjoyed this one. We unpacked the announced 🇺🇸-🇨🇳 AI dialogue, China's approach to AI agents, and how it's thinking about labor displacement. Check it out!

[Matt Sheehan @mattsheehan88]

New reporting from WSJ confirms what I'd heard about He Lifeng getting quite concerned about AI x jobs. Always helpful to get some corroboration on what you hear from the rumor mill.

My piece on the topic linked in reply.

[Scott Singer (宋杰) @Scott_R_Singer]

What @AISecurityInst has built in such a short time is nothing short of remarkable. Many of the people in these roles have taken substantial pay cuts to be in government. The UK government has built something they, and many of us on the outside, believe deeply in

[Xander Davies @alxndrdavies]

I moved to London 3 years ago to join @AISecurityInst, at the time a few people with visitor passes and a whiteboard. Since then AISI has become the world’s largest and best-funded group in gov focused on AI security & safety. Fun to be in @nytimes!

[Seán Ó hÉigeartaigh @S_OhEigeartaigh]

On balancing safety cooperation with competition, Beth Knight and Scott Singer's article is one of a growing body of great pieces.
https://t.co/y6Kf2SHDGO

Bucknall et al is also essential reading.
https://t.co/no589L2APR

[Scott Singer (宋杰) @Scott_R_Singer]

@rubyscanlon @CatieEdmondson Great work, Ruby!

[Scott Singer (宋杰) @Scott_R_Singer]

https://t.co/6zHrqCpJEj

[Scott Singer (宋杰) @Scott_R_Singer]

Great to chat with @PuckNews's @IKrietzberg on Biden-Xi. It ultimately matters less right now whether the U.S. and China agree internationally for an agreement. What matters much more is whether the U.S. and China do a risk mitigation seriously themselves. It's time to get going.

[Scott Singer (宋杰) @Scott_R_Singer]

Super important and timely article from the remarkably talented @ZilanQian. Explains rapid individual-level AI adoption in China –– fear of job loss, grounded in the historical reality where that was a lived reality for the previous generation. The fear of AI dressed as techno-optimism lives in the U.S. too but with different history, culture, and economic systems.

[Zilan Qian @ZilanQian]

Why is the Chinese public so optimistic about AI? Why are they not worried about job displacement? Why do so many people rush to use OpenClaw?
People in China and the US are not different species with opposite societal and technological expectations. It is because the massive layoffs three decades ago taught the Chinese that every transformation is "the last bus" — miss this one, and history will progress without you.

My new article @asteriskmgzn. Thanks @anton_d_leicht and @Scott_R_Singer for their feedbacks, and the editorial team for all the hard work.

[Scott Singer (宋杰) @Scott_R_Singer]

It's time to build

[Alec Stapp @AlecStapp]

We are entering a new era where the binding constraint on philanthropic startups is no longer funding.

The scarce factors are good ideas & the talent who can execute on them.

Now is the time for entrepreneurial people to start new organizations that create public goods.

[Matt Sheehan @mattsheehan88]

From China's Ambassador for Disarmament Affairs. I've heard that this dept at MOFA is leading on AI diplomacy, and they've beefed up their team a bit. If they are MOFA's lead for US-China talks, that'd be an improvement on the North America bureau people who led the 2024 meeting.

[Scott Singer (宋杰) @Scott_R_Singer]

You can just do things

[Markus Anderljung @Manderljung]

My new hobby is making oddly-specific AI-generated music.

E.g. one album about working hard, another about unsung heroes, and one about CrossFit.

I just put some on Spotify if you wanna listen.

[METR @METR_Evals]

Could an AI company lose control of its own agents? To find out, Anthropic, Google, Meta, and OpenAI let us (1) test their best internal models with CoT access, (2) review non-public info about capabilities, alignment, and control.

The result: our first Frontier Risk Report.

[Scott Singer (宋杰) @Scott_R_Singer]

Fantastic piece from two rockstars, @lucaslfluegel and Nick Corvino, on one of the most important companies in biotech across the U.S. and China. Love to see it @jordanschneider

[Scott Singer (宋杰) @Scott_R_Singer]

https://t.co/tdKCXo19Cx

[Scott Singer (宋杰) @Scott_R_Singer]

Good to chat with @Jonathan1Gibson at @thedispatch on AI during the Trump-Xi meeting. Sound pretty simple, but simply explaining how policies work can do a lot to combat the most extreme risks.

[Scott Singer (宋杰) @Scott_R_Singer]

https://t.co/40hIXcW16i

[Scott Singer (宋杰) @Scott_R_Singer]

This is welcome news. As I wrote last week for @CarnegieEndow, basic dialogue on AI safety is in each country's enlightened self interest, and there are some clear, no-regrets policy options (which risks are most serious, how to structure evals, post-deployment safeguards). Glad we've moved at least a bit past 2024.

But we need to move into the substance, quickly. This statement is basically a punt on the more granular work that needs to begin. What each country does –– both for each other, and more importantly, for themselves –– will shape whether this dialogue actually makes both sides better off.

[Vincent Chow @vince_chow1]

Beijing confirms China-US intergovernmental dialogue on AI guardrails.

“Foreign Ministry Spokesperson Guo Jiakun held a regular press conference May 19. A reporter asked, 'It is reported that the leaders of China and the United States exchanged views on the regulation of AI during their meeting and agreed to conduct dialogue and cooperation. What is China's comment on this?'
Guo stated that as two major AI powers, China and the US should work together to promote the development and governance of AI, and to help AI better serve the progress of human civilization and the common well-being of the international community. During President Trump's visit to China, the two leaders had constructive exchanges on AI issues and agreed to conduct intergovernmental dialogue on AI."