Narutatsu Ri

PhD Student @PrincetonPLI interested in AI safety/misalignment. Previously BS @Columbia

[Narutatsu Ri @narutatsuri]

A bit overdue, but "Do Thinking Tokens Help with Safety?" was selected as the 🏆 Best Paper at the ICML 2026 AI4GOOD workshop!
The prize money will, of course, be put to good use (i.e., API credits) 😉

(And all credits to @qw3rtman for taking the great photos!)

[Sanjeev Arora @prfsanjeevarora]

Surprising finding about AI safety from our group. We find that in small open-parameter models, the thinking tokens have almost no effect on safety. On unsafe prompts, the refusal/compliance is predictable from a simple trained head given the prompt alone. Great work and thread by @narutatsuri

[Luxi (Lucy) He @LuxiHeLucy]

What happens when AI is guided by law-like principles? Can we design some computational tools to "debug" rules?

Check out our new work 📝 𝕊𝕥𝕒𝕥𝕦𝕥𝕠𝕣𝕪 ℂ𝕠𝕟𝕤𝕥𝕣𝕦𝕔𝕥𝕚𝕠𝕟 𝕒𝕟𝕕 𝕀𝕟𝕥𝕖𝕣𝕡𝕣𝕖𝕥𝕒𝕥𝕚𝕠𝕟 𝕗𝕠𝕣 𝔸𝕀🧑‍⚖️ to find out more!

🧵(1/10)

[Hayoung Jung @hayounggjung]

First paper of my PhD with my amazing advisors!

There’s been a ton of hype and media coverage on OpenEvidence as an “AI co-pilot for clinicians”… and our long-horizon benchmark puts them to the test!! Our results suggest they are far from reliable for downstream use.

[Liane Galanti @lianegalanti]

Robot policies can move but can't think.
LLMs can think but can't move.
So we connected them.
Real robot: 16.7% → 97.3%
Sim (LIBERO-PRO): 12.8% → 53.3%

[Furong Huang @furongh]

While FIFA has captured everyone’s attention, something consequential happened in academia this week, especially for international students in PhD programs.

DHS issued a final rule changing how long F-1 students may remain in the United States.

A thread on what it means and what students and universities can do. 🧵

[Narutatsu Ri @narutatsuri]

@MH2023ML @qw3rtman Thanks!! Haha, I'm not at all the qualified person 🤣

[Narutatsu Ri @narutatsuri]

A bit overdue, but "Do Thinking Tokens Help with Safety?" was selected as the 🏆 Best Paper at the ICML 2026 AI4GOOD workshop!
The prize money will, of course, be put to good use (i.e., API credits) 😉

(And all credits to @qw3rtman for taking the great photos!)

[Sanjeev Arora @prfsanjeevarora]

Surprising finding about AI safety from our group. We find that in small open-parameter models, the thinking tokens have almost no effect on safety. On unsafe prompts, the refusal/compliance is predictable from a simple trained head given the prompt alone. Great work and thread by @narutatsuri

[Narutatsu Ri @narutatsuri]

@chengyun01 @qw3rtman Thanks Catherine!!

[Narutatsu Ri @narutatsuri]

@qw3rtman Thanks :)))

[Shmuel Berman @ShmuelBerman]

I am excited to share my work as an @AnthropicAI Fellow!

Can language models control robots? ⬇️

[Simran Kaur @kaur_simran25]

New Paper: Rethinking On-Policy Self-Distillation for Thinking Models

On-policy self-distillation (OPSD) is a promising approach to recursive self-improvement. Thinking models seem especially well positioned to teach themselves with privileged information since they can deliberate, verify, and self-correct.

Surprisingly, we find that OPSD can make thinking models worse.

(More details in 🧵below)

[Narutatsu Ri @narutatsuri]

@cindy_x_wu Congrats again Cindy!!! 🎉🎊

[Rico Angell @rico_angell]

It’s deployment time!

You’ve done the pre-deployment evals. You THINK your model is safe, so you ship it 🚀

🚨 After deployment, reports of misbehavior start trickling in

What happened?? How could you have caught it?? 🤔

@icmlconf 2026 Spotlight!

🧵

[Zixuan Wang @zzZixuanWang]

A question on synthetic data generation: If we want a language model to solve k-step arithmetic problems (such as a+b*c-d=?), with operands from 1 to 100, which training distribution should we use?

A. Uniform distribution: Sample these k operands uniformly from 1 to 100
B. Power law: randomly shuffle 1-100 and impose an artificial power law. Sample these k operands according to this power law.

⚡Our ICML 2026 (spotlight) paper shows: Option B is better! Surprisingly, the same idea extends far beyond this simple example to many reasoning tasks that require implicit composition of multiple atomic skills, including multi-hop QAs and synthetic GSM problems.

📄Paper: https://t.co/8LVlmCCBz9
📝Blog: https://t.co/t2yhIJ0s1P

[Sanjeev Arora @prfsanjeevarora]

Surprising finding about AI safety from our group. We find that in small open-parameter models, the thinking tokens have almost no effect on safety. On unsafe prompts, the refusal/compliance is predictable from a simple trained head given the prompt alone. Great work and thread by @narutatsuri

[Narutatsu Ri @narutatsuri]

I’m also at #ICML2026 and will be giving an Oral presentation at the ICML 2026 AI4GOOD Workshop (https://t.co/kdmkHSdp8t)!

This will be on July 10th 13:30–14:45 at Grand Ballroom 103. Please stop by if you're around and have time!

I’ll be around all week, so please feel free to reach out if you want to chat about AI safety and/or (mis)alignment!

[Narutatsu Ri @narutatsuri]

New Paper: Do Thinking Tokens Help with Safety?

Reasoning models generate “thinking” tokens before answering, which improves performance on many tasks like math and coding.
A natural hope is that models can also use thinking to deliberate about whether to refuse or comply a request.
But how true is this?

We find a surprisingly sobering answer: current reasoning models often do not use thinking tokens for real safety deliberation.

Work with: Abhishek Panigrahi, @prfsanjeevarora at @PrincetonPLI

(More in 🧵below)

[Narutatsu Ri @narutatsuri]

(5/n) Finally, we study how existing inference-time and training-based defenses change model behavior: while many are motivated to “activate safety reasoning inside the thinking trace,” do they actually improve the ASR-ORR trade off and/or improve deliberation signals?

We find that most do not: reducing harmful compliance often comes at the cost of more over-refusal, and current methods actually suppress the already scarce deliberation signals in the thinking trace instead of increasing them.

[Narutatsu Ri @narutatsuri]

New Paper: Do Thinking Tokens Help with Safety?

Reasoning models generate “thinking” tokens before answering, which improves performance on many tasks like math and coding.
A natural hope is that models can also use thinking to deliberate about whether to refuse or comply a request.
But how true is this?

We find a surprisingly sobering answer: current reasoning models often do not use thinking tokens for real safety deliberation.

Work with: Abhishek Panigrahi, @prfsanjeevarora at @PrincetonPLI

(More in 🧵below)

[Narutatsu Ri @narutatsuri]

(3/n) Next, we ask whether the visible thinking trace contains real safety deliberation.

We use frontier models to annotate each sentence in a reasoning trace and identify “oscillations”: places where the model appears to deliberate between refusal and compliance. At these points, we roll out many final responses to test whether the text-level deliberation actually changes the final refusal/compliance distribution.

Result: most visible deliberation signals have no meaningful effect on the final decision (grid-shaded region in right bars).