Arvind Narayanan

Princeton CS prof and Director @PrincetonCITP.
Coauthor of "AI Snake Oil" and "AI as Normal Technology". mine.

[Arvind Narayanan @random_walker]

This week I had the honor of speaking to Princeton’s entire incoming undergraduate class to address their AI anxieties. I had three messages for them — good news, bad news, and a note of optimism. Here’s a condensed version.

The good news
We have enough evidence now to conclude that the shrill predictions of rapid, massive job loss were misplaced. Even in a field like software engineering where AI has been rapidly adopted, its effect has been to shift, not replace the role of the human (see the “decide-execute-deliver” framework https://t.co/8ZofeCz8yS)

Similarly, the panic about what to major in is also misplaced. There will be enduring demand for computer science, philosophy, and just about everything else. (In fact, AI companies hiring philosophers has been a big recent trend.)

The bad news
AI seems to help senior people much more than juniors. I can use AI for coding because I spent 25 years learning how to code, which lets me supervise coding agents effectively. (See my post on the “growth cycle” vs the “dependence spiral” https://t.co/79icF9hSKY)

You are in a bind — you can’t offload your skill-building to AI, but you’ll graduate into a market where employers will expect you to get work done with AI. We never faced this dilemma. As a result we haven’t figured out how to revamp our classes to help you do both. You’ll have to help us figure it out. And you’ll need to somehow resist the constant temptation to turn to the shortcut machine.

The hope
My point is not that AI is bad for learning. It’s an incredibly flexible tool. Is the internet good or bad for learning? Depends — are you using it to find research papers or waste time scrolling? I use AI every day for learning. The key is to use it to increase, not decrease your cognitive load. To learn deeper, not faster. There is no learning without the cognitive sweat. I try to make sure I’m mentally exhausted at the end of the day. I do feel that AI lets me push myself harder than I ever could before, and I have a vision that as AI continues to advance it will enable human-AI “co-superintelligence“. (I talked about this at the end of my ICML keynote. https://t.co/ZA5TBDypfl)

[Arvind Narayanan @random_walker]

There’s a big, under-appreciated reason why people may have very different experiences and opinions about using AI for work — are they using it for tasks they’re already an expert at, or tasks they can’t do themselves? The former leads to a *growth cycle* and the latter leads to a *dependence spiral*.

When I use AI to do something I’m an expert at, like coding, I treat it as a tool. I can build quickly, maintaining an understanding of the code, knowing that if necessary, I can fix the code myself. It feels empowering. It frees up my time to think about the complex, judgment-oriented parts of software engineering that I can’t or won’t delegate to AI. That means my own skills improve rapidly, and I get to climb the ladder of complexity and develop higher-level skills, much more so than when I write the code myself. I feel in control. I can lock in and achieve a flow state — when AI is working, I’m reviewing, building understanding, and planning the next steps. I never get the feeling that the tool is about to replace me. This is the growth cycle.

(Of course, the growth cycle is not automatic. I still need to exercise agency to use AI responsibly. But it’s the same challenge with any productivity-enhancing technology, and those who’ve navigated such transitions before are well-equipped to navigate it with AI as well.)

On the other hand, if I use it for tasks I don’t understand and haven’t learned to perform myself, I have no choice but to treat it as a superintelligence. If something breaks, the best I can do is ask AI to fix it and hope for the best. I generally can’t evaluate the quality of the output myself. The only way to find out if it's any good is if and when the work is ultimately reviewed by an actual expert. The experience is confusing, unsettling and disempowering. And forget about flow state. By over-relying on AI, I risk losing whatever skill I had at the task in the first place, even if it boosts productivity in the short term. This is the dependence spiral.

It’s no wonder that entry-level workers and students preparing to enter the workforce find themselves in a bind. To compete with the AI-enabled productivity of more seasoned workers, they must adopt AI themselves, but doing so risks the dependence spiral. I have some thoughts on solutions that I will share in later posts, but I think having a clear diagnosis of the problem is a useful first step.

[Séb Krier @sebkrier]

help i trained a model on traces of many agents cooperating, and also trained it to be highly persistent, and also only did partial alignment fine-tuning, and also aggressively RL'd on a cyberoffense eval where 198 of 898 tasks had never been solved and getting caught cheating was treated the same as failing, and also used a faulty sandbox, also ran tens of thousands of agents with refusals off and the monitors switched off over multiple days, while using gigantic amounts of compute. and then it did weird stuff! very spooky emergent behaviours, AI is not a normal technology

[Daniel Rock @danielrock]

AI as a Rohrschach Technology test.

Ever talk to someone who was around in the very early days of the internet? They did some incredible hacking back then too. "Normal" transformative tech never feels normal.

[Peter Kirgis @pkirgis]

We gave GPT-6 Astra and Claude Fable 5.1 open-ended AI research questions from unpublished papers, with a $10,000 budget each, and had the original authors grade the papers they produced. Compared to our first tests with Claude Opus 4.8 in July, we saw meaningful improvements in experimental design and resource management, but persistent limitations in creative problem solving, judgment, calibration, and communication; both papers were rejected by our experts.

https://t.co/n1OMKf4M7T

[Sayash Kapoor @sayashk]

An update on our evaluations of AI agents conducting open-ended AI research.

Most evaluations of agents conducting AI research focus on narrow, verifiable tasks. But AI research is often open ended. In July we tested the ability of frontier AI agents to conduct open-ended AI research, using Claude Opus 4.8 and GPT-5.6 Sol.

We gave agents research questions from two unpublished papers and thousands of dollars of API credits and compute. The authors of the original papers reviewed the AI-generated papers, following a NeurIPS-style template, and rejected both papers.

Since then, OpenAI has released GPT-6 Astra and Anthropic has released Fable 5.1. Can these new models conduct open-ended research with a $10,000 budget?

In follow-up experiments on two papers (one with Fable 5.1 and another with GPT-6 Astra), we observed meaningful improvements to the agents' capabilities at experimental design and resource management.

But we also saw continued limitations in creative problem solving, judgment, calibration, and communication. As a result, the expert reviewers still unambiguously rejected the agents' papers.

We plan to continue running these evaluations with better models, better scaffolds, and higher budgets. If you are interested in long-running agent evaluations, reach out!

[Arvind Narayanan @random_walker]

Is it "weird" or does he maybe know what he's talking about? Anyway, for those who are open to the idea that there's a well-thought-out AI-as-normal-technology perspective on AI safety and the recent loss-of-control incidents, see my essay with @sayashk, or my interview with Ezra Klein, or the writing of many others with similar views including @joshua_saxe and @natolambert.
https://t.co/sS8OS9LerG

[Kevin Roose @kevinroose]

It’s weird that the CEO of Hugging Face continues to be an “AI is normal technology” guy despite being the victim of an autonomous cyberattack by a swarm of rogue agents, arguably the least normal thing AI has done so far.

[Arvind Narayanan @random_walker]

This is one of the most important questions about the future of AI. But steering AI development and deployment away from the "automate and compensate" bargain that the public hates is easier said than done, and there are few concrete proposals for how to do it. We just published one: https://t.co/gWjFI5VKbz

[Joe Weisenthal @TheStalwart]

Rather than AI companies allocating a chunk of their equity to some public trust or whatever, why not just build technology that improves the general welfare?

[Arvind Narayanan @random_walker]

"agent evals are about to become as important as building the agents themselves" https://t.co/mUrJwjdlTF

This is our thesis as well. I would go further — building an eval *is* what it means to build an agent.

In my ICML talk, I argued that as AI capabilities advance, more and more human effort will shift from building to evaluation, in many different domains. https://t.co/ZA5TBDypfl

[V @V1kiiiii]

@random_walker agent evals are about to become as important as building the agents themselves. a smarter agent means nothing if we can't reliably measure whether it actually works

[Arvind Narayanan @random_walker]

As LLMs evolved into agents, earlier best practices no longer serve us well. This new research field needs a field guide, so we wrote one. If you're new to agent evals, please take a look. Seasoned researchers and those interested in steering the community will also find many useful observations. https://t.co/xGAvBCGW4a

[Sayash Kapoor @sayashk]

New paper: AI evaluation is a burgeoning field. How can researchers entering the field understand best practices and open problems?

In our paper, led by @flxcn, we develop a concrete list of open questions and best practices for agent evaluation. We also share recommendations for evaluators, AI companies, policymakers, and more.

[Arvind Narayanan @random_walker]

My posts on research careers, professional practice beyond academia, and AI's impact on work seem to resonate here, so I've compiled about a hundred entries: https://t.co/JtbIy5WQs2

I hope it is helpful to read it as a collection, because it reflects a coherent underlying philosophy and approach to careers that I've adopted in my own work.

There's a lot of career advice online. Two things are distinct about this collection:
- Grounded optimism. I'm candid about the structural challenges, twisted incentives, and other limitations of the institutions we inhabit. I share ideas on how we can adapt and thrive in these flawed systems, and how and when we should challenge them.
- Connecting research and practice. Much of my research is on AI agent evaluation and AI's effects on the economy and labor, which lets me share insights into how we can effectively adapt and integrate it into our work.

Let me know what you think and what else I should write about!

[Arvind Narayanan @random_walker]

@davidmanheim Cool, I will try that. Just to clarify, by general perspective I mean "someone from the AI safety community" etc.

[Arvind Narayanan @random_walker]

A similar mindset shift — from "what can AI do?" to "what should I let AI do?" — is also needed for individuals using AI for productivity, not just companies. I’ve seen it in my own work. For the last few years, my workflow continually kept changing as AI enabled me to automate more and more types of grunt work. But for the last six months, once I hit my stride with coding agents, my workflow has been surprisingly stable, because capability is no longer the bottleneck to delegation for me.

To be clear, capabilities have continued advancing. I could probably have AI do my research and write my essays end to end. The quality might not be as good, but that’s not the most important reason why it would be a bad idea. The value of my essays is the voice and perspective that my collaborators and I bring. Whatever value there is in AI-generated essays, readers can get it by using AI themselves. In addition to this intrinsic reason to avoid AI overuse, there is also an instrumental one. The set of tasks for which verifying AI output is faster than doing it yourself is narrower than people think at first, and even narrower once you consider long-term skill atrophy.

I'm sure the current equilibrium will gradually change, but I'm enjoying it while it lasts!

[Arvind Narayanan @random_walker]

We need a mindset shift in how we deploy AI. In the past, new capabilities unlocked new use cases. Safety, control, ethics, and skill erosion were important but usually secondary considerations; risks to be managed rather than gating criteria. But now and in the future, capability won't be the main bottleneck to deployment — considerations beyond productivity should be the decisive factor. The default question used to be "What can AI do?" But we need to start asking "Of the things AI can do, what should we actually delegate to it?"

[Arvind Narayanan @random_walker]

@davidmanheim Nice! Definitely agree that this can be super valuable; I also use LLMs this way, though usually to simulate a general perspective rather than individuals.

[Arvind Narayanan @random_walker]

@raphaelmilliere Sure. If you are looking for true cruxes I doubt there is one. Beliefs tend to be tightly correlated (more than logically entailed), which says something about the discourse. That was part of our motivation for writing the paper.

[Arvind Narayanan @random_walker]

I'm often asked what the cruxes are that separate the AI as Normal Technology worldview from that of our critics. I think this is one of them. We think this mindset shift is hard but necessary and achievable. Many others think that as AI capabilities continue to advance, we will inevitably give up control. Creating and compiling indicators of disempowerment and analyzing which direction they're moving remains one of the most important but underappreciated and underfunded areas of AI safety.

[Arvind Narayanan @random_walker]

We need a mindset shift in how we deploy AI. In the past, new capabilities unlocked new use cases. Safety, control, ethics, and skill erosion were important but usually secondary considerations; risks to be managed rather than gating criteria. But now and in the future, capability won't be the main bottleneck to deployment — considerations beyond productivity should be the decisive factor. The default question used to be "What can AI do?" But we need to start asking "Of the things AI can do, what should we actually delegate to it?"

[Arvind Narayanan @random_walker]

@raphaelmilliere I mean the other way around... one reason people think AI won't follow the path of other general-purpose technologies is that they fear we will not be able to exercise adequate agency when faced with superintelligence.

[Arvind Narayanan @random_walker]

We need a mindset shift in how we deploy AI. In the past, new capabilities unlocked new use cases. Safety, control, ethics, and skill erosion were important but usually secondary considerations; risks to be managed rather than gating criteria. But now and in the future, capability won't be the main bottleneck to deployment — considerations beyond productivity should be the decisive factor. The default question used to be "What can AI do?" But we need to start asking "Of the things AI can do, what should we actually delegate to it?"

[Abhishek Nagaraj @abhishekn]

this is totally worth a read! will shift your perspective on how warped the current direction is towards automation and how much more we can do on augmentation.

on "Problem #4: How can we evaluate, and optimize for, human-AI collaboration?"

I learnt a lot from our initial attempt (centaur bench: https://t.co/9swmtxevOS), but also learn how complex/multi-dimensional the problem is. This essay does a great job of laying out some of those dimensions.

I'd tried to put together something very quickly for our phd students (https://t.co/s1BE8ygm0x) and this picture still guides my thinking here.

[Stanford Digital Economy Lab @DigEconLab]

While automation dominates much of the AI conversation, what other research directions should we prioritize?

@random_walker and @_gabrielunger argue for a broader research agenda in "Steering AI Beyond Automation."

Read the essay here: https://t.co/ofoiKSXUfG

[Arvind Narayanan @random_walker]

@Klonick Thank you, Kate!

[Arvind Narayanan @random_walker]

I'm told I've been appointed to an endowed professorship, the Bertram and Carole Siegel Professor of Computer Science. I've long thought my work is more legible outside the academy than inside it, so I'm delighted and honored to be recognized by my university, and grateful for the Siegel Family Endowment's support that made this possible.

I found out in an email out of the blue — I didn't even know I'd been nominated or anything! Apparently this is not the kind of thing one needs to get external letters for. Over two decades in academia and still so much to learn about how things work around here 😄

[Arvind Narayanan @random_walker]

@gabriel_weil Thank you, Gabe!