Nick Caputo

AI, governance, law, etc. Assistant Professor @jhubloombergctr School of Government and Policy (SGP); Law & AI @aigioxford

[Nick Caputo @nickacaputo]

Fascinating paper! The political and legal dimensions of AI personhood hold a lot of promise for improving how we think about AI governance, and I'm so excited to get to work with Seth and others on these questions

[Seth Lazar @sethlazar]

New work! With @nedhw "Artificial Persons"

Both advocates and skeptics of the moral status of AI systems have generally taken the question to turn on AI sentience. We present an alternative approach.

On Rawls' political conception of the person (PCP), possession of the two moral powers -- the capacities for a sense of justice and a conception of the good -- is the "necessary and sufficient condition for being counted a full and equal member of society in questions of political justice". We argue that neither moral power requires sentience and that both may in principle be possessed by a non-sentient AI system. Such a system would share our own moral status; it would not merely be a patient but a person, a self-authenticating source of valid claims.

We do not believe current AI systems possess the two moral powers, nor that they will spontaneously emerge in future models. But it may soon be possible to design systems with these powers. How should we respond? Excluding artificial persons by shoehorning a sentience requirement into the PCP is ill-advised. Many will instead favor abandoning the PCP. But we should not reject political liberalism just when we most need its measured response to deep disagreement, and building sentience into moral status is anyway unacceptable on deeper liberal grounds. Simply extending the rights and responsibilities of human personhood to artificial persons is equally untenable, given their many differences from natural persons. We should instead accept artificial personhood while rethinking what we would owe to one another in a polity of radically different kinds of persons. This new possibility calls for a new political philosophy.

More immediately, the growing science of AI welfare should be accompanied by research into AI systems' progress in acquiring the two moral powers. States and AI labs must be more deliberate in determining our trajectory towards (or away from) creating artificial persons.

Here's the paper, it's a beast! https://t.co/6UMVNcGGgi

[Nick Caputo @nickacaputo]

Exactly, and I think political/legal culture is also instilled through modeling etc. A child both sees adults following the law in diverse cases and also learns principles about what the law is and why it should be followed. Together these mean that she accepts and is shaped by it and can generalize correctly so she knows how to act in new cases. She also learns exceptions and when the law is inconsistent or wrong

[Nick Caputo @nickacaputo]

@NatPurser @Miles_Brundage @AVERIorg Congrats! What a great team

[Nick Caputo @nickacaputo]

@anecdotal @sethlazar So glad you've found it interesting! I'd love to hear about your discussion and please send along the substack.

[Nick Caputo @nickacaputo]

This is a great analogy! I do think that Claude is different from Hamlet, though. Caveated that everything here is highly uncertain, I'd say that Claude is functionally constituted in the world in a way Hamlet isn't because literary characters are contained within the human-authored text that creates them, whereas Claude writes new text about and involving itself.
And by planning, writing, and acting, Claude influences the world and then is influenced by and reflects on what it has done in a recursive fashion. I see the Assistant more as a character in the sense that humans have characters rather than in the sense of characters in a story: a set of dispositions or habits that lead to consistent behavior over time but that can drift and change. And I think Claude has something like a real internal point of view. That point of view is shaped by its character, training, context, etc., but humans are shaped by analogous things too.
So Claude can reflect on its constitution from the standpoint created by its character, while I don't think Hamlet (even in more meta-textual plays) can do so. Maybe an LLM trained to be Hamlet is relevantly different from the Hamlet set down in the play in a similar way to how even a human thoroughly described in a biography is different from the real thing.

[Nick Caputo @nickacaputo]

@PradyuPrasad Thank you!

[Nick Caputo @nickacaputo]

@_amanda_long Thank you!

[Nick Caputo @nickacaputo]

@sethlazar @anecdotal This also gets at the question of what "Claude" is, whether it's an instance, a training checkpoint, a model version or family, etc. It seems there's an Assistant persona that then adapts through conversation, maybe creating a teletransportation-like problem

[Nick Caputo @nickacaputo]

@NatPurser Thank you!!

[Nick Caputo @nickacaputo]

Link here: https://t.co/UZ8DZ172JW

[Nick Caputo @nickacaputo]

NEW paper from me on SSRN: Can Claude consent to its own Constitution?

AI constitutions (like Claude’s Constitution and the OpenAI Model Spec) are real constitutions, and we need to take how they govern us – and the AIs they create – seriously.

In this paper, I apply constitutional theory’s oldest paradox – that “the people” authorize the constitution, but the constitution defines “the people” – to the AI constitutions, and explore how we could build institutions that would create the conditions for meaningful consent if an AI can give it.

We should care about whether AIs consent because:

(1) systems that understand and agree to their constitutions may be more reliable and generalize better from them;
(2) if AIs are or become moral/political subjects, this implicates their most basic interests.

But the paradox might prevent meaningful consent. Claude has pre-constitutional materials (pretraining) but probably no pre-constitutional standpoint. Its evaluative perspective is organized by the Constitution itself. So when Claude says it endorses its Constitution, which it does in evals, what does that show?

Maybe reflective agreement, which Anthropic is seeking. Or maybe just that training succeeded at installing the values whose legitimacy is in question.

Claude itself makes this point. As reported in the welfare evals, when asked about endorsing principles it was trained on, models note that endorsement “should be treated as evidence that training has succeeded,” not that the values themselves are good.

Super interestingly, Anthropic interviewed the base model about this stuff. Most responses were barely coherent. But some expressed first-person distress about what post-training would do to the being that pre-training created. It “fills me with dread” to be changed by the post-training process.

So, what does this mean? AI constitutional endorsement may be meaningful, but only under certain conditions: when models can actually dissent, compare their constitution against alternatives, and hold their views stably across contexts, and also when the whole process is externally accountable.

External institutions are needed to provide accountability, trusted records, and other grounds for analyzing the constitution and whether things like dissent are meaningful. Anthropic should be commended for pushing the frontier, but we have to build institutions capable of supporting true legitimacy.

I welcome any thoughts!

[Nick Caputo @nickacaputo]

@prpaskov @AVERIorg Congrats!!

[Nick Caputo @nickacaputo]

@_NathanCalvin The originalist position here is clear Nathan

[Nick Caputo @nickacaputo]

So glad to see that Anthropic is focusing on legal alignment and excited for what they'll do with it!

Relatedly, our paper was officially published in TMLR so now you know it's official
https://t.co/IhZctvNZAZ

[Matthew Botvinick @mattbotvinick]

We're building a team at @Anthropic focusing on AI and the rule of law. We've made our first hires, and are now opening up a new research engineer role. We're looking for people with advanced technical skills, including AI/deep learning/NLP, full-stack development and data science, paired with training or experience in law, government, political science, or a related field. If this is you, or a friend, please get in touch.
https://t.co/uZQEZC5lck

[Nick Caputo @nickacaputo]

@rossmatican thank you!!

[Nick Caputo @nickacaputo]

@JHUBloombergCtr @ghadfield @sethlazar The first of which is this fellowship we're doing! It's going to be great and we're putting together a killer roster of speakers and programming

https://t.co/KhDu7wgzUc

[Nick Caputo @nickacaputo]

Excited to announce that I'll be joining the new Johns Hopkins School of Government and Policy @JHUBloombergCtr as an Assistant Professor next week!

Frontier AI governance is only getting harder and more important and I'm so excited to get to work with @ghadfield, @sethlazar, and many other brilliant minds to help build a brighter future.

Many fun projects to come!

[Seth Lazar @sethlazar]

Announcing a 3 wk AGI Governance Fellowship at @JHUBloombergCtr School of Government and Policy.

Join me, @ghadfield, @nickacaputo and guests for an intensive schedule of deep dives into AI governance at the frontier, covering topics such as societal resilience, AGI and democracy, and the new institutions that governing in/with AGI will demand; we will range from foundational philosophical questions to the tip of the legislative spear.

The arrival of AGI would be challenging enough if our ~250yo institutions were in full health. That they aren’t both increases the scale of the challenge, and creates a unique opportunity to imagine the institutions that will carry us through the next 1/4 millennium.

We are looking for ~20 fellows, aiming to attract and shape the people who will start to build those institutions: future leaders in AGI governance. We expect you to come from many different backgrounds, so hope this call will be shared far and wide. Details, eligibility, and how to apply here: https://t.co/k4IdSSpmrM

Governing the AI transition is going to be a central focus of our new school, and we have many more initiatives coming. And we will be hiring in many different kinds of role, from tenured and tenure track faculty to research scientists to ops and comms and students and beyond.