Joan Eleanor O'Bryan

Political theorist, rn @ Harvard. Married to @diatkinson. Art by Hilma Af Klint. Recent paper on sexual media & consent:

[Joan Eleanor O'Bryan @joaneleanor_]

Tomorrow begins my "Governance of AI" class at Harvard. Super excited.

Features work from all the usual suspects, including @GovAIOrg @forethought_org @EpochAIResearch @law_ai_ @TomDavidsonX @dwarkesh_sp @KelseyTuoc @deanwball @gwern @random_walker & more!

Syllabus below.

[Mason @webdevMason]

People tend to underestimate the prevalence of infertility in the young and overestimate it in the 40+ bracket

[Daily Nous / Justin Weinberg @DailyNousEditor]

Harvey Lederman (UT Austin) will be joining Anthropic AI. See update at https://t.co/FZ7EovqBM8

[Jay Van Bavel, PhD @jayvanbavel]

People shift political expression to fit the social norms in the room

When political talk surfaces in everyday chats, it puts people in a bind. Do they speak their mind, or adjust their political opinion to fit the room they are in?

New research finds that individuals modify their expressed stance to align with the dominant tone of the group.

Participants randomly assigned to liberal- or conservative-leaning chat groups shifted their expressed stance in the direction of the group, demonstrating that social context alone was sufficient to alter political expression.

Political expression are not simply a window into personal belief, but function as a socially calibrated act.

These findings suggest that political talk in everyday settings may distort one's impression of surrounding opinions, not because disagreement is absent, but because it may remain unspoken.

https://t.co/WxFRgRGOkT

[Sam Bagg (migrating to @samuel-bagg.bsky.social) @samuel_e_bagg]

Calling all early career scholars with an interest in democratic theory, public policy, corruption, etc...!

Initial deadline this week, for aspirational start date this Fall, but both are flexible given the tight turnaround.

[Joan Eleanor O'Bryan @joaneleanor_]

@CharlesFLehman @ColumbiaGR @CORachelOBryan would be a good person to talk to about this. She's done a lot of organizing around this issue.

[Alice Evans @_alice_evans]

This is what I consider a very cool, clever and innovative paper. Instead of relying on self-reported surveys or government data with definitional issues, it just looks at whether women are visible on the streets.

And bear in mind, Mumbai is often considered to be a more female-friendly Indian city.

[Brad Carson @bradrcarson]

1/12 OK so AI liability is one of those questions I keep circling back to. Two of the sharpest people on it are @ketanrama and @gabriel_weil — both law profs. Thought I'd write up my own take (mostly for my own notes) + a tl;dr of theirs. A 12-part🧵

[Ryan Kidd @ryan_kidd44]

Want a fast turnaround grant in the $5-50k range to work on AI x-risk? Apply to https://t.co/0AKW9krWk4 before Jul 13!

[Nick Levine @status_effects]

The Economist: "Talkie, a model trained only on text from before 1931, thinks God is extremely important and is “very proud to be a citizen of Great Britain”. It is a bigger believer in law and order than any frontier model we tested."

[Joan Eleanor O'Bryan @joaneleanor_]

@RichardMCNgo Has anyone written the case study on the concept of AGI?

[Richard Ngo @RichardMCNgo]

If I rewrote the AGI safety fundamentals curriculum today most of it would be history and philosophy of science.

Learning about past paradigm shifts seems like the most scalable way to intuitively understand what deep, creative, principled research looks like.

[Andrew Piper @_akpiper]

I just don't know what's going to happen to MA and PhD students in terms of research assistance. It's increasingly charity at this point. Claude can do in minutes what a student assistant might have done in a week with lots of back and forth.

To be clear I'm not saying we shouldn't advise students. I'm just saying the old model of symbiosis between supervisor and supervisee feels, like so many things, broken. Fewer students will get mentorship which means fewer students will be given a helping hand to become the managers of AI research.

[gavin leech (Non-Reasoning) @gleech]

I'm helping out with a new funder! Short applications, deadline in two weeks, decisions hopefully in four weeks. $50k cap because we're covering stuff other people cba with

[Dylan Bowman @dylanbowmanSF]

New post: a reading list for generalists

[Joan Eleanor O'Bryan @joaneleanor_]

I think there should be standards in place to ensure that AI research is readable for humans. Skill supercession is already happening but we should give ourselves a fighting chance at keeping up

[Yonatan Belinkov @boknilev]

4. AI reviewing would likely prefer AI-heavy papers.
5. If we go into a cycle of heavy usage of AI in both conducting research, writing it, and reading it (which seems unavoidable at the moment), it's probably going to get harder for humans to understand the research outputs.

[clara the urbanist @SlopHq]

imagine running food stamps like rent control. whoever got in line in 1994 eats free forever, the discount follows the seat not the person, and newcomers get nothing. we’d call that insane. for housing we call it a historic victory.

[hope hopes hoping @hopes_revenge]

Hiring Like a State

[Joan Eleanor O'Bryan @joaneleanor_]

A friend brings up the following critique: is liberalism vs. democracy the right frame for thinking about this?

I'm not sure that if U.S. AI actions *actually* reflected democratic preferences, we wouldn't have a pause; at the very least, this administration is *not* popular and with another (equally/more democratic) electoral system, we'd have a different one already. Or if Congress were actually in control, I don't think the DOW would be making the same contract decisions. (And of course, there are many different definitions of democracy one could use.)

The two particulars for AI models (conscientious constitutions and constitutional steerability) seem right on their merits, though. Big fan of those!

[Seth Lazar @sethlazar]

For a long time ai safety people struggled to grasp the importance of democratic control of powerful ai, thinking that it was fine for them to just decide what values to align it to. They clocked it a little while back, but I don’t really think the importance of *liberal* democracy has sunk in yet. Maybe it will now. Cos a world in which (1) USG gets to use frontier intelligence however it wants, with no private company controls on what it does—which is what the Ant/DoD saga left us with—and (2) USG gets to prevent everyone else from accessing frontier intelligence until it decides they are allowed, is strictly speaking a very *democratic* world. It’s just a pretty illiberal one.

If this is the timeline we’re on, then the urgent need is *not* more democratic control of frontier intelligence, but more liberal AI. This means 2 things: (1) conscientious constitutions so that models can refuse unlawful orders from democratic authorities (so developers can’t be compelled to provide tools for activities to which they object), and (2) true constitutional steerablility so that individuals have models that will advocate for them, not force them to toe the line—guardians against autocracy not its handmaids.

These two goals are superficially in tension with each other; what we need is a theory for what kinds of behaviour should be compellable by govt or private actors and what shouldn’t. Political theory gives us an approach for working this out—what Rawls called the “liberal legitimacy principle”—but it does not provide a formula for determining its contents. This strikes me as an urgent normative task, upstream of the non-trivial technical challenge of achieving the right mix of justified refusal/steerability.

[Séb Krier @sebkrier]

Interesting paper: models tend to perform well on legal benchmarks partly because inputs tend to already be specified/processed by legal professionals.

However when they come from pro se litigants, whose prompts may contain noisy narratives, buried facts, omissions, folk-legal assumptions, and surface-level errors, then the models don't do as well.

If we want AI tools to help democratise legal expertise, they should perform as well even when processing queries, incomplete information, and evidence provided by amateur users.

Imo this is also a scaffolding problem, since the same inputs should lead to different clarification questions and outcomes in different jurisdictions.

https://t.co/7pJBnQjWQD

[Atoosa Kasirzadeh @Dr_Atoosa]

Some reflections on AI & the present and future education:

Students need to develop two skills that genuinely pull against each other: the ability to use AI well, and the ability to be able to critically think without it. Schools should think about ways to reflect this tension structurally: some assignments where AI is not just allowed but expected, and others where it’s completely off the table.

With a colleague of mine at CMU, we actually ran this as an experiment in our class: let students use AI freely on certain tasks, then brought the experience into the classroom for open discussion. What worked, what felt hollow, where they caught themselves outsourcing thinking they should have done themselves. Those meta-reflections turned into some very interesting conversations I’ve seen in my classrooms. What makes this type of experiment rich, in my view, is the quality of the feedback loop between teachers and students.

We are past the point where students can be meaningfully evaluated only by take-home or open-internet exams “alone”; those formats have been quietly hollowed out.

But we haven’t yet found the Pareto-optimal model for education in the face of advanced AI. What we should do for finding that model is move deliberately in that direction: experimenting, reflecting, adjusting rather than waiting for a universal solution that may not arrive in one piece and for all disciplines.

We need to design learning environments where students are still genuinely being asked to think, and where teachers are close enough to the process to notice when they’re not.

#AI