Dave Hoffman

William A. Schnader Prof. of Law @pennlaw.

Teaching writing & tweeting mostly about contracts.

Against et al. Pro legal jargon. Neutral on legal process.

[Dave Hoffman @HoffProf]

@LakeShowLeg True enough!

[Dave Hoffman @HoffProf]

The DOJ suit against Alaskan certification is a press release dressed up as a complaint, to be sure.

But also, dems pushing a candidate to act as a honeypot for confused voters is hard to defend on any ground apart from that it's literally lawful.

[Dave Hoffman @HoffProf]

@Utahdesert @PermianLigand

[Dave Hoffman @HoffProf]

@Utahdesert @PermianLigand I just said this exact thing.

[Dave Hoffman @HoffProf]

@PermianLigand I certainly think it makes it clear enough, which is why it doesn't violate the law to do, as I said!

[Dave Hoffman @HoffProf]

A core worry is iterated exchanges between AI authors/reviewers converging on forms of proof and writing that humans can't follow, and, long term, w/out injections of new human text, unravelling. This isn't limited to domains like law (though its worse in them).

[Dave Hoffman @HoffProf]

AI-screening across domains (including government services and adjudication) is the way to deal with AI-supply, but getting it right at equilibrium seems incredibly challenging.

[Yonathan Arbel @ProfArbel]

Yesterday, @thehoffprof asked how we can manage law review article selection in times of the slopalypse

Today, a new study offers some evidence for fighting AI flooding with AI. In head-to-head comparisons, AI-generated reviews beat reviews by paid human experts. AI judges overwhelmingly preferred them. Three former journal editors also preferred them in 26 of 30 judgments.

That doesn't mean that AI should select our future scholarship. The downsides are obvious. The study doesn't really test selection decisions, much less questions of scholarly impact. But it makes the case for using AI-assisted triage to direct scarce human attention toward interesting, rigorous, and creative articles

Complain about it, by all means. But the flood is coming either way. If you don't have a better AND realistic solution, the complaint won't get us very far.

[Dave Hoffman @HoffProf]

@ProfArbel @apublicgood His solution was to hire you and have you help students with selection.

[Dave Hoffman @HoffProf]

Maybe I can make on to the LPE blog after all.

[Kim Krawiec @KimKrawiec]

“Contract law is not fundamentally a coordinating enterprise. It just doesn’t do that very well.” Dave Hoffman takes us from one $1 million NIL provision to the larger problem of an unraveling college-sports market. New on Taboo Trades. https://t.co/Sh74psdBUN

[Dave Hoffman @HoffProf]

@apublicgood @ProfArbel sure!

[Dave Hoffman @HoffProf]

Again, this screening use of Pangram is *way better* that later offer revocations, or relying on unread disclosure mandates whose details will vary across journals and time.

[Dave Hoffman @HoffProf]

I've been thinking about how law reviews (and academics) should respond to AI writing, and this 👇 is my basic starting point. Pangram is a badly designed purity stick.

[Peyman Milanfar @docmilanfar]

People treat Pangram as an infallible oracle. They weaponize its probabilities to shame others by some arbitrary purity standard. The irony is that it’s mostly redundant: obviously AI-generated stuff is already recognizable without it. And for more nuanced writing, Pangram fails to distinguish between text written by a human and edited by AI, vs text that is generated whole-cloth from a prompt.

At the end of the day, @pangram ’s goal is about monetizing people’s anxiety about AI - it’s not about protecting creative integrity. They sell a false binary certainty to those gullible enough to ignore probabilities.

[Dave Hoffman @HoffProf]

So, though pangram *can be manipulated*, journals might conclude that the post-hoc fussing itself solves some of the problems I've just listed, and authors who fail to do so should be held back at the door.

[Dave Hoffman @HoffProf]

@MarcODeGirolami

[Dave Hoffman @HoffProf]

A problem with this clever idea is that there are fields -- I'm looking at you, conlaw -- where the directionally correct prior should be that legal novelty is stupid.

[Robert Anderson @ProfRobAnderson]

We need an inverse-Pangram. It tells you what percentage of your article is substantively original (i.e., doesn't map onto existing literature, whether word-for-word OR paraphrased, whether AI generated or not).

[Dave Hoffman @HoffProf]

@KimKrawiec Shockingly I actually did say this.

[Dave Hoffman @HoffProf]

@r_tallarita @OrinKerr I agree with Roberto, as is typically the case.

[Dave Hoffman @HoffProf]

In this vision of the thing, the clients are readers of legal scholarship, and I for one am very grateful for editors' service.

[ʕ•ᴥ•ʔ the average bear @an_average_bear]

@brianlfrye Counterpoint: Law reviews are about exposing students to the experience of doing extra homework at the beck and call of tedious jerks on behalf of ungrateful clients

[Dave Hoffman @HoffProf]

@BrianLeiter You read the thread where I said "journals should use pangram even though it is imperfect" and you said "that's incorrect." And it's not my substack, but rather Roberto's, one of a 1000 explaining why it's a noisy signal. Maybe you are just moving too fast!

[Dave Hoffman @HoffProf]

@P_Wannemacher Since the relevant population is legally-trained readers, a bit hard to access at scale, what kind of effect size would you think is reasonable to hypothesize, and consequently what kind of power analysis would you recommend.