Spent 17 years at one of the best litigation shops on earth. Then AI got good enough that I couldn’t stop thinking about what I’d build if I started over. So I started over. For more, check out my conversation with @DavidLat
Opinion: Leading litigator Chris Kercher is leaving Quinn Emanuel, where he represented clients such as Elon Musk and Ken Griffin, to launch his own AI-powered boutique, Exclusive Jurisdiction columnist David Lat writes. https://t.co/FWl1GXrK0v
I'm a peaceful man, Orin, I'll never fight a hypo. I read your question as being motivated primarily by concern about assigning credit. If you want the more direct analysis, then it is still good to use the credit system. This is because, as you identify, the amount of slop increases dramatically in the generative scholarship world. A human that submits a lot of slop will get discredited. But people who are judicious about what they publish (AI-assisted) will get valuable credit. In this world, your name on a paper is less a claim that you wrote it and more a bond that it's good. So yeah, by all means, submit the German paper, but do it at your own risk.
My bet: experienced litigators who know how to use AI will be very hard to beat.
I started Kercher Law to stake that bet with my career.
Now Ali Miller, Josh Card, and Afi Blackshear are joining me. 🧵
the ai thing makes this really really obvious but basically 99% of white collar labor is currently massively overcomped, and 1% is massively undercomped but companies are forced to smooth all to prevent rebellion with both pitchforks and delaware judges. hugely bad outcome
ego is going to be the career-killer of the next decade. experienced people too stubborn to learn. polished people too cool to be cringe. people who take pride in being on the edge hiding how behind they are.
know nothing! learn everything! be free!
Writing density is bandwidth question. Writer had ideas to share, reader has limited bandwidth to absorb. Lots of things (email, slack) should be rendered not as written, but with the necessary context and in light of absorption potential of the reader.
Chat assistants writing in a very low density style must point to the fact that humans, on average, prefer lower density writing that doesn’t say much beyond a core idea.
It makes me rethink what the purpose of writing and communication is in society. Perhaps it’s useful for writing to be a sparse soothing hum that’s mostly empty of meaning.
A lot of writing turns out to be more like a prayer or rhythmic, tribal chant. Repeating a core idea. Reaffirming it from 50 angles.
As practicing litigators know, the real gold standard is having to stand up in court and defend what you wrote.
Opposing counsel isn’t going to grade your answer - they are going to try to take it apart. They’ll find the bad fact you glossed over, the case you stretched, and the concession you didn’t realize you made. That experience changes how you evaluate the next piece of work.
My preferences are well and good and a worthwhile data point, but the more useful signal comes from what happens when the work is actually used.
The gold standard for evaluating complex legal work is partner review.
Partners can cost over $3,000 an hour and associates $1,000, making expert review expensive at scale.
Evaluating a model on LAB through expert review alone would cost millions of dollars.
In practice, we combine rubric-based LLM scoring with sampled human preference.
But rubrics miss errors beyond predefined criteria, and human reviewers get fatigued and make mistakes at scale.
We built a generative reward model to bridge this gap by training agents to approximate partner review.
We give these agents the original outputs, web search, and other tools to check our systems’ work.
We find that their judgments correlate strongly with expert lawyer review.
This approach will help us scale human review across model training and production products, and will be central to training Tenet 1.5.
Interesting. By the same token, while more nimble practices can decide faster because the decision has to traverse less distance, I don’t have any more attention than the owners at BL firms. Too many startups don’t optimize for the attention of their audience and spend too much attention on things like going around the zoom sharing favorite colors. Imo
This is a big challenge in big firms - the tech is evolving so much faster than committees can move, esp when those committees have to straddle IT/procurement/finance and practitioners.
Call after call with BigLaw innovation types to map out months long projects to consider Eno PDF.
Meanwhile, every day, random people just sign up and try it. Some like it, some don’t. But they find out in like 20 minutes and move on.
Historically it was reasonable to set up a months-long evaluation and if the committee meeting got moved a week or two to accommodate someone’s schedule, no big deal. This is not the case anymore
Faced with increasing consolidation in the asset management industry, the firm says its new tool may cut the billable hours required to draft one required form submission in half. https://t.co/lWPkUjBpol
A strange truth;
1. The models are intelligent enough to solve Navier Stokes when prompted to
2. The models do not have the agency to solve Navier Stokes without a prompter
The AI’s aren’t taking over,
but beware of the humans
who wield them