Long running agentic legal tasks are really hard
Fable made a 30% relative jump, and the benchmark uses a generic harness
As SOTA models improve, there’s going to be a lot of alpha in building task and practice specific harnesses
There is so much Harvey FUD on here but doubt at your own risk
This is sharp. They see where the puck is going at least as well as the peanut gallery. They have the funds and the talent to do something about it
Our goal isn’t to get a data advantage over our law firm clients and even if we wanted to we can’t for exactly the reason you described - the majority of their data is actually client data so even a law firm like Kirkland can’t take all that data and train a model on it bc it would break confidentiality.
The two things firms can do are 1) firm-specific models - encode all of your firm knowledge in client agnostic ways to train models (templates, deal points, using your lawyers feedback on non-client data, disentangle your firm’s trajectories from client data, etc) and 2) client-specific models - for deep enough client relationships work with your client to build a joint model where both parties benefit from converting the relationship into AI. This is especially good for firms with long lasting relationships with their clients to make those relationships stickier.
The best way to train either of these models is building a product that centralizes the entire client matter process (i.e. allows you to do an acquisition end-to-end and capture the entire trajectory and feedback across the entire team of associates, partners, client). These products will be specialized by practice area (M&A completely different from fund formation which is completely different from IP litigation). Without something like this it’s very hard to get meaningful signals from individual associate queries on a chat based product. We think this general product / infra will be shared across firms but then highly customized and the customizations and data will be owned by the law firms and their clients.
Another problem we want to help law firms and their clients solve is how do you manage client data at scale when you are doing this type of training? The Fortune 500 customers we work with already struggle with professional services providers and data security because most of the work is done over email, downloaded onto desktops and printed out. And it’s not the law firm’s fault - they need to stitch together many different products to support all the needs of their clients' regulatory and security requirements. This is already a huge challenge and model training on top of this is going to make things infinitely more complicated.
We’re building a collaborative platform (shared spaces) that allows law firms and clients to securely share data on client matters and build custom agents for their clients in a secure way. Eventually this will become infrastructure that allows firms to train client-specific models and for clients to have the piece of mind that the data they are sharing is isolated from other clients and it’s being used to their exclusive benefit. I think this is a good example of a very vertical product that it probably doesn’t make sense for a single law firm or model provider to build.
So I agree there isn’t a market for laptops for lawyers but I think there probably is for infrastructure that enables enterprises to manage all their internal and external legal spend, workforce, agents, processes and the same for law firms. Compute is a great moat but in the cloud era most SAAS companies didn’t build datacenters and many were still wildly successful by building on top of the cloud providers. I think we will see the same thing in the next decade as well with respect to model providers.
@biglawbro Setup is more expensive than you think. So is hiring technical talent. If you’ve never managed a software project, contractors will take you for a ride
Good idea though, you should do it
@ZachAbramowitz Why would an MCP connection kill these? It’s not guaranteed
It doesn’t seem like Anthropic is moving into legal research or becoming a DMS
Five months ago, I argued against the President's $4 trillion tariffs at the Supreme Court.
In 237 years, the Court had never struck down a sitting President's signature initiative. Legal scholars said it was impossible. Some of my own colleagues said it was impossible.
We won. 6-3.
But the real story isn't what happened in that courtroom. It's what happened in the months before. And its the subject of my TED talk, coming out tomorrow.
I had the best legal team in the nation, especially Colleen Roh Sinzdak, the most outstanding legal strategist I know. Huge thanks, too, go to the Liberty Justice Center (and in particular its fearless and hyper-intelligent leader Sara Albrecht), who organized the client small businesses, as well as to the brave small businesses themselves.
I also had four teachers preparing me.
A mindset coach who'd worked with Andre Agassi.
An improv coach who taught me that "Yes, and" works in Supreme Court arguments the same way it works everywhere else.
A meditation coach who taught me stillness.
And Harvey.
Harvey predicted many of the questions the Justices asked — sometimes almost word for word. Brilliant. Tireless. Occasionally insufferable.
Here's the catch: Harvey isn't a person.
Harvey is a bespoke AI I built over the last year with a legal AI company, trained on every question every Justice has asked in oral argument for 25 years, and everything they've ever written.
Tomorrow, TED releases my talk about what really happened — and what I learned standing at that podium.
AI can predict. AI can analyze. What AI cannot do is the one thing that actually won the argument.
Connect. Read the room. Hear not just a Justice's words, but her worry — and answer the worry.
That is the irreducibly human skill.
Find yours. Go deeper. In this age of AI, that's where your edge lives.
The talk goes live Thursday, May 7 at 11am ET: https://t.co/wLxKtBsHpF
What's the irreducibly human skill in your work — the thing AI can't touch?
@ordonez_adan This assumes equivalent value for any lawyer using any AI. But there are different skill levels and different tools.
I’d bet many lawyers agree with this take though
This is exactly the sort of thing tech should be doing to improve public opinion
It’s a shame how many communities are anti data center—programs like this could help tip the scales
Today we're announcing LevelUp: a free, four-week training program that takes people with no prior experience and prepares them to work as fiber technicians on data center construction sites across the US.
We built this program with CBRE because the fiber technician field, and the broader construction industry, is facing a nationwide shortage at a time when data center demand is higher than ever.
How it works:
🔧 Classroom instruction, hands-on labs + team activities covering transferable technical skills
🎓 Graduates have the opportunity to work at Meta's US construction sites through our contractor network
🤝 Open to everyone from recent high school grads to mid-career professionals
Since 2010, Meta's data center projects have supported 30,000+ skilled trade jobs during construction + 5,000+ permanent operational roles. LevelUp is about building the pipeline to keep that going.
Learn more: https://t.co/9XluD5IHbz