mark williams

Co-Director Vanderbilt AI Law Lab

[Daniel W. Linna Jr. @DanLinna]

Are Berkeley Law faculty prohibited from using AI-detection tools, or merely warned that AI-detection tools may be inaccurate and biased? I'm curious what schools are doing about this, as I've only seen warnings.

It's news to me that it was obvious in the 2000s that ML is biased. If that were accurate, a number of groundbreaking studies published starting around 2016 wouldn't have been such a big deal.

Prohibiting AI detection tools does not solve the bias problem. As @olawaleidowu_ pointed out, the concern is AI Police who overconfident in their ability to identify AI-generated content. https://t.co/EXgoRyD6vm

[Andrej Karpathy @karpathy]

Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.

[John Nay @johnjnay]

we’ve developed proprietary benchmarks for legal reasoning maintained by attorneys at Norm.

because Norm Law attorneys, while serving as outside counsel to hedge funds, PE firms, etc. deploy AI agents in their day-to-day work, we can uniquely build real legal AI benchmarking.

we've been tracking frontier models across generations, and the trend is clear: models are improving substantially in their legal reasoning capabilities.

the latest generation of models are nearly indistinguishable from each other in accurately answering legal questions.

most models are increasingly consistent, with the most recent generation of frontier models reaching the same conclusion ~90% of the time.

but for high-stakes legal work, even the best models on this benchmark reach a different answer often enough that, at scale, users receive contradictory answers to the same question every week.

to integrate ai agents into high-stales legal workflows, you need both (1) purpose-built systems that can constrain, verify, and govern AI reasoning automatically, and (2) the human overlay of expertise for live workflows fully intertwined/integrated w/ ai agents in a deliberate process.

[Ed Walters @EJWalters]

If you think clients buy documents and hours from law firms, this is bad news for lawyers. But if you believe that clients seek discernment, counseling, risk management, and advice from law firms, and documents are just artifacts, this is terrific news for lawyers and clients.

[Dean W. Ball @deanwball]

Think about the power Hegseth is asserting here. He is claiming that the DoD can force all contractors to stop doing business of any kind with arbitrary other companies.

In other words, every operating system vendor, every manufacturer of hardware, every hyperscaler, every type of firm the DoD contracts with—all their services and products can be denied to any economic actor at will by the Secretary of War.

This is obviously a psychotic power grab. It is almost surely illegal, but the message it sends is that the United States Government is a completely unreliable partner for any kind of business. The damage done to our business environment is profound. No amount of deregulatory vibes sent by this administration matters compared to this arson.

[Dean W. Ball @deanwball]

The belief that AI is mostly fake remains shockingly common, including, as Derek points out, among journalists and takesmen, but also *people with AI in their job titles.* There are people *who do AI policy for a living* who think AI is mostly fake and don’t use it.

[Rany Jazayerli @jazayerli]

The clear winner of the first half: Patrick Mahomes.

Y'all wanted someone other than the Chiefs in the Super Bowl. You reap what you sow.

[Ethan Mollick @emollick]

If you are considering taking a job offer, you may want to ask what your token budget will be.

[Kevin Roose @kevinroose]

don't worry guys, they're just stochastic parrots

[John Palmer @johnpalmer]

you kinda seem more like a Claude Cowork user (derogatory)

[Anthony @omgitsbirdman]

Josh Allen always better than Pat until it's time to actually be better than Pat

[Hammer and Nigel @hammerandnigel]

TRANSFER PORTAL UPDATE:

(And if you get this joke, we could totally drink moonshine together and be friends)

[Ahmad @TheAhmadOsman]

me watching Claude Code write the code for me

[Chase Snyder @ChasingSnyder]

Here’s the specific 5 minutes everything changed at Arrowhead and it became Arrowhead. Everyone knows it when you go through history. Broncos/Chiefs, 1990. False start, Broncos. Elway goes back to the line again, realizes they’re screwed to even operate, and has to BEG for help.

[Idaho Football @VandalFootball]

Flagship ✌️

[Ethan Mollick @emollick]

I meet a lot of very smart AI critics who never seriously try to make AI work for them by spending a couple of hours with a frontier model. People can be (and should be & are) critical after realizing what AI can do, but experience leads to better-informed and sharper critiques.

[Ethan Mollick @emollick]

The problem is not just the proliferation of devices that let you record people without their knowledge, but the fact that multimodal LLM let you use recordings in ways that neither law not society anticipated.

Everyone has an easy way to mine hours of
footage. No forgetting.

[Ethan Mollick @emollick]

So every major model is already exceeding or will soon exceed the EU's systemic risk FLOP limit when it comes into effect next year.

[Ethan Mollick @emollick]

Many firms built around the limitations & cost assumptions of GPT-3.5 class models, and are now stuck with complex solutions that are more expensive & worse than a reasoner without any scaffolding

You need to build solutions with an eye towards riding the cost/performance curve.

[Ethan Mollick @emollick]

This study is being massively misinterpreted.

College students who wrote an essay with LLM help engaged less with the essay & thus were less engaged when (a total of 9 people) were asked to do similar work weeks later.

LLMs do not rot your brain. Being lazy & not learning does.