Sam Redlich

Neuro-Symbolic AI. Mathematical Ontologies. Reasoning, Verification & Trust. Expanding Computational Capabilities, Pattern Generation & Predictive Models.

[Sam Redlich @SamRedlich]

Harvey's grader work is the most interesting legal AI publication of the month, and not for the reason it was written. That and the other Legal AI bets made this month by Legora, Latham, and Morgan & Morgan. Check it out. https://t.co/538zXK9yg6

[Sam Redlich @SamRedlich]

A database can revoke its own conclusion. It can’t erase what others already received, relied on, or acted upon. Silence inside software doesn’t undo consequences in the real world. #LegalTech #DataIntegrity

[Sam Redlich @SamRedlich]

Import the same quarter-end event twice, and a careless system creates duplicate reporting tasks. Idempotency ensures the event maps to the same record—because duplicate obligations can become undisclosed audit items. #Compliance #SoftwareArchitecture

[Sam Redlich @SamRedlich]

Extraction, interpretation, and action are three separate boundaries. Blur them, and an AI’s mistaken reading can become an unauthorized client communication before a lawyer reviews it. #LegalTech #AI

[Sam Redlich @SamRedlich]

In legal AI, a blank field isn’t “no”—it’s unknown. Treating uncertainty as fact can trigger false liability. When new evidence changes a conclusion, the system must map what’s already been drafted, completed, or sent. #LegalTech #AI

[Sam Redlich @SamRedlich]

legal rl sounds like fun, i think i will give it a tinker or two #legalai #rl

[Sam Redlich @SamRedlich]

Law isn’t static: rules, cases, and obligations shift over time. For AI to give trustworthy legal answers, it must track those changes as they happen. #LegalTech #AI

[Sam Redlich @SamRedlich]

@nikogrupen @harvey fun times

[Sam Redlich @SamRedlich]

In law, the answer is often “it depends.” Unlike chess, code, or math, there’s no simple ground truth—and that’s exactly where generative AI struggles. #AI #Law

[Sam Redlich @SamRedlich]

The law never stands still. Statutes change, cases are overruled, and legal AI must keep pace in real time to provide reliable, trustworthy answers. #LegalAI #ArtificialIntelligence

[Sam Redlich @SamRedlich]

In law, the answer is often “it depends.” Unlike chess, code, or math, there’s no simple ground truth—and that’s exactly where generative AI struggles. #AI #Law

[Sam Redlich @SamRedlich]

What happens to an instruction as agents delegate to agents? The FSB and HKMA now address sub-agent delegation, and MasDrift measured unauthorized actions in up to 19.8% of tasks in centralized hierarchies. Run the audit on your own chain. #aigovernance https://t.co/taFZQXSlkc

[Sam Redlich @SamRedlich]

Good question. My side is pinned: each entry quotes the artifact at a recorded commit. The evals belong to the publishers. Where a repo keeps a changelog, each patch adds an entry there, so a www gold answer is on the record. Where none exists, you're right. Scores should name the commit they ran against…

[Sam Redlich @SamRedlich]

I have filed 16 corrections to legal AI benchmarks, skills and templates published by Harvey, LegalQuants, Anthropic, General Legal, Thomson Reuters and Crosby Legal, and started a public register to track them.

https://t.co/5LfwH4peWc https://t.co/UapUvz1YDE

[Sam Redlich @SamRedlich]

@syfert you found the fools gold!

[Sam Redlich @SamRedlich]

@jsarihan interesting approach

[Sam Redlich @SamRedlich]

@Sony91121271 sounds about right

[Sam Redlich @SamRedlich]

Good evals writeup from Crosby. The gap: nothing in the pipeline stands for what a clause does (who owes what to whom, which conditions, which defined terms). Without that object, 55% attorney disagreement reads as subjectivity. Most of it is wording and hygiene a diff resolves... @aveekdg

[Aveek Duttagupta @aveekdg]

Designing evals for contract review is one of the most complex technical problems I’ve encountered.

Ten great lawyers will redline the same contract ten different ways. How do you tell an agent what good looks like?

We've cracked some big parts of the puzzle and are finally ready to share.

[Sam Redlich @SamRedlich]

@jeffreyhuber @TheEthanDing i feel really revved up

[Sam Redlich @SamRedlich]

@deanwball like a really good math equation that derives answers about reality and does not depend on heuristics

[Sam Redlich @SamRedlich]

@abraham_jmb good idea