New blog post on harness optimization. We hit Sonnet 4.6 performance with a 7x cost improvement.
Fable 5 was the first frontier model release that evaluated on legal tasks. It only scored 13%, the worst performance among all benchmarks evaluated.
@Harvey released this benchmark called Legal Agent Benchmark (LAB) just a month prior. It contains a set of realistic legal matters. Each task gives the agent a closed workspace of documents (contracts, emails, spreadsheets, slide decks) and asks for a concrete deliverable: a diligence memo, an issue list, a redline, a draft. An LLM judge grades the deliverable against a long rubric containing 61 distinct binary criteria each on average.
Many frontier models such as Gemini 3.1 Pro don't surpass 0% all-pass rate (all rubric criteria passed). With automatic harness optimization, we manage to push DeepSeek V4 Pro from 0% to 5% all-pass rate, achieving parity with Sonnet 4.6 for 1/7 of the price.
Read the blog post for the details: https://t.co/kBrWrQkgJW
BREAKING: The Pentagon reportedly delayed announcing the U.S. strikes on Iran until after the stock market closed at 4 p.m. ET on Friday, with the timing intended to minimize the immediate impact on financial markets, according to NBC News.
Introducing GLM-5.2: Frontier Intelligence, Open Weights
- Significant improvements in coding and agentic tasks
- Strong long-horizon capabilities with a 1M context window
- Two levels of reasoning effort: GLM-5.2 (max) pushes the limits, while GLM-5.2 (high) strikes a strong balance between performance and token efficiency
- MIT-licensed open weights
- Same API pricing as GLM-5.1
Tech Blog: https://t.co/LAsxUdN0JZ
Weights: https://t.co/g0A1C4UWx4
API: https://t.co/Kc3E22cbN7
Coding Plan: https://t.co/Nk8Y98HNhU
Chat: https://t.co/WCqWT0qCQb
Learn to ship. Shipping is a skill distinct from coding. Shipping is designing, coding, QAing, story-telling, teaching, marketing, selling, pivoting, iterating…
It used to be that coding dominated in importance because of coding ability scarcity. AI will push you to go further.
Here’s a fun comparison between GLM 5.2 and Opus 4.8 on a one-shot reproduction of the SDPO paper
This is a hard task: the model must resolve messy verl issues and then run ablations to completion and confirm the paper’s claims.
- GLM 5.2 costs $6.21 while Opus 4.8 cost us $46.35
- Both models spent a bulk of their tokens resolving initial verl issues. GLM 5.2 attempted 14 failed runs before first success while Opus 4.8 attempted 9 runs.
- GLM 5.2 surprisingly took 2.65M tokens (excl re-reads) compared to 4.53M tokens for Opus 4.8
Introducing Claude Tag, a new way for teams to work with Claude.
In Slack, Claude joins as a team member with access to the channels and tools you choose. Tag Claude in and delegate tasks to it while you focus on other work.
So the whole focus is on limiting distribution and not really plugging the leak! 🚁📶
The ministry has already accepted the fate that paper will indeed be leaked. 🤦
Centre’s ban on access to Telegram in India till June 22 challenged before Delhi High Court.
The plea was mentioned before Justice Tejas Karia who allowed urgent listing for today.
#Telegram#DelhiHighCourt
राजस्थान यूनिवर्सिटी ने बेशर्मी की सारी हदें पार कर दी है।
समाजशास्त्र के सेकंड सेम के पेपर में सवाल की जगह जवाब और थ्योरी डाल दी। कारण? AI से जेनेरेट किया गया पेपर बिना चेक किए बांट दिया।
हंगामा होते ही परीक्षा कैंसिल कर दी गई।
This is state of our education system.
🇬🇧 British Prime Minister Keir Starmer has announced a forthcoming total ban on social media for children under 16, saying that such platforms are "making children unhappy".
➡️ https://t.co/YalZqGPrHL
a YC founder i'm working with was worried because one customer had already built a simple version of his product in-house.
i told him that's not a bad sign, and it's actually the best proof point he has.
if a company is spending expensive engineering time on the problem you're solving, that validates a bunch of things at once:
- the pain is very real
- other vendors are not good enough
- many companies likely have the same pain too
"‘Every Judge must at all times be conscious that he is under the public gaze and there should be no act or omission by him which is unbecoming of the high office he occupies and the public esteem in which that office is held."
— Restatement of Values of Judicial Life, adopted by Full Court of Supreme Court of India, 7 May 1997.
I guess the standards are different these days.