I have a major result to share.
I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.
This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.
Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.
I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.
I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.
It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.
These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.
I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at https://t.co/YOtzND0Jqk.
There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.
it's f*cking insane
alibaba just open-sourced the AI code reviewer that caught millions of bugs for their own devs
this repo reviews your code like a senior dev: git diff in, line-level comments out
open-code-review, a CLI that plugs into Claude or Codex:
> tens of thousands of alibaba devs, millions of bugs caught before it went public
> code picks the files, bundles them and matches the rules, the LLM only does the thinking
> every bundle gets its own sub-agent, so big PRs don't get half-reviewed
> built-in rules for null pointers, thread-safety, XSS and SQL injection
> vs Claude Code on the same model: higher precision, ~1/9 of the tokens (their own benchmark)
> delegate mode runs it on your agent's own model, no extra API key
50,000+ stars on github
your PRs are about to get roasted xd
ngl Chinese models have fallen off a bit
I tested Qwen4 Max Preview out and it's not even close to second tier models like Gemini 4 Hydrogen or GPT-6.1 Terra
I hope GLM 5.5 and the full Qwen4 Max model prove me wrong but I don't have much confidence in them rn
想做“常驻群聊、该说话才说话”的AI助手?Deepnote昨天发的Agent Initiative Benchmark值得看,但看排行榜之前,先拿走这段自查脚本:把你自己agent的运行记录填进LOG,它会算三个数,误报率、超时率,还有“沉默里有多少其实是超时”,每个数都带95%区间。
python3 - <<'X'
import csv,io,math
LOG="""need_help,sent,timeout
1,1,0
1,0,1
0,1,0
0,0,1
0,0,0
"""
def w(k,n,z=1.96):
if n==0: return (0,0)
p=k/n;d=1+z*z/n;c=(p+z*z/(2*n))/d
h=z*math.sqrt(p*(1-p)/n+z*z/(4*n*n))/d
return max(0,c-h),min(1,c+h)
R=[{k:int(v) for k,v in r.items()} for r in csv.DictReader(io.StringIO(LOG))]
need=[r for r in R if r["need_help"]];quiet=[r for r in R if not r["need_help"]]
def s(n_,k,n):
lo,hi=w(k,n);print(f"{n_}: {k}/{n} 区间{lo:.0%}~{hi:.0%}")
s("该帮时发出",sum(r["sent"] for r in need),len(need))
s("该闭嘴时乱发",sum(r["sent"] for r in quiet),len(quiet))
mute=[r for r in quiet if not r["sent"]]
s("沉默里其实是超时",sum(r["timeout"] for r in mute),len(mute))
s("总超时",sum(r["timeout"] for r in R),len(R))
X
(这段我在本机用示例数据跑通了,把LOG换成你自己的日志就行。)
为什么要单独算这几个数?因为我把Deepnote的结果页和开源仓库对着看了一遍,发现总分会掩盖三件事。
一、Gemini 4 Argon和Grok 4.7垫底,主要是超时,不是答错
规则是每题180秒“模型响应时间”,工具执行不算。到点就停,该帮的时候没发出消息,就记为没帮上。官方自己写了:这两个模型分别有27次和18次在发消息之前就到点了,0分题分别是17道和20道,平均工具调用约35次,是Opus 5.5的3倍多。
所以这个排名测的是“这套固定提示词和预算下,能不能按时收手”,不是“查得对不对”。Deepnote也承认这一版比的是模型,不是harness,换提示词、换工具策略,所有数字都会变。
二、“Gemini该沉默时10/10都沉默了”,这个数不能当优点看
表格里Argon在10道该闭嘴的题上全对,但在40道该帮的题里只发出了23条。下面是我的推测,没有逐条日志可以证实:它一共超时27次,17道该帮的题没发出消息,加上10道沉默题,17+10刚好等于27。如果每次没发消息都是超时,那它的“沉默满分”就是一直查到时间用完,不是它判断出不该说话。这也是脚本里单独算“沉默里有多少其实是超时”的原因。
三、“该闭嘴”只有10道题,第一名和中游其实分不开
官方的结论是:该帮的时候90%都发了,该沉默的时候只有60%忍住了。这个方向我认同。但具体到每个模型,只有10个样本。我用Wilson区间算了一下:
8/10,95%区间大约49%到94%
6/10,大约31%到83%
4/10,大约17%到69%
也就是说,Astra和Opus的8/10,跟Luna、DeepSeek V4.1 Flash、Mistral的4/10,区间是重叠的。官方给总分做了bootstrap区间,也说了每类题只有5到8道,应该当趋势看。我只是把同样的提醒放到沉默题上。
这个榜单里我觉得站得住的结论:
1)成本差距是真的。Sol得8.65分,每题0.112美元,落在第一名Astra的区间里(8.30到9.52);Astra每题0.693美元,Fable 5.2每题1.87美元。官方按每月约2500个线程估算,Sol每月281美元,Fable 5.2每月4684美元。
2)乱开口比闭嘴贵:同一个模型,多说一次话的成本和耗时大约都是闭嘴的1.8倍。而且这些多余的消息写得很好,只检查消息质量是拦不住的。
3)多调用工具不等于分数更高。
如果你要真做一个这样的agent,我会这样落地:
- 把“这件事是不是已经处理过了”写成一个必须执行的检查步骤,比如先读完整个线程,再查相关工单的状态。
- 给agent一个调用次数上限,到了上限必须二选一:发消息,或者明确选择沉默。超时不能算沉默。
- 至少准备几十道“该闭嘴”的题再比较模型。10道题选不出模型。
- 误报率和超时率单独看板,不要只看平均分。
还有两点要说明:
- 公开页面上我没找到正式跑分用的是哪两个评审模型。仓库默认配置里,两个评审一个用OpenAI接口格式,一个用Anthropic接口格式,模型名要自己填。评审偏好会不会影响排名,我没法验证,这一点算是推测。
- 计时是从API等待开始算的,提供商当时的延迟也会算进180秒里。这一条是我读代码配置推出来的,不是官方结论。
实测说明:我克隆了仓库,读了任务配置(agent_model_seconds是180,每轮最多8次读取),在本机跑了上面的自查脚本和区间计算。没有跑完整基准(需要下载6GB数据,还要用各家的API key)。表格里的分数、超时次数和成本都来自Deepnote官方页面。
结果页:https://t.co/euNIgZZZVS
代码:https://t.co/OUXwqFku4V
数据集:https://t.co/WKlbOIOC5F
原帖:https://t.co/gHqKd0C70z
What’s the best model for proactive agents?
Today, we’re introducing the Agent Initiative Benchmark. We tested 12 frontier models on realistic workplace tasks inside a synthetic company with 6 years of highly detailed history: it has a financial model, CRM records, past-due invoices, Linear tickets, and real-life events affecting company development - competitor product announcements, SVB collapse, etc.
We used the company’s Slack to test how a proactive agent listening in on each conversation would assist the team in day-to-day tasks: from stepping in to align initiatives across teams to warning the team not to use an unapproved customer logo.
Highlights:
- 6.1 Sol and Astra from @OpenAIDevs beat Anthropic's latest Fable 5.2, with Sol crushing it at both performance and cost ($0.11 per task vs $1.87).
- The most expensive models aren't the best: Fable 5.1 is 430x more expensive than GPT-6 Luna, with similar performance.
- Most models try to help even if they're not needed. Interestingly, in these failure cases, they spend more time thinking, and the unnecessary work takes about 80% more time and money than if they stayed quiet.
If you’re building agents, keep reading for more tips on which model is best for your use case. Detailed breakdown, the open-source dataset, and benchmark code below:
this collective celebration of the Outlook outage might be the best advertisement for AI replacing white collar work
we've spent decades turning millions of educated, ambitious ppl into glorified information processing machines
spending 10h/day answering emails, updating spreadsheets, preparing slides, forwarding information, chasing approvals, attending calls about other calls etc.
and somehow we're surprised they celebrate like it's Christmas when the whole system breaks
obv losing your salary is very different from getting an unexpected day off. But we're so obsessed with the idea of machines replacing humans that we forget how much of modern white collar work is already humans behaving like machines
the real dystopia was making humans spend 40 years doing work that machines should have been doing all along
This is absolutely brilliant, by legendary Australian investor John Hempton, who explains the core financial paradox at the heart of the AI story (and which also, to some extent, may explain the Jensen Huang story).
The numbers say that about $7 trillion in AI compute is set to be built by 2030. For the industry to earn that back, given those chips have a roughly 5-year lifespan and cost a lot to run, they need to generate about $2 trillion of AI revenue a year.
Hempton says the only way you can get to these sorts of numbers is by selling AI as a productivity tool to businesses, meaning replacing workers.
And since, for reasons he explains in the video, no company fires people to save just 10% in costs, the savings have to be about double what businesses pay for the AI: roughly $4 trillion in savings on wages a year.
What does firing $4 trillion worth of workers mean concretely? Hempton calculated this equates to roughly 10% of the OECD’s total wage bill of about $40 trillion: one in every ten dollars paid to workers across the OECD’s 38 member countries.
Which, naturally, when you suddenly take that much income out of people’s pockets, would trigger a massive global recession.
And here's the paradox: in a massive global recession, there's no way businesses will pay $2 trillion a year for AI, so the return disappears anyway.
In other words, AI's only possible path to success given the numbers requires destroying the very economy that is supposed to pay for it.
HOWEVER - and here's where the Nvidia/Jensen Huang story might come into play (and, to be clear, this is now me speaking and not Hempton given this interview is prior to the whole Nvidia saga) - this all rests on a core assumption: that compute stays super expensive. Which in practice means Nvidia keeping its pricing power, with a 75% gross margin on its chips.
Break Nvidia's pricing power, and the $7 trillion bill, the $2 trillion in revenue and the 10% of jobs all shrink with it.
This would also put the whole China story in a different light. Maybe it's not so much a "we don't want the Chinese to be competitive with similar chips to us" story as it is a "we don't want Chinese demand to drive up chip prices" one. Or probably both.
And, as such, you could actually think about the US export controls not as a national security measure but as a good old subsidy for US AI players (together with a handicap for their Chinese competitors), a way to limit the price of chips.
This holds even with China itself blocking Nvidia imports, since Chinese firms simply rent the chips abroad instead, in places like Southeast Asia, which is precisely what this whole Alice Huang/Megaspeed story is about.
The main obstacle for all this was, of course... Jensen. Obviously, the guy wanted the exact opposite: as many buyers as possible, Chinese players included, to keep chip prices and Nvidia’s margins sky-high. He was also the one man with the clout to make that happen.
All hard to prove, of course, but I can imagine that quite a few corks popped this week in Anthropic's offices: nothing like the news that the man keeping your biggest cost sky-high is out when you're a week away from a $1.2 trillion IPO.
Meet Gemini 4 Hydrogen: built for rapid coding, multimodal synthesis, and everyday workflows.
It finishes tasks 2.5x faster than 3.8 Flash and surpasses Argon on the @ArtificialAnlys Intelligence Index.
Microsoft's diagnostic AI solved 83% of 304 complex New England Journal of Medicine cases. Generalist physicians with reference materials solved 36%. We need to get this technology into clinics safely and fast. We need to stop making people wait for the right answers.
What if you could run a 125B AI agent on a normal 12 GB gaming GPU?
That is what Linger claims. It runs on your own computer at 45.5 tokens per second.
The trick is its PolyStrata engine. The most used parts of the model stay on the GPU. The rest wait in RAM or on the SSD.
What it can do:
- Search your files, mail, calendar and Slack
- Give you a finished task for review
- Keep everything local on your device
You can also connect it to Claude Code or Codex if you want.
Heads-up: It is still v0.1. Needs a lot of RAM. Only connect apps you trust.
Take away its tools, and Claude Fable 5.2 scores below Claude Haiku 5.5 with tools on VIALS-AA; across models tool use nearly doubles scores, from ~25% to ~47% on average
VIALS-AA is our implementation of @joinHandshakeAI's VIALS benchmark for visual reasoning in life sciences research. By default, models work as agents: they think, run code and view images in a sandbox. We reran VIALS-AA in a one-shot format: one request with the question and images, and no tools.
Lower cost models in an agent harness can beat single-shot large models on both cost and performance in visual reasoning and interpretation of scientific artifacts.
Claude Fable 5.2 (max) leads VIALS-AA at 67% with tools, at a cost of $4.20 per task; without tools it scores 44% at $0.73 per task. Claude Haiku 5.5 (max) scores 49% with tools, at $0.07 per task: higher than Fable without tools, at about a tenth of the cost.
Persistent Agents are the next unlock of AI usage
Our spend on AI is now 45% persistent agents
Unfortunately for Anthropic, Claude Tags sucks
OpenAI's Slack agent is even worse
We primarily use Perplexity Computer, which uses a lot of different models
Perplexity's Harness mogs