vlado

having fun

[vlado @jawdawoe]

@doomslide Do you really believe autonomous intelligence won’t have milllitary uses?

[will depue @willdepue]

That means I can take a published model, pick a chip and a seed, generate what the starting weights would have been and compare them to the real ones. With the right chip they line up and with the wrong chip they don’t.

This is that search on DeepSeek-V4-Flash. Every processor count gives noise until 148 (Blackwell), where the weights match.

[vlado @jawdawoe]

Think this makes-up for his doomerism earlier. Very interesting catch!

[will depue @willdepue]

I have a major result to share.

I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China.

This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model.

Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves.

I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems.

I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used.

It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20.

These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs.

I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at https://t.co/YOtzND0Jqk.

There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.

[vlado @jawdawoe]

Kiln

[Nikolas Huebecker @nhuebecker]

gun pointed to your head: name a 10/10 sf restaurant

[doomslide @doomslide]

Straight up evil AI alignment is probably much safer than whatever convoluted contradiction-packed corporate-speak BPD-brain narcissistic personality training Anthropic is postraining Claude with these days.

[vlado @jawdawoe]

@adhidesign Would you consider doing an eng blog on the design?
It is conceptually very different than either of your major competitors

[vlado @jawdawoe]

@AndyMasley "1) Humans don't have plot armor." says you?

[vlado @jawdawoe]

@willdepue @niloofar_mire @OpenAI so what? Is our superiority to be determined by whether a machine is smarter than us?

[vlado @jawdawoe]

@oscarmoxon Yeah, it's a very low self-esteem perspective. A tad like saying the car is gonna be the end of human athleticism.

[vlado @jawdawoe]

@PAHoyeck A famous singer under a brutal conservatorship I think

[vlado @jawdawoe]

Anthropic models give massive HAL-9000 vibes

[vlado @jawdawoe]

I love how excited @GeminiApp Carbon gets when it makes an advance!

[vlado @jawdawoe]

Strengthened OpenAI's problem N. 029 to hold unconditionally for arbitrary finite sets:
https://t.co/6mSUsCWjkQ

[Doug Colkitt @0xdoug]

1/ Our long national nightmare is over. The wall has been smashed. New community checkpoint for integer multiplication:

κ = 0.00268647 ≈ 2⁻⁸·⁵⁴⁰
T(n) = O(n(log n)⁰·⁹⁹⁷³¹³⁵³)

A 66% improvement since last announcement. A 2^173 fold improvement from the OAI result

[vlado @jawdawoe]

@SIGKITTEN Has OAI finally wisened up and migrated to Java ;) ?

[vlado @jawdawoe]

Michael Tracy has never seen a rape he didn’t support

[Michael Tracey @mtracey]

There's just no serious doubt that the "rape" prosecution of Russell Brand, for which he has now been convicted and jailed, was first precipitated by his very public turn toward vaguely right-wing (and admittedly often wacky) political and religious commentary.

In 2022, the Times (UK) published a big feature lambasting him for his purported transformation into some sort of dodgy right-wing "conspiracy guru."

Then in 2023 comes the big expose about Brand's allegedly sordid history of "sexual assault" and "abuse." The Times claimed they first started hearing of such allegations in 2019, but apparently had little urgency to follow up for several years.

Then all of the sudden, in "autumn 2022, Channel 4’s Dispatches formally began collaborating with The Sunday Times to investigate Brand."

Another notable event that occurred in "autumn 2022": Russell Brand was announced as a featured broadcaster on the right-wing (ish) video platform Rumble, alongside the likes of Donald Trump Jr. and Dan Bongino.

Soon enough, boom — here comes the newly frenzied "investigative" activity by the joint Times/Channel 4 task force — eventually yielding the big splashy 2023 "documentary" (more like "slopumentary”) which today they are bragging "led to" Brand's conviction in London for various sexual assault and rape offenses.

(He was also acquitted on multiple counts, in a very curious mixed verdict. NONE of the allegations aired in that 2023 propagandocumentary were ultimately charged at trial.)

Really makes U think...

[vlado @jawdawoe]

@thsottiaux @ohayou792 As if StuckKey isn’t funny enough already!

[vlado @jawdawoe]

“My AI is so dumb it wont help me write science fiction. Here’s why that’s a good thing”

[Kelsey Piper @KelseyTuoc]

I wish Anthropic would remain just as neurotic as they are about bioweapons (they're obnoxiously, outrageously neurotic about it; last night my Claude got interrupted and downgraded while discussing a work of science fiction set in the 1890s that contemplated cholera as a weapon of war) and become absolutely zero neurotic about everything else. The bioweapons stuff matters. If the company were only neurotic about that and recursive-self-improvement, it would convey that that's a different ballgame.

If you /also/ have your AI flip out over making video games or using non-PC language or helping me do election research, then it just looks like you are extremely neurotic in general, which diminishes your credibility on the serious stuff.

[vlado @jawdawoe]

@imjustnewatai you're retarded