Yuanjing Shi@NeurIPS2025

post-training at Apple. Ex-Nvidia/Snowflake. Opinions are my own.

[Yuanjing Shi@NeurIPS2025 @shingjan_]

If github sells consoles I will snap one in no time

[GitHub @github]

We heard you. And we agree.

In light of recent developments in physical media, GitHub is proud to announce that you can now obtain your public repo on CD-ROM.

Keep it. Lend it to friends. Pass it on to your children.

Your code is physically yours, forever. Until you lose it, let's be real.

Order yours today.
https://t.co/z041pdMH7h

[Yuanjing Shi@NeurIPS2025 @shingjan_]

One can also see the other way around. Frontier labs got their own chips and profit margins grow exponentially. A lot more chip makers than frontier labs

[the tiny corp @__tinygrad__]

In the future, frontier models will mostly all be given away by the chip and hardware makers. It makes their hardware more useful and drives sales. We will do it if others don't.

[Yuanjing Shi@NeurIPS2025 @shingjan_]

> if you're smart enough you can make anything look like progress in the meantime

Real progress is often hard to quantify. And not everything quantifiable is real progress.

[Susan Zhang @suchenzang]

[episode 123 of frontier lab gossip: OH on X dms]

> yo did you hear about the MSL packages and lmsys scores?
> apparently folks have $XXX mil in comp tied to however many weeks they can get their models to stay at #1 on lmsys
> they seem to still think that's a flex
> when its obvious that you can hack it

>> ...don't they remember llama4 drama?

> no. they got rid of everyone who touched that model.
> either fired, moved to FAIR, or dispersed to random parts of the co
> as a soft layoff until they quit

>> surely the new MSL folks can push back against gaming lmsys?

> why would they
> they're being led by someone who doesn't understand that training on test is bad
> their legacy at [[ previous co ]] was to get ICs to synthetically benchmaxx for impacc
> everyone else just planning a 2 week/3 month/6 month exit
> going through the motions before retirement
> if you're smart enough you can make anything look like progress in the meantime
> esp to people who have no taste beyond lmsys

[Yuanjing Shi@NeurIPS2025 @shingjan_]

There is no secret sauce. You just try, fail and try again until something works

[Kabir @kabir_j25]

question for researchers/eng at ai labs:

how do you validate a new architecture before scaling it to billions/trillions of parameters?

what signals at small scale give confidence that a full training run is worth the compute?

papers/blogs appreciated

[Yuanjing Shi@NeurIPS2025 @shingjan_]

mps just for love of the game

[the tiny corp @__tinygrad__]

Today, if you were writing a bunch of kernels, what would you reach for? Raw CUDA? tile-lang? Triton? ThunderKittens?

[Yuanjing Shi@NeurIPS2025 @shingjan_]

Anything that helps boosting their IPO, one would presume. What’s the PBC part about then?

[Nathan Lambert @natolambert]

I don't really want to have to go to bat against Anthropic, but they've just been unnecessarily antagonistic to all of China, then not so subtly to open weight models, and now more broadly open AI research. What's next on the list?

[Yuanjing Shi@NeurIPS2025 @shingjan_]

From a first principle perspective, model itself doesn’t need all the context to work the coding agent magic. It just tries to predict the next token. All the context, session history, long term memory can be stored in a model provider agnostic way. The really moat is the prompt

[Percy Liang @percyliang]

I stopped using ChatGPT a few months ago. Since then, I have been only using oa-chat. All chat history is stored locally. Each query is sent to OpenAI under a temporary key which is unlinkable to any other query. I’m not a privacy nut, but oa-chat is such a convenient drop-in replacement for your favorite AI assistant that there’s no reason not to try it out.

[Yuanjing Shi@NeurIPS2025 @shingjan_]

The new tinytorch incoming

[Andrej Karpathy @karpathy]

New art project.
Train and inference GPT in 243 lines of pure, dependency-free Python. This is the *full* algorithmic content of what is needed. Everything else is just for efficiency. I cannot simplify this any further.
https://t.co/HmiRrQugnP

[Yuanjing Shi@NeurIPS2025 @shingjan_]

Can’t wait to see some of that mHc and Engram magic working together. Pretraining is not dead yet

[Lin Qiao @lqiao]

Excited about Deepseek V4. Likely to be after lunar new year.

[Yuanjing Shi@NeurIPS2025 @shingjan_]

@typedfemale Why anthropic tho?

[Yuanjing Shi@NeurIPS2025 @shingjan_]

𝗛𝗼𝘁 𝗧𝗮𝗸𝗲:

What if intelligence is mostly compression?

Not scale. Not data. Compression.

From that view, attention and sparsity aren’t optimizations—they’re the point.

What does that imply for future architectures?
👉: https://t.co/lzDCsOwEzP

[tetsuo.mlir @tetsuo_cpp]

Oh, you're writing CUDA kernels? Everyone's on Triton now. Just kidding, we're all on Mojo. We're using cuTile. We're using ROCm. We have an in-house DSL compiler targeting the NVGPU MLIR dialect but wait, Tile IR just dropped so we're going to target that instead. Our PM is on TileLang. The team lead was on CuTe but now she's back to handwriting PTX. If you're not on Pallas, you're ngmi. Our intern is building on TT-Metalium for our Wormholes. Our CFO approved an order for some big chungus wafer-scale chips so now we're porting our kernels to CSL. Our CTO is working on a kernel-less graph compiler so we won't need to write kernels anymore. Our CEO thinks we're talking about the Linux kernel. We're building Claude for dogs.

[Yuanjing Shi@NeurIPS2025 @shingjan_]

Scale was split three-way with the skeleton staff and small investors hanging to dry. The well is poisoned indeed

[Aakash Gupta @aakashgupta]

Big Tech just paid ~$40 billion in two years to avoid buying anything.

The math is absurd. Google spent $2.4B on Windsurf to hire 40 people. That works out to $60M per head. Microsoft paid $650M to gut Inflection and take 70 employees. Amazon spent $400M+ on Covariant for three founders and 40 engineers. Google dropped $2.7B on Character AI to rehire Noam Shazeer, who they’d let walk in 2021. Now Nvidia announces $20B for Groq just three months after it raised at $6.9B.

Every single one of these companies explicitly stated “we are not acquiring this company.” Jensen Huang literally told employees: “While we are adding talented employees to our ranks and licensing Groq’s IP, we are not acquiring Groq as a company.” Microsoft said the same about Inflection. Google said the same about Character AI and Windsurf. Amazon said the same about Adept and Covariant.

The semantic gymnastics exist for one reason: antitrust.

Traditional acquisitions trigger Hart-Scott-Rodino filing requirements. Regulators review. Competitors object. Deals take 12-18 months to close. In an AI arms race where model capabilities improve every 6 months, that regulatory timeline is existential. By the time a deal clears, the tech is already outdated.

So Big Tech invented the “reverse acquihire.” Pay billions to license IP, hire the founding team, leave a shell company behind with a new CEO and a skeleton crew. Google did it with Character AI (Noam Shazeer + 30 researchers, left behind a co-op structure). Microsoft with Inflection (Mustafa Suleyman + 70 staff, left Sean White as CEO of nothing). Amazon twice with Adept (David Luan + research team) and Covariant (three co-founders + 25% of staff). Now Nvidia with Groq (Jonathan Ross + senior leadership, Simon Edwards inherits a cloud business).

A whistleblower complaint filed with the FTC, DOJ, and SEC in January 2025 alleged that the Amazon-Covariant deal was “deliberately and unlawfully structured” to dodge antitrust review. The complaint claimed Covariant’s new CEO told employees that if Amazon had tried to buy them outright, regulators would have killed it. The deal terms reportedly restrict which licenses Covariant can sell without paying Amazon a fee.

The FTC opened investigations into Microsoft-Inflection and Amazon-Adept. Both appear to be at a standstill. Amazon’s Adept deal closed without further action.

The exposed logic: buying a company twice (once for talent, once for the husk) now costs less than waiting for regulatory approval of a single acquisition. Windsurf got split three ways in 72 hours. Google paid $2.4B for leadership and license. Cognition paid ~$250M for what remained. OpenAI walked away with nothing after Microsoft objected to IP terms.

Big Tech found a loophole wide enough to drive $40 billion through while regulators debate whether hiring someone’s entire executive team and licensing all their IP counts as “control.”

[Yuanjing Shi@NeurIPS2025 @shingjan_]

@zhuohan123 @giffmana Why the switch? Jax still uses xla under the hood right?

[Yuanjing Shi@NeurIPS2025 @shingjan_]

@LiyuanLucas @MSFTResearch @thinkymachines Would love to catch!

[Yuanjing Shi@NeurIPS2025 @shingjan_]

@awnihannun Honestly model compilation feels so 2022. Eager mode with fast kernel makes so much sense now

[Yuanjing Shi@NeurIPS2025 @shingjan_]

Just got the tinker invite from @thinkymachines let’s see what we can cook RL with it

[Yuanjing Shi@NeurIPS2025 @shingjan_]

Would be in San Diego Dec 1 to 6 for NeurIPS. Cannot wait to meet and catch up with folks!

[Yuanjing Shi@NeurIPS2025 @shingjan_]

@lunwang1996 Hi Lun, would be there for the week. Nice to catch up!

[Yuanjing Shi@NeurIPS2025 @shingjan_]

@tydsh See u there and love to catch up!