Jihoo Lee

SNU Math | CF IGM | IMO 2023 Silver | ICPC Seoul/Bangkok Regional Gold | prev. @Krafton_AI

[Anthony Bonato @Anthony_Bonato]

A student said they're not sure about completing their degree because AI will do everything in math and CS soon and that makes me sad

[Max For AI @MaxForAI]

在看了下DV4.1 Flash论文里的结构图然后和大佬们讨论之后，我觉得这次真正值得关注的，不是参数也不是跑分，而是DeepSeek又开始改Transformer本身了。

这次至少有三个很大的变化。

第一，又走回了Encoder + Decoder。

V4.1 Flash 上20层Causal Encoder + 20层Decoder。
当然，这不是回到传统 BERT/T5 那套，而是一个新的非对称结构。
Encoder负责读取、理解和压缩输入，输入侧只激活 8B；Decoder负责推理和生成，输出侧激活16B
本质上是在处理一件很合理、但过去大家很少思考的事：
读100万Token 和生成下一个Token，本来就不是同一种计算任务，为什么一定要用同一套结构？

第二，CSA 升级成了 CSA2，Attention 开始跨层复用。

CSA2 直接分成三种模式：
Full：KV和Top-K Index全部重新算。
Reindex：KV复用，只重新决定这一层该关注哪些 Token。
Reuse：连 KV和 Top-K Index 都一起复用，只更新当前层需要变化的部分。

模型变成了Full → Reuse → Reuse → Reindex → Reuse……
也许DeepSeek 的判断是，Hidden State 每层都在变化，但「哪些 Token 值得关注」没必要每一层都重新决定。
于是能缓存的缓存，能跨层复用的直接复用。

第三，V4缺少的Engram回来了。

首先Engram本质上是一套Conditional Memory。
一些固定搭配、实体、局部模式、静态知识，不一定非要让几十层神经网络每次重新算一遍，可以直接查 Memory。
所以现在V4.1的架构变得很有意思：
MoE：不是所有参数都要激活。
Sparse Attention：不是所有 Token 都要看。
CSA2：不是每一层都要重新决定看谁。
Engram：有些东西甚至根本不用算，直接记住。
再加上 Encoder / Decoder 非对称设计，V4.1 Flash 的路线其实已经非常统一了：
输入和输出不用一样算，不同层不用重复算，能直接记住的东西甚至不用算。

过去几年主流的LLM主要是在 Scale 同一种 Transformer。
但DeepSeek一直在思考Transformer里面，到底哪些计算是真的有必要存在？

有点不利好HBM啊 /doge

[Jihoo Lee @numberingai]

@oh_meed To be fair, this is pretty much the first time I've seen this kind of "credit drama," so it might just be me complaining about losing an illusion.

[Jihoo Lee @numberingai]

I cried looking at X today.

When a Millennium Problem got solved, I thought there'd be praise for the people who did it. Attempts to understand their work and explain it to the rest of us. Cheering, and that feeling of anticipation, because human knowledge had climbed one more step.

I thought that's what it would be like.

Ten thousand machines, and this ugly fighting. I never even imagined it..

Is this what all that damn "progress" was for? I'm just.. so, so sad.

[OpenAI @OpenAI]

We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.

The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.

The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.

[Jihoo Lee @numberingai]

@oh_meed I also always figured AI getting this far was a matter of when, not if. And the problem actually being solved, I saw as a good thing. What let me down was the way "people" acted about it...

[Ryan Orhan @rynorhn]

whatever happened to openai’s whole “working with academia” thing

if a researcher gets close to a breakthrough and your response is to burn millions in tokens just to beat them to it

how is that helping academia? you’re literally trying to beat them to their own breakthrough.

[Gary Marcus @GaryMarcus]

Still more absolute 🔥 from Terence Tao:

“We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field”

[OpenAI @OpenAI]

We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.

The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.

The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.

[Jihoo Lee @numberingai]

crazy that verifiability, the whole reason I fell in love with math and programming, is the exact reason my career prospects are gone

[Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) @teortaxesTex]

Astra made me realize that for some people, "not AGI" means "not an omnipotent, omniscient, infallible God".

Yeah, it can stall or fail.
What about you, carbonoid?

[さもカス @418814]

【AIに奪われない仕事ランキング】

1位 無職

[DADAS @dadas0812]

競技プログラミングサイト「DOJ」を運営しています！

問題を解いたり、コンテストに参加したり、レーティングを競ったりできるオンラインジャッジです。

DOJ:
https://t.co/Pa4Hu8tnyP
公式Discord:
https://t.co/DR617zXHQJ

興味があればぜひ遊んでみてください！
#競プロ #競技プログラミング

[Sam Altman @sama]

GPT-6 Astra is here.

We hope it will begin to enable a new generation of entrepreneurship, scientific discovery, and building.

We believe it is the best model in the world for computer use, professional work, science, coding, cybersecurity, and more.

It took us some extra time to ensure that we could meet the safety and alignment standards required for this capability level, but we think you’ll find it worth the wait.

It scores 98% on FrontierMath Tier 4, 99.9% on ARC-AGI 3, and 100% on ExploitBench.

[Sam Altman @sama]

we made a chip and it is fast

[Rabay Hasan @Rabay_Hasan]

Agree or disagree ☠️

[Geogristle @geogristle]

I do not understand how they are verifying these if the proofs are each 100+ pages. They post the result with an implied claim of absolute certainty and then just yeet a proof sketch pdf online after the fact without any comment. Makes me not even want to look at the proofs. OpenAI too

[Yu Xiang @YuXiang_IRVL]

My student told me that he didn’t code during his internship. All the code was written by AI lol.

We are really entering a different era of software engineering.

[Sean Geiger @sean_geiger]

When I worked at Apple, I remember seeing code that was 15 years old. Every file had a comment at the top with the date and the author’s name. You’d open a file and read through it, sometimes touching a feature no one had touched in a decade.

Everything felt very permanent, like a house made of bricks. Everyone who made a brick put their name on it before slotting it into the house. Even in a massive company, you were proud of your bricks. Each one fit so satisfyingly into the bricks around it.

When someone was renovating a room you had worked on years ago, they’d reach out and say, “I found this brick with your name on it. Can you tell me what it was doing here?”

Sometimes old bricks would get taken out during a renovation, but they were usually kept in a pile in the corner to reuse in the future.

Now you can will whole houses and neighborhoods into existence. You don’t even see the bricks go in. They just tumble in from the sky. You walk into the new house and wave your hands to push walls back, throw up a chimney, or drop in a kitchen.

It's so fucking cool. But you forget what a brick feels like.

And it’s hard to know how sad to be about that. The houses are better, they go up faster, and building them increasingly feels like magic.

Still, every once in a while you miss holding a brick.

[Inbok @inboklee]

IOI 2026 여섯 문제 중 두 문제가 우리나라에서 출제한 문제입니다.

[Colin Duffy @itcamefromthed]

@lacker Well math is almost always impossible until you figure out how to solve the problem, whereupon it is incredibly easy.