Junjie Chen

PhD. love structure discovery for languages. On mental recovery.

[Junjie Chen @harpyyseal]

And most of the time the difference is that people use a more distant signal than direct distillation

[Dimitris Papailiopoulos @DimitrisPapail]

BTW this is the reason why i find it interesting when I hear claims "we did not distill". Everyone indirectly distills whether one likes it or not, unless you train on the common crawl of 2022 and use data vendors that don't use any AI tech.

[Junjie Chen @harpyyseal]

It sounds like a way to use LLM as a zero-shot machine (just generate the weight) without paying the hefty cost of running the LLM on every instance

[Jacob Lauritzen @jacsebl]

Got fable to hand-write weights of a 100k param model to recognize real versus fake legal citations.

Basically one-shotted a full NN that’s better than training it

[Junjie Chen @harpyyseal]

That’s my feeling. Skills to do signal processing or machine learning won’t buy much, as AI often do them better. But the understanding of the signal or model do. It’s just that with AI we can now do the signal processing without deep understanding, unlike in the old time.

[Addy Osmani @addyosmani]

https://t.co/2gyp2Erpxx

[Junjie Chen @harpyyseal]

@WHMaoXian 问题是人们发现只有astra/opus5.5/gemini argon 有这个能力，然后这两个模型又刚好是rsi开始炒起来的时候。不得不令人怀疑openai/anthropic/Google究竟在用什么任务训练它们

[Karim Abdel Sadek @Karim_abdelll]

I am not gonna comment on the results per se, but if you believe your work is valuable and insightful, why not tune the baselines?

This took in total about 45 min of training on an A6000.

[Junjie Chen @harpyyseal]

@korymath I have a feeling that multiagent will end up in a boot similar to sensor fusion. Yes it’s very powerful when agents/sensors have complementary information and we can easily identify which to trust. But once the signal and noise are not that clear, they often ended up worse.

[Jessica Ruan @jessicawruan]

"ETH is false" situation be like 😛

Mathematicians: 3-SAT is slow because you’ll always have to brute-force at least a fixed fraction of the switches, forever
OpenAI, with assistance of ChatGPT: ackshually, you’re wrong because the fraction shrinks!!
Mathematicians: :O :O wdym

[Junjie Chen @harpyyseal]

@quxiaoyin Not really. AI is quite good at coding and doing experiments, but they are terrible at telling us what is good for a somewhat ambiguous requirements. Senior MLE will hardly go away

[Longxuan Yu @Loy004Yu]

Hot take: the community keeps asking whether diffusion LLMs can beat AR. That's the wrong question. Diffusion is a sampling method, and it can sit on top of an AR model to make it faster instead of replacing it.

Speculative decoding already speeds up AR, but it has two long-standing problems. Training a drafter takes a long time, and you have to build a complete pretraining corpus for it. Even then, token acceptance drops sharply on out-of-domain data.

Uno @ssahoo_ offers an elegant way out: it decouples sampling training from capability training. There's no separate drafter. The model itself, plus one LoRA, drafts several tokens in a single forward pass, and the LoRA learns to match the model's own next-token distribution, so there's no drafter corpus to build and no labels to collect.

We applied this to the Looped Transformer Nanbeige4.2-3B: 1 hour of LoRA training gives a lossless 2× speedup on agentic task. We haven't optimized anything yet, and we believe the acceleration can go much further.

The takeaway: diffusion doesn't need to beat AR. Any model that can score its next token can get a fast diffusion mode in an hour.

Page: https://t.co/SCkbkrtf2z
Lora Adapters: https://t.co/Awks2jdkx0

[Andrew Ho @andrewho03]

@alexolegimas Jumping straight to drug discovery, one of the least verifiable fields in existence, makes no sense to me. Like, the models can barely serve as a competent accountant/lawyer, isn't drug discovery 1,000,000x harder?

[けいひぐ @keihigu]

そしてこれは去年の12月に書いた壁の話

[Talia Ringer 🕊🪬 @TaliaRinger]

It's weird how right now RSI has a sci-fi meaning (somehow magically getting better at everything always in a way that leads to "intelligence explosion") and also a concrete research meaning (self-modifying harnesses or successor-training models tested on held-out benchmarks)

[Max Lv @m0d8ye]

Linux 核心维护者 Greg Kroah-Hartman（GKH）在 Kernel Recipes 2026 上谈 LLM 与内核安全：

所谓Mythos发现 79 个内核 Bug 的大新闻，实际看很多是假报警、编造数据或已修复问题，真正有效的补丁仅抵得上社区 1 小时的工作量。• LLM 生成的补丁约有 50% 是错的
视频链接：

GKH 认为 LLM 某种程度上只是在做 fuzzy patern match。

另外他强调： GKH 开发者“Run local models”（只跑本地模型），并且用非常严厉的语气警告“Never, ever, ever upload anything that's non-public”

[Junjie Chen @harpyyseal]

@yvbbrjdr 超级+1

[Kory Mathewson @korymath]

Think
Outside
The
Prompt

[Junjie Chen @harpyyseal]

@Jaeyeon_Kim_0 @sitanch @du_yilun amazing

[Junjie Chen @harpyyseal]

@yoavgo A lower acceptance threshold is just an invitation to more paper flooding

[Junjie Chen @harpyyseal]

个人感受：claude opus 5.5有很强的prior，在prototyping的时候很好用，但在写工程代码的时候经常丢三落四。gpt与claude相反，对细节的recall很强，但没有大局观。不知道是不是产品策略

[Junjie Chen @harpyyseal]

Won’t this go down a very bad path that destroys the double blind systems? The reason that we went double blind is to protect people from the correlation that senior researchers are more likely to get their papers through?

[Michael Trinh @_mctrinh]

AI researchers are destroying themselves.

[Guitar Cat + LLM 「Will @ SGLang Summit in SF」 @GenAI_is_real]

我自己就在一家发展非常快、而且我非常看好的 startup 工作。

但即便如此，我依然认为：对于刚毕业的人来说，加入 startup 是一个风险相当高的职业选择。

不是因为 startup 不好。

一家真正优秀的 startup，完全可能把几年的成长压缩到几个月。你可以真正 own 一个问题，直接面对用户，做远高于自己 nominal level 的决策，而且每天都能看到自己的工作怎样影响公司。

但很多人对它的 downside 理解得太简单了。

最显眼的风险当然是公司失败，equity 归零。

但对职业早期的人来说，我觉得更重要的风险是：

你的学习环境可能和公司一起失败。

工作的最初几年，很大程度上是在建立你对于 good engineering、good management、good product judgment 和 good execution 的内部标准。

在成熟公司里，这些系统虽然绝不完美，但通常已经存在：review process、senior engineer、历史经验、明确 ownership、客户、metrics，以及见过类似问题的人。

而在非常早期的 startup，你自己还在学习，整个 organization 也在同时学习应该怎么运转。

如果周围的人非常优秀，这会是一种惊人的成长环境。

但反过来也可能很危险。

公司可以把“没人有时间带你”包装成 ownership。

你可以直接向 founder 汇报，却几乎得不到有质量的反馈。

你可以同时碰 strategy、product、sales 和 engineering，感觉自己每天都成长飞快，但最后没有任何一个方向真正做到优秀。

你也可能因为整个团队只有你知道某个混乱的流程怎么跑，而变得“不可替代”。

这证明你对公司很重要，但不一定证明你的能力真的变强了。

而且在变化极快的环境里，人尤其容易把“我已经习惯了混乱”误认为“我已经学会了很多”。

我见过不少朋友真实地在这种状态里挣扎。

他们非常聪明，加入了看起来非常 exciting 的公司，每天工作极其努力，一个人顶五个人用。

可一两年以后，如果问一个很简单的问题：

相比入职的时候，你现在究竟在哪件事情上强了很多？

答案反而并不清楚。

所以如果是 new grad，我判断一家 startup 时，不会首先看 founder 有多耀眼，而会看它的 learning loop 到底好不好：

谁会 review 我的工作？

谁能够指出我的判断为什么错了？

我有没有机会真正 end-to-end own 一件事情？

我能不能看到真实用户和真实结果？

这个团队能不能告诉我，入职 3 个月、6 个月、12 个月之后，什么叫做得好？

如果这家公司两年后不存在了，我到底能带走什么能力、判断和可以证明自己做过事情的成果？

以及一个非常现实的问题：

这家公司到底有没有足够的 runway，让我真的待到能够学会这些东西？

我认为最好的 startup 工作有一个非常宝贵的特点：

action 和 consequence 之间的距离特别短。

你做一个东西，ship 出去，它坏了；你和用户聊，发现自己理解错了；回来修，再看结果。

如此反复，你会非常快地形成 judgment。

这是 startup 能够��供的巨大��值。

但如果缺少好的 mentorship、明确 ownership 和真实 feedback，同样的“快”，也可能只是在更快地产生噪声。

所以我依然非常看好 startup。

但当有人告诉应届生：

年轻就应该去赌一把。

我会非常谨慎。

风险本身不是回报。

只有当风险能够换来某种东西时，它才值得承担。

而对于职业生涯刚刚开始的人，最重要的回报可能既不是 title，也不是离 founder 有多近，甚至不一定是 equity。

而是：

即使两年以后这家公司消失了，你依然已经在某件重要的事情上，变成了一个明显更强的人。