Ni Lao

scientist @googledeepmind, ex-scientist @Apple, ex-chief scientist @MosaixAI, PhD @LTIatCMU, geometry, NLU, Neuroscience, AGI, robotics

[Ni Lao @lao_ni]

after using coding AIs extensively this year I feel that RSI is probably not the right word, since agents still need to work in a verifiable environment with human Guidance. Maybe RGI is more appropriate

[Ni Lao @lao_ni]

look forward to more improvement of Gemini Robotics

[The Humanoid Hub @TheHumanoidHub]

Google DeepMind just introduced Gemini Robotics 2.

It's a single VLA that unlocks physical dexterity across different end effectors, hands or grippers, from one model checkpoint.

Apptronik's Apollo 2 humanoid, with the 22-DoF SharpaWave hand, ties knots and seals a ziplock bag

[Ni Lao @lao_ni]

But what is agi? It is probably more than context management

[Max For AI @MaxForAI]

‼️突发：DeepSeek @deepseek_ai 创始人梁文锋在一场据称长达四小时的投资者会议上的发言内容曝光。

过去一个月里，这场对话的片段一直在中国 AI 圈和投资人圈内流传。
最引人注目的是梁文锋反复强调的“不”字。
不搞“天才”神话。
不追求利润最大化。
不搞闭源。
不盲目争夺用户。
不做视频生成、3D、世界模型，也不做下一个“超级应用”。

他的解释很简单：
“克制是一种策略。放弃一些东西，是为了提高实现 AGI（通用人工智能）的概率。”

以下是会议中的核心观点：

- DeepSeek只有一条主线：AGI
梁文锋认为，现在不是追求产品收益最大化的时候。
核心是追逐人工智能最高峰，即AGI。
他的AGI路线图非常清晰。
去年的台阶是思维链，今年的台阶是Agent。
Agent之后，是持续学习。
持续学习之后，是AI自我迭代。
然后，AI开始帮助人类研发更加先进的AI系统。
最后才是具身智能。
梁文锋将这个过程称为“渐进的奇点”。
根据这个思考，DeepSeek目前正在主动放弃许多热门方向。
“很多东西不在我们的主线上，包括3D和视频生成。”
他也质疑了世界模型的重要性：
“世界模型跟智能的上限没有太大关系。”
多模态对产品和C端用户很重要，但它仍然只是一个组件。
“它不是主线，也不是智能本身。”

- 下一代模型必须拥有持续学习能力
人类可以在工作中不断学习。
但AI每次开始一个新任务，都需要人类重新提供所有相关上下文。
“这几乎是不可能的。”
这也是梁文锋认为，当前AI还无法真正替代员工的原因。
“下一代模型必须具备持续学习能力，否则就不能叫下一代模型。”
同时他表示智能的终点可能是具身。
他的逻辑很简单：
人类真正需要的，并不是另一个计算机界面。
人类需要的是劳动力。

- DeepSeek不追求利润最大化
梁文锋表示，DeepSeek不会以收入最大化为目标进行定价。
“我们只赚一个合理的利润。”
DeepSeek V4最初担心需求过多，所以价格定得比较高。
后来，DeepSeek把价格降到了四分之一。
“公司群里很多人都在欢呼。”
因为做出这个模型的目的，就是让更多人能够充分使用它。
备受关注的低成本不仅是一种商业策略。
“低成本是架构带来的结果。”
更低的成本可以让AI变得更加普惠。
但还有一个更深层的原因：
单位计算成本越低，DeepSeek就越能够在有限算力下训练更大的模型。
“大公司可以通过增加资源解决问题，我们优先考虑成本效率。”
有意思的是梁文锋并不觉得API生意有多么好。
“我不觉得卖API这个事情有那么大的吸引力。”
DeepSeek只需要一个很小的团队维护服务。
几乎没有客服，也不需要销售，用户自己就会来。
“我们一直在商业化，只是不以商业化为目标。”
他认为，DeepSeek彻底转向商业化的时间点仍然非常遥远。

- 开源是DeepSeek的战略甜蜜点
对于员工来说，开源能够带来成就感和组织凝聚力。
对于社会来说，它能够帮助研究者、企业和普通用户。
“开源是一种让利。”
梁文锋认为，AI最终会成为一个规模极其庞大的产业，任何试图垄断它的人，最终都会失败。
“如果AI最终占到人类社会GDP的10%，任何想要独占这部分利益的人，都会被历史抛弃。”
梁文锋表示，DeepSeek不会开源一个较弱的版本，同时在内部保留更强的模型。
他不担心竞争对手部署DeepSeek模型。
小型创业公司可能缺乏资源和意愿，去进行前沿模型研究。
大型公司可能拥有资源，却存在组织上的困难。
梁文锋认为，DeepSeek刚好处于一个罕见的中间位置。
“这是属于我们这个规模公司的sweet point。”

- 中美AI的差距主要在资源
梁文锋认为，中国并不真正缺乏人才。
DeepSeek希望改写中美AI竞争的叙事：
用几分之一的算力，把差距缩短到6个月，甚至3个月。
他作出这个判断的原因是他依然相信Scaling
“我们相信Scaling，规模越大，效果肯定越好。”
DeepSeek训练当前规模的模型，并不是因为这个规模已经足够。
而是因为现有算力只允许它做到这个规模。
梁文锋认为，前沿模型的竞争，最终会由三个因素决定：
成本、时间、用户体验。
“成本排在第一位。”
如果两家公司提供的模型质量相同，胜负将取决于谁能以更低的成本提供服务。
时间排在第二位。
早几个月和晚几个月，结果可能完全不同。
用户体验可以形成一定的黏性和壁垒，但梁文锋并不认为它是最本质的护城河。

- 团队稳定是梁文锋唯一不能让步的事情
他将其视为DeepSeek面临的最大风险之一。
原话是：“只有一个是没法退让的：必须保持团队的稳定性。”
而最近的一轮融资，已经大幅降低了这个风险。
“只要能够保持团队的稳定性，我一定能做成AGI。就这么简单。”
有趣的是DeepSeek的组织既是自上而下，也是自下而上
自上而下的部分，被梁文锋称为“做正事”。
但他不希望被安排的工作占用员工一半以上的时间。
另外一半时间应该保持自下而上。
研究人员可以自由探索他们认为重要的方向，不需要提前审批，也没有固定要求。
与很多AI团队不同，DeepSeek不鼓励过度加班。
“做研究需要一个相对松弛的环境。”
另一个原因是聚焦。
DeepSeek的许多产品并不完善，但公司会主动选择不去补齐所有问题。
梁文锋认为，这同样是一种克制。
有趣的是DeepSeek靠愿景驱动，而不是KPI
“要实现某个KPI，不是我们的方式。”
梁文锋表示，DeepSeek的愿景甚至不一定被正式写下来。
它存在于公司的做事方式里，也存在于公司对待世界的态度里。
“愿景不是挂在墙上的标语，不是怎么说，而是怎么做。”

梁文锋最后的警告，可能也是最重要的一句话
“如果你的愿景是拿得更多，你就已经输了。你可能会面临更大的困难。这个世界就是这样。”

DeepSeek不想赢下每一个AI市场。

它想保留实现AGI所需要的专注、成本优势、团队稳定性和组织自由。

而梁文锋似乎愿意放弃几乎所有其他东西，只为增加实现AGI的概率。

他在做真正很Cool的事情。

[The Humanoid Hub @TheHumanoidHub]

New Episode: I visited Proception in Mountain View, a humanoid hand startup founded by two ex-Tesla Optimus engineers.

Co-founder and CEO Jay Li (@JayLiStanford) showed me the office and the line producing hundreds of hands, then sat down to share his story, the technology approach, and where he thinks the future headed.

0:20 Jay Introduces Proception hand
1:44 Data collection with the glove
2:49 Office and production line
5:32 Testing lab and sensor printing
6:53 Evolution tactile sensors
9:13 From Apple to Tesla Optimus to here
10:33 Why tendons over direct drive
11:38 Why match the human hand
13:33 Use cases for 5-finger hands
16:16 Tendon reliability and power needs
20:03 Sim-to-real challenge
23:03 Tidbit from a hand surgeon
24:49 Why so bullish on humanoids
28:30 Price for a pair of hands

[Ruben @rdominguezibar]

Kimi's founder was flagged as a generational talent 12 years before K3 shocked the world 🔥

In 2014, at his Tsinghua scholarship defense, his mentor introduced him as "the most outstanding and talented student I've seen in years"

The résumé at the time:
▫️ 95+ in nearly every course, full marks in all programming classes, #1 in his year
▫️ 3 first-author papers at KDD, WSDM and CIKM in a single year
▫️ Algorithms adopted by Tencent, Sina and Huawei
▫️ Beat Stanford and Columbia to win a global cancer-prediction contest
▫️ Founded a campus band on the side, as drummer and songwriter

The overnight success was 12 years in the making

[Jim Fan @DrJimFan]

We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 sec), instantly forgetting what just happened. We pushed to 3 orders of magnitude beyond SOTA.

Introducing RoboTTT. Test-Time Training (“TTT”) carries a tiny model *inside* the model. Every incoming sensor reading triggers one gradient step on that tiny core, so the history keeps getting compressed into its weights. The hidden state has a fixed size (literally a small neural net), so the robot can “grok” arbitrarily long experience with little overhead. Learning continues indefinitely after deployment.

We can then put an entire video in context as prompt! RoboTTT enables one-shot in-context learning from human video: in circuit board assembly, a human demonstrates a never-seen configuration once, and the robot imitates it faithfully.

Humans drop things all the time, but we pick them up so fast that we don’t even notice. That reflex to fix is half of our physical competence. RoboTTT shows self-improvement on the fly: the robot is skilled at recovering from its own errors mid-episode, and each fix enters its context to inform the next move. The TTT core distills a general-purpose, failure-to-correction mapping from the training data.

One more thing. What excites me the most is a new Context Scaling Curve: from 128 to 8K timesteps, closed-loop performance hill-climbs steadily with no sign of saturation. 8K-context pretraining beats 1K by 62%. What LLM enjoys, robotics should too. Soon, even 1M context is not a fantasy.

Deep dive in thread:

[The Humanoid Hub @TheHumanoidHub]

The engineers were allowed to cook.

[Ni Lao @lao_ni]

AI does make it a great time to calculate the "Ultimate Question to Life, the Universe, and Everything". Created this ancient philosophy website with just a few prompts
https://t.co/y14gAtjDbt

[Kaiyue Wen @wen_kaiyue]

The agent did something I never managed to do (despite trying multiple times with @xingyudang): combining cautious weight decay with weight norm controlled update. The trick is to apply it after you scale the update norm.

[Ni Lao @lao_ni]

A fascinating discussion about Google I/O from The Verge. Sounds like Google can head to a universal operating system that contains everything on the internet and is developed by billions of everyday users. https://t.co/oILnz9BzVZ via @YouTube

[Ni Lao @lao_ni]

skating better than me already.

[The Humanoid Hub @TheHumanoidHub]

The SOTA in humanoid locomotion is already insanely good. The control stack is almost fully leveraging what the hardware can do.

For real-world use, the active safety is still remain unsolved: Common-sense intelligence to move safely around people, and graceful handling of possible hardware failures in the wild.

[Ni Lao @lao_ni]

Modeling spatial relationship is a fascinating and increasingly important problem. Many thanks and congratulations to my co-authors. Zeping Liu, @ZhangyuWangUCSB, @junfengjiao, @mai_gch
https://t.co/kIsmfMhirm

[Chen Liang @crazydonkey200]

@karpathy Very inspiring as always! We are also open sourcing part of our infra on automated research for Gemini to evolve itself at https://t.co/WH7JBEEm9h More complex than the nanochat setup but closer to SOTA LLM pre/post-training while staying as minimal as possible. More on the way.

[Ni Lao @lao_ni]

similarly if all the softwares are free then ideas and theories are more important.

[Ni Lao @lao_ni]

Random thought about AGI -- If AGI will be good at doing everything, we will still need many humans to express their diverse needs, verify the result of AGI, and, more importantly, understand and improve the AGI behavior if possible.

[Larry Dial @classiclarryd]

New NanoGPT Speedrun WR at 99.3s (-5.6s) with a bigram hash embedding that is added to the residual stream before every layer. Inspiration from Svenstrup et al 2017 paper on Hash Embeddings, and Deepseek's Engram. Modded-NanoGPT now uses fewer training tokens than its parameter count, a radical divergence from the 20x Chinchilla ratio. https://t.co/8hQdx3dX0c

[Michael Truell @mntruell]

Watch Cursor build a 3M+ line browser in a week

[Ni Lao @lao_ni]

This is a very beautiful work, but one question remains -- what is the optimal prior/regulation for learning?

[John Carmack @ID_AA_Carmack]

#PaperADay 7
Cautious Weight Decay
https://t.co/EzgZbK4WRJ

This is a 36 page paper about a very simple idea:
Don’t apply weight decay when it is in opposition to the current optimizer step.

If the step is moving the weight farther from zero, there is no decay. If the step is towards zero, decay moves it in faster.

They spent 20,000 H100 GPU hours (about $60k!) testing this across multiple optimizers and models, and it looks like it is basically always a modest improvement, with no changes to any hyperparameters.

My current models use weight norm on most of the parameters, but there are still some with traditional weight decay. A first test with this idea does seem to be a tiny improvement, but I will need to do more runs to have confidence in it.

Two modifications of the idea come to mind:

Use the current gradient instead of the optimizer step (which includes momentum) for masking, as in https://t.co/Q7yiRAyw19. If using a cautious optimizer, explicitly masking the optimizer step before calculating cautious weight decay would do this automatically.

Weight decay is an exponential effect, which mixes with a linear learning rate. It might be interesting to just have a larger learning rate when the step is heading towards zero than away from it, which would do similar things, but with different learning dynamics.