Rui

时空大数据博士生，专注 AI for Science。用大白话分享 AI 前沿、科研方法与实测笔记｜独立开发者

[Rui @Ruixue_forai]

https://t.co/MfQ5Lwfms5

[Rui @Ruixue_forai]

为什么AI天气预报总爱下毛毛雨？因为评分规则在逼它。预报位置偏一点点会被罚两次，报了没下、下没报，学界叫 double penalty，模型学聪明了，干脆把雨报得又弱又散，分数反而最高

[Rui @Ruixue_forai]

https://t.co/fIqlWQ7X44

[Rui @Ruixue_forai]

@ylecun @haider1 working with spatiotemporal data I feel this in my bones, our models crush static benchmarks yet lose track of a moving scene within minutes, the missing piece is not more text, it is a memory of how the world keeps changing

[Rui @Ruixue_forai]

最扎心的其实是那句20小时学会开车，我天天跟时空数据打交道太懂了，真实世界的数据从来不按分布来，传感器一漂移路况一变前面学的全得推倒，刷题刷得再猛也刷不出这种应变

[Yann LeCun @ylecun]

That video is 2 years old.
Two years later, we are still far from human-level AI.

Sure, AI has superhuman performance in a number of tasks (particularly in mathematics and coding and answering questions with known answers).

But where is my Level-5 self-driving car?
(Before you ask, Tesla FSD is rated Level 2, Waymo is Level 4).

Where is the self-driving car that can teach itself to drive in 20 hours of practice like any 17 year old?
(Before you ask, we have billions of hours of training data and still can't successfully train a self-driving system by imitation learning)

Where is my domestic robot?
Where is the robot that can do what any 8 year-old child can do?
Where is the robot that can learn a new task as quickly as an 8 year old?

AI still has a hard time with the complexity and messiness of the physical world.

[Rui @Ruixue_forai]

把视频倒着放，再问AI什么先发生，答案居然经常不变。10月1号的新论文把帧顺序倒过来测了三个视频大模型，很多时候模型答得纹丝不动，等于承认它根本没在看顺序，是在靠单帧和常识蒙

[Rui @Ruixue_forai]

https://t.co/psW486OOJl

[Rui @Ruixue_forai]

@GKev1n 最怕的其实是中间那步引理，看着特别顺，顺到你都懒得查，等审稿人一问才发现整段是模型自己脑补的，这种标注早该成标配了

[Rui @Ruixue_forai]

这句可能是今年对推理模型最准的一刀，以前是直接猜答案，现在是先猜出能生成答案的程序，所以 test-time compute 才砸得下去，也难怪纯刷题数据的收益掉得这么快

[François Chollet @fchollet]

The critical distinction between base LLMs (2024 and earlier) and modern LRMs is not symbolic tool use. It's the switch from a transductive paradigm (intuit the answer to the query) to an inductive paradigm (intuit the program/instructions that produce the answer to the query).

They're trained to be inductive, and they perform test-time induction, i.e. test-time prediction of a NL program / reasoning chain. This unlocks entirely new capabilities -- in particular fluid intelligence. Base LLMs, to this day, have ~0 fluid intelligence. LRMs have substantial levels of fluid intelligence.

The performance of LLMs on ARC 1 (a benchmark from 2019) remains ~10-15% today. Scaling them up by a factor ~100,000x got them from 0% to 10%. Meanwhile LRMs the same size or smaller saturated ARC 1 in 2025.

[Rui @Ruixue_forai]

Chollet 有个区分我一直记着，智能是归纳，不是转导。转导是把见过的东西换个样子再认出来，归纳是从几个例子里抽出一条能管新情况的规则。现在的大模型强在前者，真正的考验是没人教过的题

[Rui @Ruixue_forai]

Karpathy这条最值得存的是顺序，文字只是最低带宽的理解方式，我给学生讲模型结果已经先让LLM出交互网页再看原文，哪里分布漂移一眼就露出来，视频适合最后定稿

[Andrej Karpathy @karpathy]

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:

Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:

Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:

Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:

Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!

In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

[Rui @Ruixue_forai]

@xiaomovps 拆层这个对做科研海报太关键了，我之前用生图做会议海报最崩的就是改一个公式整张图得重画，文字主体背景能分开的话至少能局部重跑，这才是真能进工作流的点

[Rui @Ruixue_forai]

最扎心的其实不是砍到64G，是带宽没动，273对1792差6倍，单路生成拼的就是带宽，这么一砍128G版塞大模型的唯一理由也没了，本地跑Agent先卡在吐字速度上

[实践哥 Li @MinLiBuilds]

🤔老黄这次是真交了个垃圾产品。

DGX Spark 64GB 版来了。

很多人可能没概念：

64GB 统一内存，到底砍掉了什么？

普通 PC 是两套内存。

但 DGX Spark 不一样。

CPU 和 GPU 共用一整个内存池。

所以原来 128GB Spark 最大的价值就是塞大模型

但它还有一个短板：

内存带宽只有 273 GB/s。

RTX 5090 是：

1792 GB/s。

差了 6 倍多。

而跑 LLM，尤其单路生成的时候，
内存带宽影响速度

老黄把128GB 大内存，砍成了 64GB。

真变成桌面端玩具了 😂

[Rui @Ruixue_forai]

@GKev1n 这个排队延迟太真实了，我跑长上下文评测也是，短 prompt 看着挺能打，一拉到几万 token 首字延迟直接翻几倍，Agent 连着调十几轮工具那个等待才是真成本，本地这账真得按整条链路算

[Rui @Ruixue_forai]

@wellzhiai M0IOIR

[Rui @Ruixue_forai]

https://t.co/g4zGQFOX8f

[Rui @Ruixue_forai]

@blyen46 M0IOIR

[Rui @Ruixue_forai]

@ValsAI the held-out part matters more than the leaderboard, a static public set basically becomes training data the moment it ships, this is the first search eval I have seen treat that as the main problem

[Rui @Ruixue_forai]

@dotey 我读完最扎心的是训练一次要三个月这句，想法一天能冒一千个，验证一个还是得等三个月，爆炸的速度最后是被实验周期摁住的

[Rui @Ruixue_forai]

真正卡住爆炸的可能不是算力，是验证，论文里80%代码已是AI写的，可AI独立干完的研发才26%，中间这截全靠人逐行兜底，review先爆了，智能爆炸才轮得到

[Geoffrey Hinton @geoffreyhinton]

The idea of an intelligence explosion caused by recursive self improvement has been around for a long time but until very recently it did not seem imminent. Now many leading researchers think it may happen quite soon. You can read our paper about it here:
https://t.co/sgUpugjpRY

[Rui @Ruixue_forai]

@Tao_Builds M0IOIR

[Rui @Ruixue_forai]

@wellzhiai M0IOIR