momo

01 | 研3 | 算法 | 啥都碎碎念
保持热爱 | 保持好奇
:-)

[momo @x0Er_go]

哈哈哈乐死我了

[Mikel Artetxe @artetxem]

Brilliant idea! Next up: Apple randomly reboots your Mac if you're building competing tech, Gmail silently edits your email if you mention rival platforms, and Tesla Autopilot swerves if it detects you're working on self-driving cars.

All in the name of safety, of course. Because malicious actors controlling the world’s operating systems, inboxes and cars would be extremely dangerous!

[momo @x0Er_go]

@shao__meng 可以设置限制文件夹列表

[Marc Andreessen 🇺🇸 @pmarca]

My information consumption is now 1/4 X, 1/4 podcast interviews of the smartest practitioners, 1/4 talking to the leading AI models, and 1/4 reading old books. The opportunity cost of anything else is far too high, and rising daily.

[momo @x0Er_go]

@MaxForAI @lxfater max开篇帖子讲一讲🤣

[momo @x0Er_go]

@MaxForAI 笑死，吃瓜后还能刷到安利

[momo @x0Er_go]

@op7418 确实 让我想起了沐神

[momo @x0Er_go]

@MaxForAI 培养人才不应该是公司的一种社会责任吗，都这么搞未来不敢想象

[momo @x0Er_go]

@plantegg 感觉ysy是一个很有意思的人

[momo @x0Er_go]

最近开始学车了，这周连续四天去练车，其中有三天我选择了骑电驴往返驾校，虽然白天可能有些晒，但两小时通勤在路上，吹着凉风，远离北京高楼大厦，欣赏郊外的群绿和蓝天，感觉是一种难得的奢侈...

[momo @x0Er_go]

@AltomareDonatha @howie_serious 官方原话：我们支持全球大多数认证的大学和学院。如果您的学校邮箱未被验证，请通过[email protected]与我们联系。

[momo @x0Er_go]

个人使用心得
- Notion：项目管理 - 看板
- Obsidian：知识管理 - 知识库

[momo @x0Er_go]

@MinLiBuilds @grok 哈哈哈感谢哥🤣

[momo @x0Er_go]

karpathy每次都能掀起一股热潮

[momo @x0Er_go]

@fkysly 我网页只敢用 无痕模式

[momo @x0Er_go]

the first principle in agent area

[Chayenne Zhao @GenAI_is_real]

"PRs should be prompt requests" is the most karpathy thing ive ever heard and hes completely right. when models are good enough that implementation is a commodity, the scarce resource becomes the quality of the specification. this is exactly why system design and architecture skills matter MORE in the AI era not less. the idea file concept is elegant - you share intent and constraints, the agent handles the rest. the gap between a good idea file and a bad one is the same gap between a senior and junior engineer except now its measured in output quality not code volume @karpathy

[momo @x0Er_go]

挺有意思，苹果上一篇给我印象比较深的还是一篇评测大模型能力，通过替换一些逻辑无关的名词名称，部分模型直接降智，苹果“打假”这块真有点东西

[Berryxia.AI @berryxia]

兄弟们，原来蒸馏如此的见效快！
难怪大厂都热衷于此哈哈哈😂

Apple Research（苹果研究院）刚刚发布了一篇“超级简单却效果炸裂”的论文，标题直接叫《Embarrassingly Simple Self-Distillation Improves Code Generation》（尴尬的简单自蒸馏就能大幅提升代码生成能力）。

论文核心发现（Simple Self-Distillation，简称SSD）：

你不需要：

- 更好的教师模型
- 任何verifier（正确性验证器）
- RL（强化学习）
- 代码执行环境
- 外部标签或奖励模型

方法简单到离谱：

1. 用当前模型自己采样生成代码（带一定temperature和truncation，不用greedy解码）
2. 完全不过滤这些输出的正确性
3. 直接拿这些“原始”输出做标准SFT（监督微调）

就这么三步，模型就能大幅进步！

实测效果（震撼）：
- Qwen3-30B-Instruct：LiveCodeBench pass@1 从 42.4% → 55.3%（相对提升30%！）

- 尤其在hard problems上提升最大：pass@5 从31.1% → 54.1%

- 只需每个prompt采样1次就够

- 在Qwen和Llama系列的4B、8B、30B规模上全部有效（包括instruct和thinking变体）

论文最有洞见的解释：

很多coding模型其实已经把“正确能力”藏在权重里了，只是greedy decoding（贪婪解码）把它锁住了。

SSD通过在自己生成的数据上训练，上下文依赖地重塑token分布。

在需要精确的地方压制干扰项，在需要探索的分支处保留多样性，从而把模型的潜在能力真正释放出来。

总结一句话：

“很多coding模型其实在用自己的权重‘欠发挥’。
用自己的输出再训一轮，就能把藏着的实力挖出来，而且完全不需要外部信号。”**

地址见评论区👇

[momo @x0Er_go]

@ttt36538104 @YangGuangAI chrome页面很占内存有啥方法吗

[momo @x0Er_go]

个人还是比较信仰知识管理的，在这个信息量过载的时代，想要成为通才，或者结合 ai + 知识系统 是个捷径

[momo @x0Er_go]

最近刚好在探索 claude code + obsidian 管理知识系统

[Andrej Karpathy @karpathy]

LLM Knowledge Bases

Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So:

Data ingest:
I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them.

IDE:
I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides).

Q&A:
Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale.

Output:
Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base.

Linting:
I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into.

Extra tools:
I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries.

Further explorations:
As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows.

TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

[momo @x0Er_go]

amazing

[Obsidian @obsdmd]

The Obsidian team is growing from three engineers to four engineers.

Competitive SF salary. Fully remote, live anywhere. Apply below.