Tian Fang

software engineer

[Tian Fang @xfromhome]

@shirshakchavan I think you need a marketing person at least.

[Tian Fang @xfromhome]

@Secondmindsys @fchollet Is it possible to build one type of agent focus on proposing the problems for model to practice? E.g. an automatic math conjecture proposer, so the LLM will have infinite good problems to work on and improve itself?

[Tian Fang @xfromhome]

@__alpoge__ Are you working on the next one?

[Tian Fang @xfromhome]

@NoahKingJr Go to Mars, ship robots to build a city in Mars, if you move there, house is free.

[Tian Fang @xfromhome]

@the_codewala Be boss.

[Tian Fang @xfromhome]

@not_ellington We will have a perfect Ai agent: humble, witty, humorous, kind and helpful.

[Max For AI @MaxForAI]

YC 创始人 Paul Graham今天发了一段我觉得很值得年轻人看的话。

有人问他：

如果你现在只有17岁，你会做什么？

@paulg 的回答我觉得很有意思：

他会从零开始学习怎么构建大语言模型，然后把自己能接触到的所有硬件都用起来，尽可能训练出一个强大的模型。

如果只看到这里，很容易把它理解成一句“年轻人快去学AI”的建议，是不是很平常。

但PG又专门补了一条，我觉得这条才是重点。

他说：

注意，我不会去尝试创业。

如果自己只有17岁，他会先建立一套足够扎实的知识基础，为以后创业做准备。

因为真正深入理解LLM之后，自然长出来的创业想法，会比拿着自己17岁时已有的认知，坐在那里硬想“我应该创个什么公司”，质量高太多。

这句话从PG嘴里说出来还挺反差的，毕竟他可能是这个世界上最擅长鼓励年轻人创业的人之一。

YC过去二十年一直在告诉年轻人：早点开始，去做东西，别等。

现在面对AI，他反而给出了一个看起来很“慢”的答案：

17岁先别急着开公司，先去把LLM搞懂。

我其实挺认同这个判断。

过去两年AI创业最明显的一个现象，就是产品和idea的保质期越来越短。

24年大家觉得RAG是创业方向，今年已经成了基础设施。

去年大家还在做Coding Agent套壳，现在模型厂自己就把能力做进去了。

一个看起来很聪明的产品idea，下一次模型更新可能直接把你吃掉了。

所以现在如果一个年轻人问我：应该先找idea，还是先进入AI？

我大概率也会选后者。

尽可能靠近最前沿的模型、训练、Agent、数据和真实用户，亲手做东西，搞清楚这些系统到底能做到什么、做不到什么，以及能力边界正在往哪里移动。

因为AI这个阶段，对技术边界的理解，本身就在不断产生新的创业机会。

很多真正好的idea，可能根本没办法提前坐在咖啡馆里brainstorm出来。

你得先看到一个昨天还不可能、今天突然变得可能的东西。

然后意识到：卧槽，原来这个现在能做了。

我觉得PG这段话最值得记住的，其实就是这一点：

年轻最大的优势，是你还有时间把自己放到下一轮技术浪潮最深的地方。

公司以后可以开。

但这种从一个技术范式刚刚出现时，就开始跟着它一起成长的机会，一辈子可能也碰不到几次。

PS：其实是几代人都不可能遇到一次。

[Tian Fang @xfromhome]

@HeyAliux Jensen does not think so.

[Youssef Hosni @YoussefHosni951]

Loops vs. Graphs, clearly explained!

Loops are powerful, but they have a ceiling. A loop makes one unit of work better. It can produce an output, check it, correct it, and repeat until it passes. But the loop itself does not decide which units of work should exist, which can run in parallel, or which should be skipped entirely.

So you can end up with a very capable agent executing the wrong three steps, in the wrong order, one at a time.

Graph engineering moves that decision up a layer. The graph decides what runs, what can run at the same time, how results come back together, and what should never run at all.

You need both:

Inside a unit: the loop — produce, check, correct, repeat until the result passes.
Between units: the graph — split, fan out, merge, gate, and route work back when necessary.
Prompts → Context → Harness → Loops → Graphs

Once you move to graphs, you can introduce parallel execution, isolated contexts, conditional paths, and steps that simply do not run when they are unnecessary.

The important part is being selective about what becomes a node.

Use a model where judgment is required. Operations such as merging structured outputs, deduplication, schema validation, and deterministic checks often belong in code rather than another agent. They are cheaper, faster, predictable, and cannot be persuaded into changing a deterministic verdict.

A graph where every node and edge is another model call quickly starts paying rent on its own orchestration.

There is also one distinction that becomes important as the graph grows: graphs need two different return paths.

The correction path is short. A gate rejects one unit and sends it back to the step that produced it. This improves the current run.

The learning path is longer. An accepted result, failure pattern, or discovered constraint feeds back into the splitter or planner so future runs start with better assumptions. This improves the runs that come after.

Build only the first path, and you get a graph that can recover from mistakes but keeps rediscovering the same blind spots every time it starts.

And one rule matters a lot for convergence: when one unit fails, return that unit—not the entire batch.

If four branches run and only one fails, sending all four back means rewriting three outputs that were already correct. Repeat that a few times and the graph spends most of its time undoing its own progress.

[Tian Fang @xfromhome]

@suraj_sharma14 LLM post RL training is over? I want to see the Riemann conjecture is solved by LLM soon.

[Tian Fang @xfromhome]

@Hongyu_Lii @bowenwen_me @zhu_xinghao @du_yilun @YunzhuLiYZ @BirchfieldStan @SohaPouya @80gg_overmind @Dr_YanChang It still needs great amount of robot-specific data to train the action head to cover the whole robotic action space. Secondly, we need to study the how RL works in this training paradigm.

[Tian Fang @xfromhome]

@pmddomingos It should be, it is harder to manage a larger team than a small one.

[Tian Fang @xfromhome]

@elonmusk No one predicts the "next word prediction" can change the world at the beginning.

[Tian Fang @xfromhome]

@mfranz_on It should be, more use of the tool, better at it.

[Tian Fang @xfromhome]

@antopatrex1 Any statistical data clearly showing this trend?

[Tian Fang @xfromhome]

@vineerpasam People keep building apps in the last 20 years. If you can find something new, you are genius.

[Tian Fang @xfromhome]

@samzliu Like the self-driving car, we need a very efficient pipeline to filter the useful data to improve the LLM. It is quite like human, there are so much information everywhere, skip all noise and pick the real useful piece of information is critical for all intelligent systems.

[Sam Z Liu @samzliu]

This is one of the most insightful takes on RL I've read in a while from @BerenMillidge

I agree with the key insight but there are some areas where the framing causes confusion, particular in the double meaning of the word "bias".

The core argument is that RL works despite only giving one bit of feedback per rollout because every bit is directly related to the task we are trying to train as opposed to say SFT where only some bits of feedback matter. This means RL has near perfect SNR compared to SFT.

The mental model is that noise causes us to remain at low resolution on the jagged loss landscape, and the lower noise of RL enables us to descend further. (Note: noise in this sense is more about alignment of the gradient update to the task rather than policy variance)

This roughly matches the intuition that ICL is elicitation of existing behavior (i.e. moving with in the same resolution on the loss landscape) while RL enables us to become better at behaviors already in the support: i.e. we need a non-zero pass@K.

It also matches some empirical results that that post-training only moves the weights in a tiny tangent space around the pre-trained model, suggesting that we are merely descending down the nearby valley.

The implication is that RL over LLMs isn't really RL in the traditional sense: there's real no exploration going on, no learning new models of the world. It's more about refining and sharpening what the model already can kind of do.

However, where the post starts to waver is when it tries to extend this to a general unified framework for LLM-RL. In particular, there's a claim that LLMs cannot tolerate bias because they are precisely tuned, so the role of post-training techniques is merely to reduce the variance. There's a slight confounding of two different phrases of the word "bias" that causes a bit of confusion.

One definition of bias is the approximation error from the model class, which makes sense under the canonical explanation for double descent: in the over-parametrized regime large parameter model, the test loss is dominated by the variance term instead of the bias term and improvements come from reducing it. This matches the explanation for why smaller RL models are fine with more bias compared to RL over LLMs.

However, there's another definition of bias which gets mixed in. This bias is an estimator whose expectation isn't the true gradient toward the objective. This is confusing since GRPO and PPO use biased estimators, so RL over LLMs clearly work under bias. The further implication in the post that any large movement of the weights causes instability/collapse because of the pre-trained model sits in a finely-resolved sub-valley. However, this is confusing since when you RL, you are intentionally biasing the model toward a task that was not focused in its pre-training loss. So RL is by definition biased in two ways: in its estimator and in its objective.

Thus the bias-variance explanation doesn't explain why RL over LLMs are so sensitive to off-policy rollouts because RL is biased to begin with.

I suspect part of the answer is the usual mis-alignment concern. Off-policy rollouts don't work because of mis-alignment. However, the reward mis-alignment is implicit because the gradient update from the off-policy rollout causes the aligned reward to behave as if it was mis-aligned in catastrophic ways. In some sense, this is just the second type of bias from above, so we've merely pushed it onto the next turtle down.

(Side note: Humans also learn poorly from off-policy data. We learn less when we watch something than when we do it ourselves.)

The bias-variance point also doesn't explain why we don't have catastrophic forgetting. The magnitude of the weight shift is small, but we are yanking a lot of weights. If we were truly that sensitive, then this should destroy performance on other tasks which do not relate to the RL objective being trained on.

The real question to answer then is: why are some small weight changes good while others are bad? Are there specific directions which are preferred or not? Some of our preliminary experiments suggest that the updates which work are the ones which leave the model's NTK largely untouched, matching the intuition about "nearby valleys".

https://t.co/CFOdrR4bTR

[Tian Fang @xfromhome]

@vartekxx What is the learning club he mentioned?

[Tian Fang @xfromhome]

@firstadopter Coding is easy to verified correct or not, other things are difficult to verify. Picking code first is very smart move. OpenAi adjusts its target quickly once Anthropic proved it.