Shuyao Tim Xu

long horizon @Kimi_Moonshot

[Shuyao Tim Xu @TimXu222575]

Life update 🚀

I started hill climbing my X follower count to receive @elonmusk's payout. I must say elon is very generous!

Bear with me if I started posting crazy things. Opinions are my own

[Shuyao Tim Xu @TimXu222575]

@sun_hanchi many are saying starting from 4.7 they used a smaller base

[Dan Saunders @djsaunde]

reading tech reports, I feel:
1. many strategies work for LM training
2. any particular strategy that worked feels like it may have come down to details of their setup / tuning

these are typically supplied with post-hoc justification of their choices

[Shuyao Tim Xu @TimXu222575]

@djsaunde 100% true

[Shuyao Tim Xu @TimXu222575]

@liruifengv 等下elon又拿这个作为证据push grok build

[Shuyao Tim Xu @TimXu222575]

@teortaxesTex @nrehiew_ is the mai 22% mfu also relative to peak bf16 flops of gb200? If yes then it is indeed under optimized

[Shuyao Tim Xu @TimXu222575]

@thsottiaux may the incidents happen more and more often so we get more resets

[Shuyao Tim Xu @TimXu222575]

@olive_trees234 @Kimi_Moonshot 不过开箱即用其实也挺好

[Shuyao Tim Xu @TimXu222575]

@olive_trees234 @Kimi_Moonshot 把skill bundle进去了，之后可能会改掉

[Shuyao Tim Xu @TimXu222575]

@ClementDelangue There are many "open routers" in china doing this already haha

[Shuyao Tim Xu @TimXu222575]

Very clever and excited to see it works out well. On-policy self-distillation is perhaps the most efficient way to learn hindsight in multi-turn agent setup.

If using standard SFT distillation, it would require continuing from a snapshot of the trajectory and container, and overall adds a lot to complexity.

[Sasha Rush @srush_nlp]

On-Policy Distillation is the most active new research direction being explored in RL for LLMs. Had the chance to discuss how it works with Dwarkesh and why it fits so nicely into large-scale pipelines.

[Shuyao Tim Xu @TimXu222575]

@bochencs @elonmusk I haven't received the revenue sharing though . Need at least 2,000 followers.

[Shuyao Tim Xu @TimXu222575]

Life update 🚀

I started hill climbing my X follower count to receive @elonmusk's payout. I must say elon is very generous!

Bear with me if I started posting crazy things. Opinions are my own

[Shuyao Tim Xu @TimXu222575]

@julien_c only hear people using screen when they don't have sudo and can't figure out tmux compilation

[Shuyao Tim Xu @TimXu222575]

@eliebakouch I now have a secondary connection with elon!

[Shuyao Tim Xu @TimXu222575]

@lateinteraction Can we distribute GEPA under a skill? Maybe it can also invoke the dynamic workflow for latest Claude Code.

[Shuyao Tim Xu @TimXu222575]

@shreshthsaini A mix of creativity and hallucination like deepseek r1 or k2-0704? Just guessing haha

[Shuyao Tim Xu @TimXu222575]

From first look, it seems that the whole pipeline is very clean

It has very little bootstrapping from existing llms, which is very different from the nemotron approach

I bet this model will smell very unique and "raw", maybe something like DeepSeek R1

[elie @eliebakouch]

WOW microsoft new "MAI Thinking 1" model comes with a 109 page tech report that looks REALLY detailed, this is amazing

[Shuyao Tim Xu @TimXu222575]

This is just deploying hosted static sites though? Claude web, Google AI studio can already do this.

Maybe the novelty comes from achieving this with desktop app rather than web, which makes sense.

What's cooler? Kimi Agent mode can build and deploy full-fledged full-stack apps!

https://t.co/Ye510WJFNj

[OpenAI @OpenAI]

Building apps has never been easier.

With Sites, Codex can turn your work, ideas, and plans into an interactive website or app your team can explore, use, and share with a URL.

Rolling out to Business and Enterprise plans, before expanding more broadly.

[Shuyao Tim Xu @TimXu222575]

@willccbb but yeah, some papers should be honest and just show us how their opsd runs collapsed

[Shuyao Tim Xu @TimXu222575]

The best motivation for opsd is perhaps pushing the boundary (otherwise, on-policy rl makes fresher and more "on-policy" model)

then in that sense, a good opsd run should beat the peak rl checkpoint in a domain.

it is acceptable for me if opsd pushed the boundary but collapses eventually. many successful rl runs collapse in the end anyway