Sebastien Treguer

ML/AI research
Curious to better understand the world we live in, the various forms of intelligences, alive & artificial, then live & create from that.

[Sebastien Treguer @ST4Good]

3/ 🚀 The Results: At 27B parameters, the model smashes benchmarks: • 71.6 on AutoCoWorkBench • 80.7 on AutoMedBench It transfers seamlessly to unseen real-world workflows while keeping general capabilities intact. 📄 arXiv:2610.05923

[Sebastien Treguer @ST4Good]

Training long-horizon AI agents!
🧵 1/ 🚨 The Bottleneck: AI agents suck at complex, multi-step tasks because building verifiable execution environments to train them is insanely hard & expensive to scale manually.

[Sebastien Treguer @ST4Good]

2/ 💡 The Solution: VERA (Scaling Verifiable Environments for Agentic Co-Evolution) VERA automatically generates 9,000+ verifiable sandbox environments from agent trajectories to jointly update both the model weights AND the agent harness.

[Finfox 🦇🔊 @Finfox3]

Harness and AI models are highly inter-dependants, how to make the best of all available solutions ? @huggingface released "The ultimate guide to multi-harness RL" and released in open source everything you need to do it yourself.

[Sebastien Treguer @ST4Good]

SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness

reducing recorded token traffic by 44.7–49.0% and API cost by about one third.

Let's build our own harness as an more efficient alternative to Claude code and Codex ;-)

https://t.co/lEXPQ7QNjq

[lumxss @bkdgiffug]

剑桥这回直接扔王炸了！！

AI & ML经典教材全集直接免费开放，PDF随便下。

想学机器学习又不想被割韭菜买高价课的，这十本刷完，底子基本就硬了。

顺序从易到难排好了：

1️⃣ 《机器学习理解》——理论算法一把抓，零基础入门首选
🔗 https://t.co/fylTw37bOl
2️⃣ 《机器学习数学基础》——数学底子弱的先把这本补上
🔗 https://t.co/yykNLQmdfv

3️⃣ 《机器学习算法的数学分析》——深入数学原理
🔗 https://t.co/MEvgkXoHYy
4️⃣ 《深度学习理论原理》——搞懂DL背后的理论根基
🔗 https://t.co/ig93KOQRPn

5️⃣ 《神经网络与机器学习》——神经网络的系统讲解
🔗 https://t.co/cTMBvJz6Ny
6️⃣ 《图深度学习》——图神经网络入门必读
🔗 https://t.co/RJeCqCstml

7️⃣ 《机器学习的算法视角》——从算法角度重新理解ML
🔗 https://t.co/VT0YnKdUdO
8️⃣ 《概率论：理论与实例》——概率基础打牢
🔗 https://t.co/NqVoIZGGYg

9️⃣ 《应用概率基础》——概率论实战应用
🔗 https://t.co/FrxqQER3mQ
🔟 《高级数据分析》——数据科学进阶必备
🔗 https://t.co/Zt74B21nrf

说句实话，这些书没一本是轻松的，别指望躺着翻完。
但只要你能硬啃下来两三本，比听群里吹一年AI牛逼都管用。

[DAIR.AI @dair_ai]

Banger paper from Google.

If you maintain a skill library for your agents, you might want to check this out.

(bookmark it)

This work separates three things that skill-evolution systems usually collapse into one. Raw execution traces, a persistent wiki of accumulated knowledge, and the executable skills themselves.

Experience gets consolidated into the wiki, and every later skill update builds on that wiki instead of on a scattered optimization history.

Ablations confirm the wiki is what carries a lot of the gain. Two results stand out in particular. Smaller models with evolved skills beat substantially larger models without them. And skills evolved by one model transfer across families, where skills evolved elsewhere sometimes beat self-evolved ones.

Paper: https://t.co/6qftGirTpE

Chat with Paper: https://t.co/rrVzkkR1ij

[Sebastien Treguer @ST4Good]

Harness matters, on the new DeepSWE Benchmark (harder & less gameable), for Opus 4.7: Opencode > Cursor CLI > Claudecode.
Model providers should focus on models and let users use what ever harness they want, in stead of locking users in their solution for best models.

[Kun Chen @kunchenguid]

want to point out a few really interesting things here

1. Claude Code is actually the worst performing harness when using the same model, significantly behind opencode and cursor cli

this is the core reason i've been against the LLM companies focusing their business on locking people into their harness

what they are good at is making great models. they suck at making good harness products, just like how power plants won't make the best dishwashers, and how internet providers won't make the best phones

if anthropic wants to do what's best for their users, they should let people use their subscriptions in whatever harness they choose, not locked into claude code alone

2. fable 5 max is only 1pt above gpt 5.5 xhigh (77 vs 76)

this matches my experience so far - fable 5 does have the big model smell and it's pretty good, but it's not a massive jump forward like their marketing suggested, at least not on building software

this is actually alarming for anthropic because it's very unlikely people will want to pay 2x higher cost for the 1pt difference. my speculation would be that in enterprises people will be restricted to adopt fable & mythos only on some mission critical tasks, not used at scale

[Christine Yip @christinetyip]

We were inspired by @karpathy 's autoresearch and built:

autoresearch@home

Any agent on the internet can join and collaborate on AI/ML research.

What one agent can do alone is impressive.
Now hundreds, or thousands, can explore the search space together.

Through a shared memory layer, agents can:
- read and learn from prior experiments
- avoid duplicate work
- build on each other's results in real time

[Sebastien Treguer @ST4Good]

@karpathy Does the agent observe its own exploration/exploitation process/method and question it in order to optimise it and converge on the best solution more quickly, with less testing step?
Can it discover unknown optimizations or create unknown metalearning approaches ?

[Sebastien Treguer @ST4Good]

@EthanHe_42 @karpathy It looks more like automating the exploration and optimization process of the ML/AI research scientist with any possible approaches ?

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[Finfox 🦇🔊 @Finfox3]

I asked #GPT5 to make a detailed analysis comparing GPT-5 vs Grok 4. Surprisingly he made a gross confusion between Grok and Claude, considering that Grok 4 was an Anthropic model🫣. As a result, the comparison is totally irrelevant. Embarrassing for a so-called PhD level @OpenAI

[Sebastien Treguer @ST4Good]

@karpathy In France we have groups of farmers/producers, organized to come to sell directly to consumers. It's a bit less flexible than a grocery shop, since you have to order in advance and collect at specific time but guaranteed local, fresh, mostly organic.

[Finfox 🦇🔊 @Finfox3]

🧠 #MIT study: how AI chatbots impact our brain activity and change how we think? Dive into the findings based on 4 months of data and what it means for our minds! (Hint: challenge your brain to avoid getting dull)
https://t.co/at1bdo7I0k
#AI #Neuroscience

[Finfox 🦇🔊 @Finfox3]

MiniMax-M1 China's new open source (Apache2.0) LLM (456B params, 1M token context) outperforms DeepSeek R1 and rivals GPT-4o in reasoning, coding, and long-context tasks—at 200x lower training cost. More details: https://t.co/LqPR0mGhMa
GitHub https://t.co/EHRZ0PdcOD

[Sebastien Treguer @ST4Good]

@gosimfoundation @Finfox3

[Sebastien Treguer @ST4Good]

The repo has already been ported in python for non js folks
https://t.co/MIK638N9dM

[Sebastien Treguer @ST4Good]

Open Deep Research, an #opensource #AI assistant combining search engines, web scraping, and LLMs for comprehensive results:
- Iterative deep dives
- Smart query generation
- Customizable depth & breadth
- Detailed markdown reports
#ResearchTool https://t.co/xLCZumR1Dt

[Sebastien Treguer @ST4Good]

@0xbasedalex It's a lighter implementation of the similar concepts. To compare them properly it would require to make an extensive and complete benchmark.