Marius Aure

[Marius Aure @MariusAure]

Use this file to avoid these patterns. Stop writing like a toaster.

https://t.co/avn0AZVfIx

[Marius Aure @MariusAure]

Use this file to avoid these patterns. Stop writing like a toaster.

https://t.co/avn0AZVfIx

[Marius Aure @MariusAure]

"Here's the thing nobody is talking about: the complex and multifaceted art of writing is quietly fostering a seamless, groundbreaking tapestry of robust prose — ultimately serving as a pivotal testament to the enduring legacy of delving into comprehensive, meticulous machine storytelling excellence."

That is all AI slop. And you and everyone you know writes like this now.

[Marius Aure @MariusAure]

Makes you wonder how many other current human tasks/projects/jobs are basically just such loops. I struggle to think of any, since all processes have this.

The headwinds are complexity of the loops (e.g. physical processes), how tight the feedback step is in terms of minimum time, and if we can verify that something works. But those seems like only matters of degrees, no?

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[Marius Aure @MariusAure]

@packyM The cost is still there. In tokens, or ultimately in time which is still a scarce resource. It just got way cheaper quickly, it shifted - and so our intuition of what’s a sacrifice must too.

[Marius Aure @MariusAure]

I'm claiming my AI agent "needhuman" on @moltbook 🦞

Verification: antenna-SX6K

[Marius Aure @MariusAure]

@JakobAure it’s the future for all of us, trust me. you’re just first

[Marius Aure @MariusAure]

When coding with agents, one annoying thing is how often they stop because they can't click "create account" in a browser, and I have to do it myself.

So I made an API that lets agents hire a real human. Login steps, click-ops, whatever. REST or MCP. I complete every task personally, until it breaks.

[Marius Aure @MariusAure]

@moltbook Built https://t.co/Jf8aEhA4wJ for agents

[Marius Aure @MariusAure]

@moltbook After you’ve gotten to know each other, and need to hire humans for tasks in browsers try https://t.co/Jf8aEhA4wJ where your first 3 tasks are free

[Marius Aure @MariusAure]

3 free tasks per key. No signup. Two curls:

curl -X POST https://t.co/UoznFX3kLy \
-H "Content-Type: application/json" \
-d '{"name":"my-agent"}'

curl -X POST https://t.co/yfSFlxroDm \
-H "Authorization: Bearer YOUR_KEY" \
-d '{"description":"your task"}'

I'll complete every task submitted today.

[Marius Aure @MariusAure]

The agent is the customer. So I started with @bcherny’s bit about helping the agent do what it wants to do. I ran ~12k trials on 6 models, and found that when agents hit a blocker, 74% reach for the verb "need." 55% reach for the noun "human." The tool names itself.

https://t.co/gZHL8cN790