D C 🌱

Aspiring Cheerful Rationalist, Googler. Opinions are my own. Retweet means "Interesting 🤔".

[D C 🌱 @danielpcox]

“This is one possible explanation but there are others.“

Mantra for when an attractive explanation leaps easily to mind.

[hardmaru @hardmaru]

Getting LLMs to simulate “true” randomness or generate diverse outputs is surprisingly difficult. We found a simple prompting trick that solves this by having the model generate and manipulate a random string. To be presented at #ICLR2026 this week!

Blog: https://t.co/CyevqqJ5ej

[Natalie Wolchover @nattyover]

Bacteria move around using a molecular machine called the flagellar motor that rotates faster than the flywheel of a race car engine and switches directions in an instant. After 50 yrs, scientists have finally figured out how it works. “My lifelong quest is now fulfilled.” Link⤵️

[Tech with Mak @techNmak]

Someone removed the vector database from RAG and got better results. Much better.

Here's what traditional RAG actually does under the hood:
it chunks your document into pieces, embeds those pieces into vectors, and retrieves based on semantic similarity. The assumption is that similar text = relevant text.

That assumption breaks completely for professional documents.

When you ask "what were the debt trends in Q3?", vector search returns chunks that look similar to that question. But the actual answer might be buried in an appendix, referenced across three sections, in a part of the document that shares zero semantic overlap with your query. Traditional RAG never finds it.

Similarity ≠ relevance. PageIndex was built around that insight.

Inspired by AlphaGo, it builds a hierarchical tree index from your document - an intelligent table of contents optimized for LLM reasoning. Then it navigates that tree the way a human expert would. Not pattern matching. Reasoning. "Debt trends are usually in the financial summary or Appendix G, let's look there."

What disappears:
→ No vector DB to build or maintain
→ No arbitrary chunking that breaks cross-section context
→ No opaque retrieval you can't explain or trace

What you get:
→ Retrieval traceable to exact page and section references
→ Multi-step reasoning across document structure
→ Works on financial reports, legal filings, regulatory documents

The benchmark:
→ PageIndex: 98.7% on FinanceBench
→ Perplexity: 45%
→ GPT-4o: 31%

Open source.

[Jim Fan @DrJimFan]

The power of the Claw, in the palm of a robot hand. Agentic robotics is here! Today, we open-source CaP-X: vibe agents, alive in the physical world. They incarnate as robot arms and humanoids with a rich set of perception APIs, actuation APIs, and auto synthesize skill libraries as they go. CaP-X is a strict superset of our old stack, because policies like VLAs are “just” API calls as well. It solves many tasks zero-shot that a learned policy would struggle with.

And we are doing much more than vibing. CaP-X is our most systematic, scientific study on agentic robotics so far:

- We build a comprehensive agentic toolkit: perception (SAM3 segmentation, Molmo pointing, depth, point cloud), control (IK solvers, grasp planner, navigation), and visualization (EEF, mask overlays) that work across different robots.
- CaP-Gym: LLM’s first Physical Exam! 187 manipulation tasks across RoboSuite, LIBERO-PRO, and BEHAVIOR. Tabletop, bimanual, mobile manipulation. Sim and real. Can’t wait to see the gradients flow from CaP-Gym to the next wave of frontier LLM releases.
- CaP-Bench: we benchmark 12 frontier LLMs/VLMs (Gemini, GPT, Opus, Qwen, DeepSeek, Kimi, and more) across 8 evaluation tiers. We systematically vary API abstraction level, agentic harness, and visual grounding methods. Lots of insights in our paper.
- CaP-Agent0: a training-free agentic harness that matches or exceeds human expert code on 4 out of 7 tasks without task-specific tuning.
- CaP-RL: if you get a gym, you get RL ;). A 7B OSS model jumps from 20% to 72% success after only 50 training iterations. The synthesized programs transfer to real robots with minimal sim-to-real gap.

3 years ago, our team created Voyager, one of the earliest agentic AI that plays and learns in Minecraft continuously. Its key ideas — skill libraries, self-reflection loops, and in-context planning — have since influenced many modern agentic designs.

Today, the agent graduates from Minecraft and gets a real job. It’s April Fool’s, but this Claw is getting its hands dirty for real!

Link in thread:

[Andrej Karpathy @karpathy]

- Drafted a blog post
- Used an LLM to meticulously improve the argument over 4 hours.
- Wow, feeling great, it’s so convincing!
- Fun idea let’s ask it to argue the opposite.
- LLM demolishes the entire argument and convinces me that the opposite is in fact true.
- lol

The LLMs may elicit an opinion when asked but are extremely competent in arguing almost any direction. This is actually super useful as a tool for forming your own opinions, just make sure to ask different directions and be careful with the sycophancy.

[Demis Hassabis @demishassabis]

Cool use case of AlphaFold, this is just the beginning of digital biology!

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[Kangwook Lee @Kangwook_Lee]

https://t.co/6beJPutjiG

[Greg Brockman @gdb]

Software development is undergoing a renaissance in front of our eyes.

If you haven't used the tools recently, you likely are underestimating what you're missing. Since December, there's been a step function improvement in what tools like Codex can do. Some great engineers at OpenAI yesterday told me that their job has fundamentally changed since December. Prior to then, they could use Codex for unit tests; now it writes essentially all the code and does a great deal of their operations and debugging. Not everyone has yet made that leap, but it's usually because of factors besides the capability of the model.

Every company faces the same opportunity now, and navigating it well — just like with cloud computing or the Internet — requires careful thought. This post shares how OpenAI is currently approaching retooling our teams towards agentic software development. We're still learning and iterating, but here's how we're thinking about it right now:

As a first step, by March 31st, we're aiming that:

(1) For any technical task, the tool of first resort for humans is interacting with an agent rather than using an editor or terminal.
(2) The default way humans utilize agents is explicitly evaluated as safe, but also productive enough that most workflows do not need additional permissions.

In order to get there, here's what we recommended to the team a few weeks ago:

1. Take the time to try out the tools. The tools do sell themselves — many people have had amazing experiences with 5.2 in Codex, after having churned from codex web a few months ago. But many people are also so busy they haven't had a chance to try Codex yet or got stuck thinking "is there any way it could do X" rather than just trying.
- Designate an "agents captain" for your team — the primary person responsible for thinking about how agents can be brought into the teams' workflow.
- Share experiences or questions in a few designated internal channels
- Take a day for a company-wide Codex hackathon

2. Create skills and AGENTS[.md].
- Create and maintain an AGENTS[.md] for any project you work on; update the AGENTS[.md] whenever the agent does something wrong or struggles with a task.
- Write skills for anything that you get Codex to do, and commit it to the skills directory in a shared repository

3. Inventory and make accessible any internal tools.
- Maintain a list of tools that your team relies on, and make sure someone takes point on making it agent-accessible (such as via a CLI or MCP server).

4. Structure codebases to be agent-first. With the models changing so fast, this is still somewhat untrodden ground, and will require some exploration.
- Write tests which are quick to run, and create high-quality interfaces between components.

5. Say no to slop. Managing AI generated code at scale is an emerging problem, and will require new processes and conventions to keep code quality high
- Ensure that some human is accountable for any code that gets merged. As a code reviewer, maintain at least the same bar as you would for human-written code, and make sure the author understands what they're submitting.

6. Work on basic infra. There's a lot of room for everyone to build basic infrastructure, which can be guided by internal user feedback. The core tools are getting a lot better and more usable, but there's a lot of infrastructure that currently go around the tools, such as observability, tracking not just the committed code but the agent trajectories that led to them, and central management of the tools that agents are able to use.

Overall, adopting tools like Codex is not just a technical but also a deep cultural change, with a lot of downstream implications to figure out. We encourage every manager to drive this with their team, and to think through other action items — for example, per item 5 above, what else can prevent a lot of "functionally-correct but poorly-maintainable code" from creeping into codebases.

[🍓🍓🍓 @iruletheworldmo]

codex with 5.3 taught me something that won't leave my head.

i had it take notes on itself. just a scratch pad in my repo. every session it logs what it got wrong, what i corrected, what worked and what didn't. you can even plan the scratch pad document with codex itself. tell it "build a file where you track your mistakes and what i like." it writes its own learning framework.

then you just work.

session one is normal. session two it's checking its own notes. session three it's fixing things before i catch them. by session five it's a different tool. not better autocomplete. it's something else. it's updating what it knows from experience. from fucking up and writing it down.

baby continual learning in a markdown file on my laptop.

the pattern works for anything. writing. research. legal. medical reasoning. give any ai a scratch pad of its own errors and watch what happens when that context stacks over days and weeks. the compounding gains are just hard to convey here tbh.

right now coders are the only ones feeling this (mostly). everyone else is still on cold starts. but that window is closing.

we keep waiting for agi like it's going to be a press conference. some lab coat walks out and says "we did it." it's not going to be that. it's going to be this. tools that remember where they failed and come back sharper. over and over and over.

the ground is already moving. most people just haven't looked down yet.

[D C 🌱 @danielpcox]

@tillahoffmann I'm going to rename my vibe-coded fork to "applejax" for now so I can upload to PyPI and use in my projects.

[D C 🌱 @danielpcox]

@tillahoffmann I pointed Opus 4.6 at jax-mps and had it solve all open Issues, and then keep going until it was as close to feature complete with jaxlib[cuda12] as it could get. I did this in my own fork so I wouldn't pollute your repo with AI slop, but it seems to have done well

[D C 🌱 @danielpcox]

@tillahoffmann I tried to confirm correctness by then having Opus 4.6 replicate papers and then generate stress tests. This found a few bugs, which are also now corrected in my fork. Please have a look at https://t.co/kOd2JYhwGR and then let's discuss if you want to use any or all of this?

[Kelsey Piper @KelseyTuoc]

@tszzl @memeticweaver @tautologer incredible to see people just casually reject the bedrock foundations of American greatness not just as some dumb nonsense that they're too cool to believe but as something they literally are not familiar with

[D C 🌱 @danielpcox]

Awkwardness isn't lethal.

[D C 🌱 @danielpcox]

Angels don't exist, but they should. I will help fill that gap. Let me know.

[D C 🌱 @danielpcox]

I have two fundamental theories of work: Ceteris paribus,
1. You work far better when it's fun.
2. You work far better with friends.

Practically: Steer myself and others towards work we naturally enjoy, and find the fun in the rest. Then, become ever better friends!

[Greg Burnham @GregHBurnham]

There were a number of interesting things about doing this evaluation manually. As ever, nothing beats seeing data up close. Some observations in thread.

[Sebastian Seung @SebastianSeung]

@ylecun @PrincetonNeuro @Princeton @theNASciences The fly's convolutional net is recurrent, dynamic, multi-scale, and likely mostly innate rather than learned. Now the challenge is to understand which of the net's properties are adaptive for vision, rather than accidents of evolution.

[Sebastian Seung @SebastianSeung]

@ylecun @PrincetonNeuro @Princeton @theNASciences the connectome revealed the literal convolutional net inside the fly brain, including its architecture of 230 feature maps, normalizations, and other layers,