Chip Huyen

@aisysbooks @goodailist
AI Engineering: MLSys: @chipslib

[Chip Huyen @chipro]

My 8000-word note on agents: https://t.co/uELWfPtS9N

Covering:

1. An overview of agents

2. How the capability of an AI-powered agent is determined by the set of tools it has access to and its capability for planning

3. How to select the best set of tools for your agent

4. Whether LLMs can plan and how to augment a model’s capability for planning

5. Agent’s failure modes

AI-powered agents are an emerging field with no established theoretical frameworks for defining, developing, and evaluating them. This post is a best-effort attempt to build a framework from the existing literature, but it will evolve as the field does.

As always, feedback is much appreciated!

[Chip Huyen @chipro]

Braden and team are great! Highly recommend this program for independent researchers working on cool stuff!

[Braden Hancock @bradenjhancock]

Slingshots // Four is live!

Another batch, another fourteen projects shipping open source artifacts that I’m excited about. See below for the full list. A few themes from the batch:

1. Harness time: This quarter harnesses shot up in visibility, with increased appreciation for the parts of an agent that aren’t the model and what they can give you. This batch includes a harness supporting persistent agency (Headlong); a harness supporting self-improvement via context management (ACE); a harness supporting self-improvement of the agentic harness itself (Exo); and a harness supporting multi-agent setups (dotagents).

2. Up and down the stack: There’s innovation to be had at every layer of the stack. We saw projects worth backing at the level of kernels (Thunderkittens); inference (M*); training (Marin); data (Atlantic-CC); evals (Decision Assistants); optimization (SkyDiscover); applications (Wagglehaus), etc.

3. Talk of the town: Slingshots continue to tackle timely problems at the frontier. Concurrent with Jev taking over X, an even faster type 1 decision model (CLM) came out of academia. With RSI on the brain of every big lab researcher, we now have a public literature retrieval benchmark grounded in authors’ firsthand accounts of their research process (ScholarCatalyst). And a new RL training method (Pedagogical RL) addresses head-on the question vexing RL practitioners of how privileged information can be used to discover generalizable behaviors, not just verify.

Congrats to all the researchers in this batch and the previous three who continue to ship their research out into the world with open artifacts that can be adopted and built upon. Open science leads to better outcomes for everyone.

And the good times will continue to roll – batch five is looking just as good. If you’re a researcher with a project that’s gaining momentum and ready for an adrenaline shot, shoot me a DM or apply at https://t.co/HRUiLPRZzT.

👏 @a1zhang, @AlexKrentsel, @ankrgyl, @Azaliamirh, @bariskasikci, @bfspector, @chelseabfinn, @chipro, @Diyi_Yang, @dlwh, @drmapavone, @HazyResearch, @jackyk02, @jonasgeiping, @JonSaadFalcon, @jure, @kothasuhas, @lateinteraction, @martin_casado, @mertcemri, @michaelryan207, @MikeCafarella, @NickAJalbert, @NoahZiems, @nsagan271, @ohmyksh, @percyliang, @qizhengz_alex, @RussoMatthew, @samrmadden, @shagarw21, @ShashwatGoel7, @shulynnliu, @simran_s_arora, @souradipchakr18, @stuart_sul, @tianyu_li_, @tim_kraska, @WilliamBarrHeld, @yoonholeee, @YejinChoinka, @Stanford, @MIT, @LaudeInstitute, @CarnegieMellon, @UW, @UCBerkeley, @MPI_IS, @UWMadison

[Chip Huyen @chipro]

Interesting approach. Models can't output freeform text but can choose from a set of predefined values. Could be useful for data labeling and tasks with a fixed list of possible actions.

Unclear how reasoning would work though, but it's super cheap (output tokens are free!)

[Diogo Almeida @CompleteSkeptic]

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?

I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev

• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions

AFAICT the shortest path to AI-based economic revolution

[Chip Huyen @chipro]

@akashtattva how many context tokens do these skills take? how are you evaluating them?

[Chip Huyen @chipro]

i'm researching best practices for skills.

what's the most number of skills you've had installed for your agents?

[Chip Huyen @chipro]

real question: why does gpt 5.6 over engineer so much?

[Chip Huyen @chipro]

what's a good model tiering system? i'm sick of telling my agent orchestrator things like: "for Claude, use model X, for OpenAI, use model Y, etc."

i want to be able to tell my orchestrator: "use models tier ..." for this kind of task

[Chip Huyen @chipro]

that's the problem he should've sent them in all caps

[Anthropic @AnthropicAI]

We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis.

It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%.

https://t.co/aZDvqqhHRi

[Chip Huyen @chipro]

why don't model providers price tokens like electricity, higher price during peak hours and cheaper off peak?

[Chip Huyen @chipro]

Auto science is gonna be 🔥
Congrats anh Quoc and team!

[Quoc Le @quocleix]

Excited to co-found Discovery Loop with my long-time collaborators @JeffDean @Sanjay_Ghemawat @OriolVinyalsML . Our mission is to automate machine learning, engineering and science. Learn more at: https://t.co/JFHy15o1uw ♾

[Mira Murati @miramurati]

Our first model, Inkling. Trained from scratch, weights are open, fine-tunable on Tinker today.

[Chip Huyen @chipro]

bored at the airport so i made this https://t.co/kh4iB0poIj

[Chip Huyen @chipro]

@ComfyUI @craft_ventures @PaceCap @chemistry "Day-one model support: every major release, compatible at launch" <- this is quite a commitment! Awesome news! Congrats!

[Chip Huyen @chipro]

Congrats to @AlecRad @Luke_Metz and @soumithchintala!

It's really cool to see work produced by a group of 20-somethings without a single PhD between them winning this award.

I hope to see these three collaborate again one day :)

[ICLR @iclr_conf]

We are honored to announce the Test of Time awards for #ICLR2026 🏆 This award recognizes papers published 10 years ago at ICLR 2016 that have had a lasting impact on the field:
https://t.co/JqYiqrAvgz

[NeetCode @neetcode1]

Got to meet the wonderful Chip Huyen @chipro

She’s so nice and smart!!

[Karina @karinanguyen]

Excited to release PostTrainBench v1.0!

This benchmark evaluates the ability of frontier AI agents to post-train language models in a simplified setting.

We believe this is a first step toward tracking progress in recursive self-improvement 🧵:

[Chip Huyen @chipro]

For those that chose Never, what do you do?

[Chip Huyen @chipro]

How long do you think AI will be able to fully automate your job?

[Chip Huyen @chipro]

Comparing the AI open source ecosystem in the US, China, Europe, and the rest of Asia over time

[Chip Huyen @chipro]

I built https://t.co/4rAUTXjAhc to help me stay-up-date with new AI stuff.

It's tracking 14K open source repos so far, with contributions from over 145K developers.

Every day, it:
- searches for new AI repos (based on 123 keywords and topics)
- surfaces repos that are gaining traction, and
- categorizes each repo

The annotations are done by AI so they are not super accurate, but they've helped me find some useful stuff.

It also lets me see where the contributors are, so when I travel, I can find folks doing cool stuff in a new city or country.

[Chip Huyen @chipro]

Super impressed by the projects at the Agentic Hackathon last weekend! Many teams work on really hard/important problems:

* Long running tasks: memory management, recovering from mid-task failures, and maintaining consistency across steps and sub-agents

* Adaptive retrieval from multiple sources: databases, search indices, and websites

* Agents that work with voice, video, and even 3D environments

If you are in SF, come check out the finalist demos tomorrow! https://t.co/JytAN5xqyG

There will be talks by Douglas Eck, who is doing amazing work with Veo and Imagen and many other awesome folks.

Thanks @MongoDB and @cerebral_valley for hosting and for letting me serve as a judge for these fantastic projects.