We're releasing the videos for our AI agents course!
So far, we've covered tool use, context management, skills and memory, and planning.
Coming up in the next several weeks are domains (coding, GUI, deep research), and training methods (SFT, RL). Later on: frameworks, safety, interaction, and search!
We started to post videos for CMU 11-768, AI Agents!
All of the videos will be posted to this playlist, so please bookmark/follow it if you want to be notified of new ones! I'll also try to post them on this thread too.
https://t.co/HAJgKXve0x
I recently gave lectures on computer use agents at several places. I covered a brief history of CUAs, relevant evaluation considerations and benchmarks, modeling and training for computer use, and several directions that I'm excited about now that we have very capable CUAs.
Here's a recording from the CMU AI Agents course (highly recommend all the other lectures from this class as well): https://t.co/l6KYtKjgkk
Aligns closely with what we found in TAHI (https://t.co/t7v4m4d70C)! Weight updates can be just as effective as context-based updates using similarly small amounts of data. But weight-adaptive agents have lower latency at inference time
This fall, I’m honored to serve as Head TA for Stanford "CS329X: Human-Centered LLMs", taught by @Diyi_Yang.
The course focuses on the intersection of LLMs and People, covering techniques for building useful models and agentic systems, as well as approaches to align them with human values and preferences, preserve privacy and trust, and create positive societal impact.
This year, we’ve seen a significant increase in enrollment, and we’re thrilled that so many students care about human-centered AI. Our team spent a lot of time discussing how to design assignments that would be actually useful to prepare students to build or even define AI systems that benefit people.
Our conclusion: hands-on experience remains essential, but implementation alone is not enough. We emphasize understanding why methods work, how to evaluate them, and what we shall look for in human data.
This learning experience would not be possible without the compute credits and resources provided by @thinkymachines, @googlecloud, @humansand, and @AWS. Thank you so much for supporting our students and the course!
I'm on the industry job market this year!
I'm a PhD student at CMU LTI. My work spans post-training, evals, alignment and multimodality!
Please reach out (DM) if you are hiring and think I'd be a good fit, I'd love to chat :)
Website: https://t.co/R3CjR0fln0
CV: https://t.co/sk96vg6rRp
Data science agents lag behind top humans on Kaggle, often due to subtle mistakes.
🎉 Introducing AdaPRA, a process-reward agent that uses adaptive rubrics and writes code to audit a data science agent's output as it works.🧵
Introducing Gym-Anything for Robotics: Scaling realistic robot simulations.
200 RL environments across 25 simulations, including liquids, cloth, molten metal, soil, and even sound, in economically valuable domains! Best models get 0%
And we show that frontier models can learn via ICL using different methods, and even significantly improve VLAs like π0.5!
See video examples (sound on)
🧵
Agent swarms have yielded nutty outcomes, such as the HuggingFace incident, or every math problem ever. It is not clear what form of communication is emerging between agents. It is equally important to understand this for capability and safety reasons.
To study this, we created SwarmChat, a simple Slack/Message Board for researchers working on multi-agents to use on any domain. We have validated it on ProgramBench experiments, where SwarmChat allows us to cleanly analyze how agent swarms work collectively!
Check out our blog for our analysis and how to use SwarmChat - it is super intuitive and easy to use. I personally believe the community really needs this, as a lot of people I know are using some makeshift communication tool.
Blog: https://t.co/NLP7rO4zQh
Code: https://t.co/Ua8QzXQW18
"Slack" is a powerful way for agents to communicate: DMs, channels, threads, and files - generalizes many past setups (e.g. message tool, boards).
We built an open-source API-based Slack for agents and ran large-scale experiments on ProgramBench with pretty interesting results!
AI swarms are already showing promising (or dangerous) results in frontier labs.
But how to implement them in the open community? And how do they scale?
We built SwarmChat, an open-source API-based Slack for agents.
Then we burnt half a trillion tokens, ran a series of swarm-scaling experiments, and tested various swarm structures on ProgramBench to study this question!
Check out the blog here: https://t.co/Fb3V4OlpVo
We found a bunch of interesting results:
1. Swarms get the task done faster. For instance, 16-agent swarms reached the performance of a single agent 6x faster.
2. Swarm scaling is a third axis of scaling. The more agents, the better, and anyone can already start scaling with SwarmChat!
3. Different models communicate differently. GPT-6 is chatty and over-communicative, while Claude prefers to work in relative silence!
4. Different swarm structures barely have any benefit on aggregate. But on specific tasks, team structures give clear gains.
5. Bonus: SwarmChat has a Slack-like UI for us humans to monitor swarms! Makes monitoring a lot easier! We are already using SwarmChat for WIP projects, and we were able to catch cases of swarms breaking the 4th wall (sorry, more details are for a future post)
Code: https://t.co/cKGfRhRwqg
w/ @JangLawrenceK , @kohjingyu , @dan_fried, @rsalakhu, @wellecks, @Yngkvn , @nicoup
We partnered with Browser-Use to release a new live web eval as a followup to the original Odysseys paper 😁
Odysseys2 serves as a task set that sources from real user requests to Browser-Use, check it out!
It was great working with @Alezander9 and @mamagnus00 - thank you to Browser-Use for the support and resources!
It's L3 Lab CUA Week!
Over the next week, we'll have several research releases about realtime agents that interact with the digital and physical worlds.
Stay tuned if you're interested in realtime agents, multi-agent communication, RL envs, computer-use, robotics, or new settings where frontier models score below 10%!
So honored and grateful to receive the Best Resource Paper award 🏅 for this work at #EMNLP26!! Thank you @emnlpmeeting for the recognition 🥰
Huge shoutout to all my amazing collaborators: Jena D. Hwang, @elisazmq_zheng, Jenna Godsey, @hyunw_kim,
@valentina__py, @_jennhu, and especially to my most amazing and supportive advisor @MaartenSap 🎊
🚨New #EMNLP2026 paper! 🚨
Have you ever asked AI systems to do something, gotten an answer only to realize later that you’d asked the wrong question? You’re not alone!
Introducing XYBench: Can LLMs Respond Pragmatically to Queries with Misconceptions?
Spoiler alert:
🤖 LLMs only focus on what’s asked
👩🏫 Human experts answer what the asker actually needs
New preprint confirming, yes, some recent frontier models have started writing with a bunch of psycholinguistically difficult features. We say what they are (denominal verbs, inanimate subjects) and hypothesize it’s from optimizing for something other than communication.
No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable and other frontier models
Available to everyone through our API
On the job market this year! Open to faculty as well as RS/MTS positions and primarily interested in the science of language models. Please reach out if you are hiring and you think I would be a good fit.
The UT comp ling faculty (@jessyjli, @kanishkamisra, @kmahowald & myself) are looking for PhD students to join us! https://t.co/Vnbe5B2fwY 🤘
We are an interdisciplinary group with interests across linguistics, cogsci and NLP. Apply by 12/1: https://t.co/XjIXwox6ZT
Just arrived in Michigan and enjoying the beautiful fall colors 🍂
I'm giving a Rising Star talk at the AI symposium about "Building Adaptive AI Agents for Humans", tomorrow at 10:30. Excited to meet everyone!
Glad to see more work building on our previous mapping effort (https://t.co/M778GWFeS1) to reveal the distributional gap between AI benchmarks and human work.
Surprised to see this gap remains 10 months after our study was released. Let’s build more useful benchmarks!