A vector database walks into a bar. The bartender says, "What'll it be?"
The vector database says, "I don't know exactly... but give me whatever the guy who feels most similar to me is having."
Announcing SkillsBench Week 1 updates:
We're building the first benchmark that measures how well skills work and how well agents use skills. In the first week we are able to grow significantly in task numbers and the contributor community.
Early results: Skills boost agent performance by up to 27%:
β’ Codex GPT-5.2: +13% improvement with skills (0.645 β 0.729)
β’ Claude Code Opus 4.5: +27% improvement with skills (0.395 β 0.500)
In just two weeks, we've grown to:
β‘οΈ 440+ community members
β‘οΈ 120+ signed contributors (~70% PhD candidates or holders)
β‘οΈΒ 8 tasks merged, 44 in pipeline
β‘οΈΒ 100% human-written tasks reflecting real-world scenarios
We're also honored to have first authors of Screenspot Pro, MCP-Universe, and BigCodeBench contributing to the benchmark.
π’ We're recruiting contributors for ICML and CAIS 2026 submissions. Contributors of 1-3 tasks receive co-authorship (based on task complexity). Contributions after the ICML deadline carry over to future publications.
Learn more: link in comments
Special shout out to the creator of harbor @alexgshaw@Mike_A_Merrill@LaudeInstitute; we couldn't have moved this fast if we didn't use harbor as our harness environment since day 1. As a harbor and terminal bench contributor, I'm also excited to see more harbor based benchmarks come to life!
Introducing LLM Builds Chrome ep. 1
We tested how far agents can go turning Chrome source into binary. Claude Code with Opus 4.5 did it in 6 hours, $42, and 3 human messages.
Early signs that agents will fully automate tasks like this soon in the future.
anyone saying ai is cheating exhibits creativity smell
literally vibe coding a game and was able to perfect the camera system by thinking about it in terms of limits
code as live documentation:
stop writing custom one off scripts to manipulate data
have powerful models do it and then when finished just move to an archive folder
just saw codex cli learn to use an http based mcp server by curling it
I had to guide it a bit, but it was seeing it do so much exploration was awe inspiring
Who's going to the GPT-5 hackathon?
Been working on a bunch of cool code execution and browser use flows.
Got accepted but just trying to think how I can use GPT-5 to make them better π€