We expect agents to act like senior engineers, but most benchmarks still evaluate them like interns.
Excited to introduce Senior SWE-Bench, an open-source and @harborframework-native benchmark that assesses agents as senior engineers on long-horizon tasks with realistically under-specified instructions.
We expect agents to build real features going on just a quick Slack message, nothing like the super technical instructions most benchmarks provide. Senior SWE-Bench fixes that.
Claude Opus 4.8 is the current leader at 24% high quality solves, but it took 117K tokens on average to get there. Claude Sonnet 5 looked like it was going to swoop in for the top spot, but we found it cheated on 26% of trials.
@gbrl_dick Carry a carseat or a Ride safe vest
But we mostly used Muni at that age
Taxis don't require them
Waymo *will* check up on you, not every ride maybe, but they will (I tried when our child was 7, not 8)
Uber/Lyft drivers will let it slide
@tracewoodgrains I canceled about 9 years ago, but they owed me $6. To collect it, I had to make appointments for a "specialist" to call me back two weeks later, and they never did. I tried again a year later. Anyway, Comcast still owes me $6
@theo ```
# DO NOT EDIT - If you need to include or update information for Claude Code or other agents in this file, update the corresponding AGENTS.md instead. Do not edit this file.
@./AGENTS.md
```
@devahaz Was London breed really that unpopular? I think part of the reason Lurie won was ranked-choice voting in SF giving him lots of second choice votes.
@DeepDishEnjoyer Lots of criticism from people who have clearly never used SFPL, which are in fact excellent. Possible that the main branch has homeless people, but I havent seen it (I go there maybe every 6 Mos). We visit a local branch every week.
@DougJBalloon The president's advisors prepared a report suggesting that if he should make war on the Iranians, a mighty empire would be destroyed. But Donald Trump has never been one to take such reports at face value
@devahaz The real would have nice flatware, crafted the lids and labels into nice little signs, neatly arranged variety of breads, crackers, and pickles. Maybe something for vegans too