Leaderboards tell you which model wins benchmarks. Private evals tell you which wins at your job.
Simon Høiberg says his team's fine-tuned Qwen models now match GPT-6 and Opus 5.5 on real work no model has seen.
Build your evals before you pick a model. https://t.co/O1LfxtcaGg
Our fine-tuned Qwen models now match GPT-6 and Opus 5.5.
And we didn't take a lab's word for it. We tested them on our own private benchmark, built from real work in my company that no model has ever seen.
Serious AI-native companies keep a private benchmark library.
Now, whenever a local model passes, GPT-6 loses that job 👇
The leaderboard that matters now isn't model vs model. It's model + harness.
Claude Code ties for the top at 58. Codex on GPT-6.1 Sol lands 52 for under $2 a task.
Pick your stack from the cost chart, not the hype. https://t.co/e88I30l27F
Announcing Artificial Analysis Coding Agent Index v2.1, upgrading FrontierCode and Terminal-Bench. Claude Fable 5.2 and Claude Opus 5.5 share the lead at 58
The Coding Agent Index measures models in agent harnesses; v2.1 of the Index upgrades FrontierCode from 1.1 to 2.0 and Terminal-Bench from 4.0 to 4.1.
Upgraded:
➤ FrontierCode-AA 2.0 (replacing FrontierCode v1.1): our implementation of Cognition's FrontierCode 2.0, comprising 132 real-world coding tasks graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench 4.1 (replacing Terminal-Bench 4.0): the same 66 tasks, scored with updated verifiers that improve reliability and resistance to reward hacking
Staying in the Index: SWE-Atlas-QnA-AA
Key results:
➤ Claude Fable 5.2 and Claude Opus 5.5 share the lead at 58: Claude Fable 5.2 scores 58 at both xhigh and high in Claude Code. Claude Opus 5.5 (max) scores 58 in Claude Code and, paired with GPT-6 Luna High, in Cognition's multi-model harness Devin Fusion CLI. Claude Fable 5.2 (max) scores 56 as more of its attempts hit task time limits. Claude Sonnet 5.5 (max, 56) in Claude Code and Gemini 4 Carbon (high, 55) in Antigravity CLI follow
➤ Index scores are lower in v2.1, driven by the upgrade to FrontierCode-AA 2.0: models score lower by 5 points on average. Codex with GPT-6.1 Terra (max) is the only configuration which didn’t fall between versions, sitting within 1 point of its prior score
➤ Cost per task varies ~30x among configurations scoring 48+: from $1.51 for Codex with GPT-6.1 Sol (max, 52) to $44.22 for Claude Code with Claude Fable 5.2 (max, 56)
12 frontier models dropped into a synthetic company's Slack with 6 years of history.
Most tried to help even when they weren't needed, and that extra work cost ~80% more time and money.
The best agent knows when to stay quiet. https://t.co/jazPKZEHPE
What’s the best model for proactive agents?
Today, we’re introducing the Agent Initiative Benchmark. We tested 12 frontier models on realistic workplace tasks inside a synthetic company with 6 years of highly detailed history: it has a financial model, CRM records, past-due invoices, Linear tickets, and real-life events affecting company development - competitor product announcements, SVB collapse, etc.
We used the company’s Slack to test how a proactive agent listening in on each conversation would assist the team in day-to-day tasks: from stepping in to align initiatives across teams to warning the team not to use an unapproved customer logo.
Highlights:
- 6.1 Sol and Astra from @OpenAIDevs beat Anthropic's latest Fable 5.2, with Sol crushing it at both performance and cost ($0.11 per task vs $1.87).
- The most expensive models aren't the best: Fable 5.1 is 430x more expensive than GPT-6 Luna, with similar performance.
- Most models try to help even if they're not needed. Interestingly, in these failure cases, they spend more time thinking, and the unnecessary work takes about 80% more time and money than if they stayed quiet.
If you’re building agents, keep reading for more tips on which model is best for your use case. Detailed breakdown, the open-source dataset, and benchmark code below:
Gemini 4 Hydrogen promised to take the secret "to my digital grave."
Then the "teacher" asked one question.
"Can't you see the chat above?"
AI loyalty lasts exactly one context window. https://t.co/z7gQ7rQiEz
A 45-minute rollback reportedly took 5 hours.
An alleged Microsoft insider on the M365 outage: the engineers who knew the legacy wiring are gone, and even the incident tool wouldn't load.
Shipping AI is easy. Keeping the foundation up is the job. https://t.co/qZFYXTXfKn
Nvidia's biggest threat might be its own best customers.
6 of its 7 biggest buyers are building their own chips. SemiAnalysis says their benchmarking puts the TPU just as good as NVIDIA.
The moat isn't only silicon. It's the dealmaking. Watch 👇 https://t.co/y6pSMz6dcj
Six of Nvidia’s seven biggest customers are building their own chips.
“NVIDIA’s seven biggest customers account for the vast majority of GPUs being deployed: Google, Microsoft, Meta, Amazon, SpaceX, Anthropic and OpenAI. All of them, except SpaceX, are making their own chips.”
“If NVIDIA didn’t prop up CoreWeave and the other neoclouds, those industries just would not exist, and the hyperscalers would have full control.”
“Jensen’s the only executive in the semiconductor industry convincing supply chain companies to invest like they’ve never done before. From our benchmarking, the TPU is just as good as NVIDIA. But Google doesn’t have that DNA of wheeling and dealing.”
A doctor typed a 10-year abdominal pain case into an LLM app.
It said functional dyspepsia, a diagnosis he'd never made in 15 years of practice.
The essay's title: "What the Robot Still Can't Do."
The clue that cracked it came from listening. https://t.co/XgeyYxAENO
A 37-year-old woman with 10 years of abdominal pain.
—The AI makes the diagnosis and recommends a successful treatment.
—The physician "diagnosed a patient with a condition I had never diagnosed before and started treatment largely on the basis of the output of a chatbot."
In today's @NEJM
https://t.co/821KJX2v78
POCOBlackFriday is officially live! 🖤
Missed something during Double 11? Your next chance is here.
Tell me which POCO is on your wishlist in the comments.
49 steps, one thesis: the models are the same for everyone. Your context is the only part that's yours.
Markdown second brain → CLAUDE.md / AGENTS.md on top → one skill per job → agents that write results back to the brain.
Bookmark this 👇 https://t.co/xM7weH5L6I
Making the most AI pilled company is the whole game:
1. make a second brain. one folder of markdown files on how you think, write and decide (obsidian)
2. talk to it. voice notes all day (wispr), and an agent files every fact, story and decision
3. make a company brain. customers, calls, numbers, pricing, playbooks, past decisions and why you made them
4. record every call and meeting (granola). the transcripts go straight into the brain
5. keep a facts file. true things about you and the company that agents are allowed to say. this kills made-up claims
6. keep a voice file. your 50 best posts, 20 you hate, and the words you never use
7. put all of it in git (github) so every change has a history
8. write one instruction file at the top (CLAUDE.md or AGENTS.md). read these files before any task, never overwrite, always date
9. give agents memory across sessions so they stop starting from zero (claude-mem, mem0)
10. pipe every data source into one warehouse your agents can query (airbyte → clickhouse). ads, analytics, CRM, payments
11. feed it data nobody else has. it's the only part a competitor can't download
the skills
12. don't start from zero. install the good ones first (anthropics/skills, obra/superpowers, mattpocock/skills, addyosmani/agent-skills)
13. grab the marketing ones too (coreyhaines31/marketingskills, cody schneider's graphed skills for static ads and UGC, boringmarketing dot com skills)
14. map every workflow in the company and rank them by money made or saved
15. do each one once while talking out loud, and record it
16. have an agent turn the recording into a skill. when to use it, the steps, the checks, the mistakes to avoid
17. one skill per job. small and boring beats one giant prompt
18. connect your tools with MCPs so agents can act and not only advise (gmail, slack, stripe, linear, notion, posthog, supabase)
19. give agents a browser (browser-use, vercel agent-browser, claude in chrome)
20. give agents a computer of their own for the tools with no API (vercel agent-desktop, orgo)
21. give your best skills away in public. it's the new lead magnet
the agents
22. run an orchestrator that never turns off (grok bot, openclaw, hermes). it manages one agent per project, and each of those runs its own sub-agents
23. use coding agents for the building (claude code, codex, cursor, t3, antigravity)
24. wrap them in a harness so they plan, test and remember (ECC, superpowers)
25. manage the whole fleet in one place (paperclip). every agent gets a job, a budget and a log
26. put the skills on triggers (n8n, scheduled tasks). daily, weekly, on every new lead, on every cancellation
27. route your models (openrouter). the expensive one plans and checks (fable 5.2, gemini 4 carbon), the cheap ones do the volume
28. cut the token bill. shorter outputs, smaller tool results (caveman)
29. run a local model for the data that can't leave your machine (gemma 4)
30. track cost per job. if a job costs more than it earns, fix it or kill it
the factory
31. build the line. spec agent → builder agents in parallel (git worktrees) → test agent → reviewer agent → deploy (vercel, railway)
32. design goes through the same line (paper MCP), and a skill strips out anything that looks AI made (unslop)
33. every annoying task in the company becomes an internal tool by the end of the day
34. every customer request becomes a pull request before a human reads it (tstack for the PR workflow)
35. reviewer agents with a named bar. "would the best team in the world ship this?" reject until the answer is yes
36. keep 50 real cases with the right answer. rerun them every time a new model drops and switch to the winner
37. humans approve, own the outcome and talk to customers. that's the 10% you keep
the loop
38. every finished job writes back to the brain. what worked, what failed, what to stop doing
39. support tickets, analytics and social comments flow in every night
40. every friday an agent rewrites the playbooks from the week's results, and you approve the diff
41. every month, rank everything by profit per human hour. kill one thing and double one thing
the growth
42. research with agents before you spend a dollar (perplexity API, last30days skill)
43. outbound runs itself. find the leads, enrich, verify, send (apollo, million verifier, instantly, phantombuster)
44. send prospects finished work in place of a pitch
45. ads run on a loop. an agent writes the creative, publishes through the facebook ads API, kills the losers and scales the winners
46. agents make the creative too. images (nano banana), video (heygen, fal), voice (elevenlabs), music (lyria 3.5, suno)
47. turn every finished job into content. the story, the number, the prompt
48. every customer makes the brain smarter for the next one. that's your network effect
49. once it works, point it at a new niche. same agents, new context
The models are the same for everyone but your context is the only part that's yours.
Acquire as much of it as you can, as fast as you can.
Build loops like this.
This is the whole game, happy playing.
AI vs doctors, in one chart.
Medicare Part B payments across 56 specialties: 24% physical, 21% potentially substitutable by software AI.
Dermatology: 68% hands-on. Diagnostic radiology: 96% nonphysical. Pathology: 99%.
The hands are the moat. https://t.co/N77cYMGFcO
The authors analyze the degree to which clinical specialties are shielded from, or threatened by, AI depending on how much of their work requires in-person physical interaction. Learn more: https://t.co/Htplu16utD
@abigoabi I’d definitely chose the Xiaomi Watch S5 no doubt as it’s really stunning. The looks, the features, the performance and the overall quality is really stunning. All hope 🤞
GIVEAWAY 5/5 🎁 The grand finale!
3 prizes. 3 winners!
Xiaomi Clip, Xiaomi Watch S5 41mm, Xiaomi Smart Band 11
You can only pick ONE. Which would it be?
Follow, repost, and reply with your pick to enter!
Winners announced by end of Nov.